Neutrino Search

code
#1

Ok, this is the first app of the series. I am donating a lot of my code and making it open source.


Neutrino Search can do very fast searches, faster than Fuzzy Search.
Please run the data conversion first, DATA -> Convert CSV to SQLite and TPS.
Classes are documented nearly line by line.
You can use fuzzy search and perform the same search using the neutrino engine and you will be able to see the speed difference.
Also, on the SQLite browse, you can use granular search. Like a QbE but more visual and faster.
MIT License as suggested.
I don’t want any donations. But if you can donate here https://www.gofundme.com/f/in-remembrance-of-brahn-partridge?utm_campaign=p_cf+share-flow-1&utm_medium=copy_link&utm_source=customer it will make you feel good.
#2

thanks very much for sharing your code RAD.

I am very much interested in search and it is always interesting to see other people’s approaches.

I have done a lot of work on optimizing the searching in StringTheory and tend to use that but Clarion also has a built in “instring” function that you can use.

For example your Fast.Find method can be greatly simplified and sped up by avoiding taking any copies of the strings. You are doing a case insensitive search, so something like:

Fast.Find           PROCEDURE(STRING SearchSTR,STRING OnThisSTR,BYTE DebugMSG) 
  code
  if ~searchSTR or ~onThisSTR then return false.

  if instring(upper(clip(left(searchSTR))),upper(onThisSTR),1,1)
    return true
  else
    return false
  end

or you could do something like

if ~searchSTR or ~onThisSTR then return false.

return choose(instring(upper(clip(left(searchSTR))),upper(onThisSTR),1,1))

or even

if searchSTR and onThisSTR and instring(upper(clip(left(searchSTR))),upper(onThisSTR),1,1)
  return true
else
  return false
end

another thing would be to do the upper(clip(left())) on the search string just ONCE before you start searching rather than on each record in the database. And so on… there are always lots of ways to speed things up.

I have only had a brief look as I am heading out soon but hope that helps - and once again thanks for sharing. The interface looks much better than many “boring” standard Clarion apps.

cheers for now

Geoff R

#3

just following on from what I mentioned earlier, a couple more suggestions for your consideration.

where you are doing a search on all the fields (eg. LOC:SearchMode = ‘Everywhere’)

srch &String ! trimmed and uppered search string

on take accepted on

OF ?Neutrino_Search

have:

dispose(srch) 
if LOC:SearchString
  srch &= new string(len(clip(left(LOC:SearchString))))
  srch = upper(clip(left(LOC:SearchString)))
end

then after

ReturnValue = PARENT.ValidateRecord()

have

if not srch &= null and srch
  ReturnValue=Record:Filtered  ! default to no match
  if    instring(srch,SV_:RecordNumber,1,1)    then ReturnValue=Record:Ok
  elsif instring(srch,upper(SV_:ZipCode),1,1)  then ReturnValue=Record:Ok
  elsif instring(.... and so on ....

end

note you only need the upper() where a field can have alpha characters so not needed for numeric fields like recordNumber which is a LONG but is needed for ZipCode which is string(20).

where instead you are searching on specific fields, it would be better to exit as soon as you find one column that doesn’t have a match, rather than doing the check on all the columns then checking if they all matched.

finally I should probably mention that you might want to look at setting up a filter string instead if this was to be used in a client server situation. In that case you want to do as much of the processing on the server (back end) rather than passing all the records over the network to the client for filtering in ValidateRecord.

again, hth

cheers

#4

Good day:

Well, I am aware of INSTRING. And I actually ran my own tests and found my approach to be faster.

I don’t have anything against templates, but often I resort to pure Clarion code.

Now, that is the beauty of Open Source. To be free and able to modify the code, improve it, play with it and test it.

I noted your suggestions, and I shall say, I crossed that path long time ago, I even wanted to use ASM for the string search, but sometimes, it is enough with a mid level approach.

I wanted to have something faster than Fuzzy Search. just that.
I am glad that at least you found something amusing on the UI.

Best regards.

#5

Hum… I just signed up. First thread I see on the list is this. “Me comes here, me reads” your suggestion to this fella about INSTRING. “Me thinks”: WTH… would it really be possible that the guy who produced such code wouldn’t be aware of INSTRING?

Me; nah. I must take it easy on coffee… starting to see things.