issue_comments

Should I register the custom rank_bm25 SQLite function for every connection, or should I register it against the connection just the first time the user attempts an FTS4 search? I think I'd rather register it only if it is needed.

Here's a demo of that rank query. I had to sort by rank instead of rank desc - need to double-check that:

https://datasette-sqlite-fts4.datasette.io/24ways-fts4?sql=with+original+as+(%0D%0A++++select%0D%0A++++++++rowid%2C%0D%0A++++++++%0D%0A++++from+[articles]%0D%0A)%0D%0Aselect%0D%0A++++original.%2C%0D%0A++++rank_bm25(matchinfo([articles_fts]%2C+%27pcnalx%27))+as+rank%0D%0Afrom%0D%0A++++[original]%0D%0A++++join+[articles_fts]+on+[original].rowid+%3D+[articles_fts].rowid%0D%0Awhere%0D%0A++++[articles_fts]+match+%3Aquery%0D%0Aorder+by%0D%0A++++rank%0D%0Alimit+20&query=jquery+maps

Here's the FTS5 query: sql with original as ( select rowid, * from [global-power-plants] ) select original.*, [global-power-plants_fts].rank as rank from [original] join [global-power-plants_fts] on [original].rowid = [global-power-plants_fts].rowid where [global-power-plants_fts] match :query order by rank desc limit 20 The equivalent using rank_bm25() for FTS4 would be: sql with original as ( select rowid, * from [global-power-plants] ) select original.*, rank_bm25(matchinfo([global-power-plants_fts], 'pcnalx')) as rank from [original] join [global-power-plants_fts] on [original].rowid = [global-power-plants_fts].rowid where [global-power-plants_fts] match :query order by rank desc limit 20

I released a 1.0 (and 1.0.1) version of https://github.com/simonw/sqlite-fts4

Example query (from the tests): sql select c0, c1, rank_bm25(matchinfo(search, 'pcnalx')) as bm25 from search where search match ?

I'd have to copy almost all of the code in https://github.com/simonw/sqlite-fts4/blob/master/sqlite_fts4/init.py so I think I will add it as a dependency instead.