normalize rank to be 0..1 from database searches

[ilkka-ollakka]

Description

This gives the local search ranks in scale of 0..1, same what we assume connector confidence to be.

I'm not sure if this is desired change, as it might rule out some results that were previously found, as now the min_confidence most likely has more relevance that is given as filtering criteria.

value 32 is documented in
https://www.postgresql.org/docs/current/textsearch-controls.html#TEXTSEARCH-RANKING

• Related Issue #
• Closes #

What type of Pull Request is this?

• Bug Fix
• Enhancement
• Plumbing / Internals / Dependencies
• Refactor

Does this PR change settings or dependencies, or break something?

• This PR changes or adds default settings, configuration, or .env values
• This PR changes or adds dependencies
• This PR introduces other breaking changes

Details of breaking or configuration changes (if any of above checked)

Documentation

• New or amended documentation will be required if this PR is merged
• I have created a matching pull request in the Documentation repository
• I intend to create a matching pull request in the Documentation repository after this PR is merged

Tests

• My changes do not need new tests
• All tests I have added are passing
• I have written tests but need help to make them pass
• I have not written tests and need help to write them

[ilkka-ollakka]

pytest seems to fail on importer job checks, so most likely timing issue and some other item were also in queue at the same time so index was not the first one.

[ilkka-ollakka]

Seem that I'm unable to reproduce the issue locally :/

[mouse-reeve]

I don't think this issue is related to your code, it seems like an intermittent test failure

[ilkka-ollakka]

I added commit that github action reruns failed tests if any found, just to rule out timing issues in the future, and of course now things don't fail anymore ;)

[ilkka-ollakka]

I can split the workflow commit to separate PR if anyone sees it useful and doesn't want to check yet the normalization things.

[mouse-reeve]

A separate commit would be great, I can also re-run the workflows if that would be helpful

[ilkka-ollakka]

I splitted it up to #3559 and I'll take the commit away from this PR.

[mouse-reeve]

This seems like a sensible change -- my understanding from reading the docs is that it will keep the search ranking from prioritizing long titles/author strings over ones that are closer matches, is that right? I've been trying this locally but haven't figured out any good combinations of books and queries to see the differences. Do you have suggestions?

[ilkka-ollakka]

This mainly scales the match confidence-values to be under 1 always, so really high score (10000 for example) gets confidence of 0.9999 and score of 10 gets confidence of 0.9090... . So it shouldn't change the search ranking/ordering, just scaling the values to known range.

I didn't yet extensively check any good examples, but I can check out if I can spot any examples.

Mainly I did this so the min_confidence would have effect on local searches too, as the queries have the filtering in place.

[mouse-reeve]

I see! That makes more sense -- I fully support normalizing the rank values.