Add a "rank"/"priority" to guides

[yrodiere]

See #11 (comment)

Basically we want the Hibernate ORM guide to be boosted so that it appears before e.g. the Hibernate Search guide, because it's more mainstream.

We could achieve this with a rank feature field in Elasticsearch, computing some kind of "page rank" by determining which guide refers another guide -- or simply by hardcoding a priority for each guide in their metadata.

[yrodiere]

See https://en.wikipedia.org/wiki/PageRank , though a simplified version will probably suffice.

Note in particular:

As of September 24, 2019, all patents associated with PageRank have expired.

[yrodiere]

Here is pseudo code for some pagerank implementation that will probably be more than enough for our use case: https://spark.apache.org/docs/preview/api/java/org/apache/spark/graphx/lib/PageRank.html

[yrodiere]

IMPORTANT: We could probably achieve something "good enough" simply by hardcoding the ranks in guide metadata. We don't necessarily need to implement pagerank.

[yrodiere]

IMPORTANT: We could probably achieve something "good enough" simply by hardcoding the ranks in guide metadata. We don't necessarily need to implement pagerank.

Hey @gsmet would that make sense? This could help us expedite #40 / #305.

[gsmet]

Yeah, I think we should have some sort or rank or priority. I actually wanted to implement the priority idea to improve the index page and put the most relevant guides first.

Now what I'm unsure is that for search, wouldn't you need a rank per topic? Or you think it will be good enough?

I.e. won't you have guides prioritized for a search that is not really relevant to it but still matches?

[yrodiere]

Yeah, I think we should have some sort or rank or priority. I actually wanted to implement the priority idea to improve the index page and put the most relevant guides first.

You mean "most important" or "highest priority", not "most relevant", right?

Now what I'm unsure is that for search, wouldn't you need a rank per topic? Or you think it will be good enough?
I.e. won't you have guides prioritized for a search that is not really relevant to it but still matches?

What I had in mind was a "priority"/"rank" field in each document, and a rank feature query at search time to boost documents by priority.

The impact of ranks, be they manual or pagerank, will definitely be independent from the "natural" score of the query (based on tokens). So there will be cases where we boost a first-class guide and end up giving it a better position in results than another guide that was precisely what the user wanted.

But that's kind of the point of the whole feature: we want to boost some guides irrespective of their "natural" score. Can't have it both ways 🤷 We'll just have to tune the impact of ranks so that it's reasonable and doesn't overshadow the "natural" score.

I'm not sure what you have in mind for "a rank per topic", though... perhaps you have a better idea.

[gsmet]

I'm not sure what you have in mind for "a rank per topic", though... perhaps you have a better idea.

Well, no, I don't have a better idea. I'm just wondering if it will be good enough but... I think we should implement it and see, rather than waiting for a better idea we probably won't have :)