One of the parts of my new devlog post that I really want to stress:
This also introduces the argument that the ranking systems are really only important for underspecified queries (broad queries), so the emphasis on the problems with ranking algorithms in unwarranted. This argument hardly makes sense when the majority of searches that people make are broad. I would also argue that broad searches are most used for *discovering* pages, not for getting to a specific page. However, ranking based on popularity prioritizes what it thinks people would want, which is more suited for specific searches using broad queries, at the expense of discovery of broad topics.
[1/2]
2 years ago · 👍 martin, freezr
@maginalia That largely can depend on how you do keyword grouping and various other sorts of things in the actual query (like AND vs. OR by default). If you are ANDing all of the terms by default, you get more specific results, but worse results, in my experience. · 2 years ago
I just don't recognize this description at all from working on my search engine, which even though it has several orders of magnitude more documents than yours, as well as extracts upwards of a thousand kewyords per document, farily rarely finds itself in the underspecified domain. If you enter more than one or two keywords, you're almost always down to just a handful relevant results. · 2 years ago
Broad discovery using broad topic queries and specific searches using proper-noun queries or very specific queries are both much better ways of dealing with searches without relying on popularity.
[2/2] · 2 years ago