💾 Archived View for dioskouroi.xyz › thread › 29392702 captured on 2021-11-30 at 20:18:30. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
________________________________________________________________________________
The results keep getting "refined" so as to suit the popular 80% of queries, while getting much worse for any technical or obscure queries. Forced synonyms and "people also searched for" are typically useless and almost infuriating. Once you get off the first or second page, the results get _even_ worse-- with pages entirely unrelated to the query (e.g. not even containing the searched phrases). They are probably testing/already implemented some sort of multi armed bandit type optimization like on Youtube's search results where they just show _any_ popular pages (ignoring relevancy) to see if they yield a click.
I've used DDG for the past ~5 years, and it is typically worse without using a hashbang like !so for technical queries. I guess that is what the web has evolved to-- knowing which mega-site you want to search against rather than discover new sites?
Yes, that is also my observation. They will show content that completely does not contain your query words if that content is just popular enough. The result is that niche technical topics get drowned out by related popular discussions.
Also, I am by now 100% sure that Google has just stopped indexing the long tail. Like if I search for function names of public source code that I downloaded from GitHub, Google won't find it. But of course, it's still on GitHub.
Similarly, Google will sometimes not find a single result for some Windows API function names, despite them being publicly documented on docs.microsoft.com.
(same poster) BTW, amazon.com is in my opinion even more infuriating. I just searched for "Odense Marzipan" (which is a 100+ years old brand serving the royal danish court) and they show me pictures of gamepads made out of chocolate along with a note: Your search "odense marzipan" was automatically translated into "odicht marzipan".
Then searching for "odicht" out of curiousity, they auto-correct it to "olight". So I start with almond-based sugar sweets, follow their auto-correct twice and now I'm staring at headlamps. And even Google has no idea what "odicht" might have been, so I really wonder how Amazon decided to auto-correct from an existing product into a non-word.
Searching for "odense marzipan" including the quotes then works, but it yields the cringe-worthy message:
Your search ""odense marzipan"" was automatically translated into "„odense marzipan“".
(where the only difference between the first and the second thing is that they converted the ascii quotes to up and down sentence quotes)
Amazon also really seems to push what they _want_ to sell over what they have available.
A while back I spent a lot of time looking around for a basic SATA bluray drive on Amazon before finally giving up. All I could find was burners for several hundred dollars when I really just needed a drive to quickly rip a single disc I'd bought. I spent probably an hour scrolling through results and trying all sorts of variations on search strings.
I eventually gave up and punched it in Google to get... kicked back to Amazon to a simple, cheap SATA bluray drive that had been there all along.
>Amazon also really seems to push what they _want_ to sell over what they have available
This is exactly it. Amazon isn't a shopping site, they're a corporation using third party sellers to offload the risk and cost of providing a wide array of goods. They let customers experiment with product offerings, find products that sell using their web site, then cherry pick the most lucrative ones to produce and stock to compete with their own "customers".
They're not a service for anyone but themselves.
Amazon's search engine was systematically destroyed from within starting 5 years ago. It's entirely driven by revenue now and not at all by their faux customer obsession tenet.
Just try searching for a high-end appliance and watch all the Chinese knockoffs that will get ranked higher than the actual thing you're looking to buy. Sure, they cost less, therefore you're more likely to buy something therefore such changes tend to win web labs and go into production. But they've completely destroyed the intent of the search doing so.
I just gave up on Amazon Spain about 2 years ago already. It's just completely impossible to filter and rank stuff properly, just loads of crap you don't want. I heard it's worse in Amazon US.
What I don't get about this companies (because this seems a problem shared between google, YT and Amazon) is when they optimize for clicks or whatever their KPI, what are they thinking is the outcome in the mid term?
I mean, IDK around you, but I'm the prosumer regular folks ask for recommendations. They may be safe for the time being but of course I'm going to contribute for their competitors getting klout.
I don't have an alternative for YT, but people watches me using DDG and whe they ask for recommendations for buying stuff I don't even bother with Amazon.
I'm not the type who pushes his decisions onto others, but I already got asked why I don't use google and amazon.
Amazon UK is mildly less crap than this.
I just tried using Amazon.com to order something to Hong Kong for Christmas - sweet Jesus I don't know why anyone would ever visit that site more than once. I ended up giving up and just ordering the things with the co.uk version and sending them myself.
They extra postage cost is nothing compared to the insanity of the .com site.
my favorite is when i see exactly what I want for a millisecond...
only to have it disappear somewhere to page 2 or 3 (if even!)
This seems like an opportunity to build a "better" search engine for Amazon and reap the affiliate revenue.
There's got to be a reason no-one (to my knowledge) has done this: They probably forbid affiliates from doing their own indexing and ranking. Does anyone know for sure?
> They probably forbid affiliates from doing their own indexing and ranking. Does anyone know for sure?
They do. I came across a reddit thread where a guy had built a simple php-based search indexer for Amazon and managed to pay for his college through it. After this incident they apparently put changed their developer terms of service!
Amazon search is horrendous garbage. I was trying to buy replacement ear tips for my earbuds. I knew exactly what to search for. Yet, Amazon search only returned the Large size and none of the others. (Clicking on the large listing only had the large size available. No drop down selection or anything) I go to Google and type in the same search and put Amazon in the search too. Immediately shows up as first result and I was able to buy them. I figured they were out of stock and that’s why I couldn’t get them. Nope - Amazon search is just that bad.
Maybe I am looking upon the past with rose-colored glasses, but I recall a time when shopping on Amazon was like shopping at a huge Target. A lot of well-known name brands mixed with some no-name stuff. It was a nice relief from the limited selection, daytime hours, lines, and out-of-stock errors at department and big-box stores.
Now shopping on Amazon feels like shopping on eBay or Aliexpress with fast shipping. Everything name-brand could be counterfeit, and everything no-name is a complete gamble.
Amazon is absolutely terrible. Try this search, for example:
https://www.amazon.com/s?k=A1990+trackpad+replacement
There is an actual product (
https://www.amazon.com/gp/product/B084GKQB9T
) with all of my keywords in the title. I found it via a google search because I gave up scrolling through Amazon.
I actively avoid shopping Amazon where possible these days, just because it's such a trashy experience.
Thanks for introducing me to Odense marzipan (I have a job in Aarhus!)
I very much get the feeling that amazon and eBay "Don't like" Denmark – the former redirects amazon.dk to amazon.de saying "we deliver!" and the latter doesn't exist (but it does wholly-own a local alternative). Going to Denmark always makes me aware of how good the local Danish things are, and how inflexible and annoying the "global" options usually are by comparison. Riding roughshod over the language is a good example of this.
They "don't like" many small countries. Perhaps based on population or GDP, who the fuck knows.
They just redirect you to the nearest bigger country website and call it a job well done.
Tbf, there are no issues buying from German or French versions. UK was the best choice but now customs is a pain in the ass so...
Just a theory, but "dense" in English translates to "dicht" in German.
Yeah, Amazon search results are often infuriating. Most searches are flooded with similar items that need better filters and some notes on product differentiation. Amazon feels like a bazaar set up inside a Walmart where everyone is yelling at you to buy from them and you don't really know what you're going to get. I know it's a hard problem, but I can often look at the results and imagine some easy UX improvements. Maybe they have higher priorities than helping me buy insulated pants.
I wish shopping for consumer goods was as nice as faceted search on Octopart or Digikey. It seems that even for Amazon-sold products, either the products don't have relevant data listed accurately and consistently, or the faceted search doesn't let you query it.
(And that leaves aside the issue of the product page of a really nice wastebasket I bought years ago being hijacked by a meat slicer.)
I just wish DigiKey let you filter by price, or even sort by price on mobile.
I've searched for a product with a unique name and Amazons can't find it. "Do you need help?" not even "I'm unable to find what you need". Nice.
The product is there. It works adding the company name.
It must be a recent thing as well as I recall being able to get good results from them last year at Xmas when I was searching for some science books for the kids. This year my searches got absolute garbage in return.
"We know what is best for you, consumer!"
well, this particular instance is amazon
Are others getting the same problem? I searched for "Odense Marzipan" on duckduckgo.com, google.com, amazon.com, and amazon.co.uk, and all of them return almond-based sugar sweets in the top links.
Searching on amazon.com for Odense Marzipan works for me and is not automatically translated. No quotes needed.
Amazon.com search is probably the worst offender of all the awful search engines. You can spend hours trying to find something you know is on there.
I'm getting lots of results on Amazon UK, and also Google Shopping. maybe try setting your delivery location?
> Also, I am by now 100% sure that Google has just stopped indexing the long tail.
Rather weird if true, but I can't really disagree with your observation. It seems like large parts of the web have disappeared in the last five to ten years.
Google do most likely index the sites, but their current algorithm just don't use them, because it as much a promotion algorithm at it is search.
I have a public repo on GitHub that just doesn't show up on Google.
Search for "github drv8830pico" - it's literally not in any results.
Haha I just tried and guess what? Your comment was the only result.
Ask HN: Has Google search become quantitatively worse?
› item
4 hours ago — But of course, it's still on GitHub. ... Search for "github drv8830pico" - it's literally not in any results.
DuckDuckGo works fine, though.
Yeah, this comment chain pretty sums up my experience. Google has become AltaVista.
AV respected your querries if I remember correctly.
And now thanks to you, google recommends me to look into DuckDuckGo for that search
:D
It seems like a lot of github.com pages were removed from google’s index about six months ago.
Actually, that might be because of their antispam filters. They were always fighting websites with countless subdirectories or subdomains, having completely wiped them off their SERPs in the past.
GitHub does fit the "huge website with lots of duplicate content" description very well.
Then again, Twitter seems to be doing fine. But I have a suspicion that they have some sort of agreement (and GitHub doesn't), because I have to regularly "report as spam" (note how that implies Google decides what's spam and what isn't) Twitter emails and they still pop up in the inbox after a while.
Maybe, but then you get all the GitHub issue clone sites and all the Stackoverflow clone sites showing up at the top of the results with the original no where to be seen.
It might be anti-spam, but how is it this shit?
I mean, that's probably true, I just made this repo specifically to help people who are looking to control a DRV8830 chip from the raspberry Pico, and I'm not sure how anyone is supposed to find it if there is no combination of "raspberry pi" and "DRV8830" that would get you to that repo on Google.
Well, ads and information are at direct competition, and we all know who's winning the battle. It seems Bing is better for most searches now, even technical ones because they seem to index more of github and stackoverflow than Google.
Man, ads really are the devil.
Yeah. Absolutely infuriating if you specifically use quotes and you get all sorts of dross not containing what you're looking for. Clearly they have figured out in some way that giving the end user what they want does not maximise income.
Google News too has become flaky. Often does not find stuff you know is there, or finds it one day, but not another. Hrmph.
I'm in a similar boat with the "Google Now" feed on Android. When I say "not interested", it fails to note that as expected, and I see the same news "story" pop up (usually different sites, but occasionally the exact same site!). It's likely related to the fact that it almost NEVER gets the subject/topic correct, so I can never use that option.
There was a time I thought I'd trained it to stop telling me about what celebrity said or wore what, but those are creeping back into my feed.
YouTube has started showing me the same videos over and over again! I keep telling it I don't want to see this content because I've seen it before and it keeps forgetting!
I absolutely hate the Google has changed the semantics of search so that it doesn't really pay attention to your keywords unless you quote them. In Google's mind you're not serious about a keyword until you quote it! Garbage!
Google definitely changed something about how they index or prioritize GitHub sometime in the last year. I used to frequently use Google to(successfully) find GitHub repos based. Lately, this does not work for me anymore; adding "GitHub" to my search query helps sometimes but I'm forced to search directly in GitHub many times.
You can test this- find a site you think is not indexed, search for the URL of Google, and then look at the cache and the date. I bet it’s been cached more recently than you expect.
Yes, I actually find Bing is now better for the long tail.
Most of my searches are for really old pages or really long tail stuff and Google just simply doesn't bubble them up, if it has them at all. I keep finding web sites lately from links on other sites and find myself asking "Why the fuck did Google not find this?" .. then I go back to Google and try to find it with keywords from the site, and nothing...
I also recommend switching to Bing. It shows me the results I'm looking for pretty consistently, whereas Google/Duck will sometimes give me Google's home page or the Washington Post's home page instead of anything even resembling the query I'm searching for.
The "people also searched for" box is not just useless, but also very much messing with usability for me. Every time I click on a link, go back, then trying to click on the next link, this box shows up and I accidentally click on that (because it shows up with a little delay and an animation).
This filter takes care of that box completely:
www.google.com##.exp-outline www.google.com##[style="display: block; opacity: 1;"] www.google.com##[data-hveid]>div:style(height: auto !important)
I cannot stress enough how infuriating it is when a page loads content under my mouse cursor at such a delay that I can manage to point at a link and click on it _before_ the new content is loaded so that I click/tap an unexpected link.
This happens _all_ the time on the Twitter app search bar.
This is infuriating on all UIs, but I haven't seen one that would implement a very obvious solution: if the area around click has changed in 100ms before the event, disregard the click. Either a webpage or a browser could do that.
I'm sure I'm not the first one to think of it, so there must be some reason why it isn't done - does anyone know why?
Great idea, although from an engineering POV I think this is really hard. Most websites are super JS-heavy and hence, from a browser's POV, change constantly.
Quite a ton of difference between pixels on the screen changing and the underlying DOM changing.
> Either a webpage or a browser could do that.
The browser should do it. Websites often do this intentionally in order to trick users into clicking ads. The only way to stop these abuses is for the browser itself to stop enabling them.
Unfortunately Google would make the Gmail team very grumpy if they did this - Gmail's by far the worst offender in my experience. Every single time I open my email I have to remind myself to wait for a moment before clicking on my newest emails, because most of the time there's going to be a couple ad "emails" popping in and shuffling everything else down a few seconds after all the real emails have loaded.
FWIW, I've seen that suggestion a few times over the past 2--5 years or so.
You're likely not the first, but it's an excellent suggestion.
Yes, because it sux, in general. Maybe as an option.
If something changes within ~300ms before your click, the only way you'd be intentionally clicking whatever's there now is if you're amped up playing some online game
Now be my guest and define "something changes" please. You will see what I mean.
Gaming is exactly one of the problems and all those sites that lie on spectrum between news to gaming and utilize some aspect of dynamic presentation.
bad enough when it’s accidental, more infuriating when it’s done with a profit motive
theverge.com presents a “show comments” button that shifts out of the way to reveal the worthless taboola links, I honestly don’t know how it isn’t considered click fraud
This is my normal android experience. Use the share function, it displays a list of apps or chat conversations which actually are relevant, tap the app.... annnnnnnd the list refreshes with a new list and congrats, it buffered the tap before the menu refreshes and opens whatever app is now under your finger which is of course the wrong app. So when I use share I have to wait a few seconds to ensure the menu is refreshed and then pick an app. This behavior is throughout most of the ecosystem where input is valid and buffered before things are rendered to screen. Such a well engineered UX.
Android STILL does this? Wow.
Lazy loading is the absolute bane of usability in web UIs. The fact that this is the default/most common behavior in modern web frameworks seems simply idiotic to me.
>This happens _all_ the time on the Twitter app search bar.
Hit the nail on the head there my friend. Super annoying.
I wonder why their search is so slow though. Any ideas?
This is particularly hilarious and ironic when considering they are really punishing for cumulative layout shift in their Lighthouse tool - to prevent exactly what you are describing from happening! Seems like Google hasn't used lighthouse on their own site!
Because google doesn't need to worry about such things. For most people DDG would be a much better experience - if only they knew?
I'm using DDG for almost a year, and I'd say that DDG is slightly better because I have a bit more control. It's important when you're multilingual (common in Europe) to be able to de-localize results chose language etc.
But it isn't MUCH better. DDG is just slightly better than Google, that has become infuriating.
I use it regularly but it is getting to be unreliable for technical searches. I often have to resort to using google when trying to track down information about programming problems.
Agreed but when I say 'most people' I'm thinking of more mundane and everyday queries which DDG handles well enough.
> this box shows up and I accidentally click on that (because it shows up with a little delay and an animation)
And somewhere a team of designers and PMs got their bonus for increasing the engagement OKR. Clearly users love the animation and added delay because look at the metrics skyrocket!
Google now also has an extreme recency bias. Most things I search for will give top results for recent SEO optimized blog posts/Youtube videos instead of established authoritative resources.
If you search for something that happens to overlap with a recent movie name, good luck, it'll drown. And I constantly "have" "to" "search" "like" "this" because Google thinks it knows better than me.
While I also am annoyed how much I need to enquoten to refine search, in most cases I wish it upped the bias further. My default search applies the 'Past year' filter because otherwise I get lots of outdated answers.
I've also completely switched over to DDG and I'm seeing similar things that infuriate me about Google. Most of all the fact that ignores my "literal" searches using double quotes. The documentation [0] says
> Results for exact term [...]. If no results are found, we'll try to show related results.
But very regularly it fails to find results I know exist in not so unpopular places.
[0]
https://help.duckduckgo.com/duckduckgo-help-pages/results/sy...
this always baffles me. If my search terms return zero hits, THAT is helpful and meaningful to know, when they change my search to enable hits, that isn't helpful, because it 99% of the time will not have the information I want, and frequently I won't know that they are ignoring search term, so I'll click through to half a dozen hits before I discover what theyve done. I got to the point that as soo as I clicked a link, the first thing I'd do on the landing page was command-f, and search for the most specific term in my google query. If it wasn't found, I'd instantly hit the back button and do the next hit.
This really got to me about 6 months ago, so I changed all my default searches on all my browsers and mobile to DDG, and haven't looked back. I tried DDG ~5 years ago, and there was no way it could have replaced google for me then, but when I did it 6 months ago, it didn't seem any worse, maybe a little better.
_> The results keep getting "refined" so as to suit the popular 80% of queries, while getting much worse for any technical or obscure queries._
The fundamental, unavoidable problem is that the cost of providing high-quality results on the long-tail of possible searches tends to grow faster than the revenues that can be earned from those increasingly rare, obscure, long-tail searches. Any search service seeking to maximize profit, like Google or DDG, ultimately always evolves to perform less and less well on the long tail of possible searches.
The search service we all wish we could have -- a service seeking to maximize _the quality of individual searches_, no matter how obscure -- may not be feasible as a profit-maximizing business.
_The fundamental, unavoidable problem is that the cost of providing high-quality results on the long-tail of possible searches tends to grow faster than the revenues that can be earned from those increasingly rare, obscure, long-tail searches._
I think even two years ago, Google searches had far more depth and yet Google was quite profitable (then the searches were still biased but now stuff is simply gone). Sure, if someone looked at the marginal profitability of every single search result, it would look like what we're seeing. But there was a time when good indexing of stuff that didn't turn a profit by itself was done as a service to attract people to Google and/or to improve the Internet generally. That time has passed, clearly but it was a decision.
I've thought about it and I think that the really infuriating thing is that somehow the program wrongly _assumes_ that I made a typo, so it's wrong and _telling_ me that I'm wrong. No! It's you!! You're wrong!!!
DDG is heading this wrong direction too. Today I've searched for some ecommerce platform called Comerzzia and it showed me some Comerzia or Comercia or whatever shops near me. It shows maps if it thinks they're related and apparently I can't disable that feature.
For what it’s worth, I have been using the new you.com headed by Richard Socher as my search engine for the last 2 weeks. The condensed search results with sections from reddit, wikipedia, stack overflow or arxiv is really great. It’s really suited for technical users.
I hadn't heard of that one before, but that's actually really nice. I really like how they're trying something new with the presentation and organizing the search hits by the source like that!
Thanks for this, it looks quite usable.
Let's expand the scope too.
Try searching for any product outside your wheelhouse and it quickly devolves into an undergrad research endeavor.
I can't trust the first or second page results because of SEO. Then every page after quickly veers off topic or just features sites that aren't as good at SEO.
I don't know if anyone else has noticed this, but when I also look for technical stuff I see a lot of Stackoverflow copy-cats on the first page.
A lot of sites that just scrape github too end up high in the results. The actual github page isn't even on the first page if anywhere at all. The spam sites do a better job at crawling code than google does.
And a lot of them have stolen content from other sites.
This combined with the fact that programmers have went from highly specific names like "winamp" to just capitalizing a random english word to name their tool makes it very hard to find relevant information.
Word.
> hashbang
Maybe it's just me but those always seem vastly gratuitous. Like shouldn't the engine figure that out automatically? It's like half its job.
The engine should automatically detect that I want to be booted to a third-party site? Err, no. That would be infuriating behaviour if it happened without explicit instruction.
Most of the time they are nothing more than an site:xyz.com equivalent that is just easier to access and you can group them to say something like !dev and search hn, stackoverflow etc in one search without all the blog spam in between.
It's not even just dumbing down. It's heavily weighting to selling one or another thing. It's been getting worse for years but it's really degenerated in the last several months.
You can sort of fight it by including a term showing your topic in your search, I think.
DDG works great for ~90% of my queries. I do fall back on Google every now and then, but DDG honestly delivers results that are just fine.
I often see this line of thought come up whether topics like this float around and wonder: how can we demonstrate /quantify /prove this. Especially since I'm already partial to agreement.
by doing it better?
Frankly, I mostly receive political articles and columns when asking for more more objective things.
I remember once searching for how common same-sex relationships among teenagers are in Japan, as some say it is very common, and all I received were political opinion pieces that did not in any way come with the numbers I sought on _Google_, so I then tried _DuckDuckGo_ and to my amusement what I received with the same query was mostly pornography.
Neither particularly useful, but the contrast in how both prioritize was interesting to me.
What I'm wondering is if the efficiency of the spider itself has dropped. I've often hunted for an old Reddit post I once saw a long time ago and found no hits - maybe I got the wording wrong, but I suspect the real reason is that lower-popularity reddit content is simply not getting indexed at all. Or maybe it's the extreme recency bias others have discussed. Of course, I have no evidence because I can't find the thing that I can't find.
> The results keep getting "refined" so as to suit the popular 80% of queries
Google is simply maximizing profits by giving users results that would cause either more clicks on ads or show more ads. It's mission is to make money this quarter/year. If you believe any of their Silicon Valley-style new age talking points you probably don't have critical thinking skills.
If their products are getting worse for you perhaps you are not part of a profitable segment for them.
Shocking that tight emotional connection is more valuable to most people than throwing spaghetti at the wall.
It’s almost as if the government and big companies have spent a lot of effort understanding human biology, cognitive function and applying what was learned.
While selling the masses on a contrived story keeps them believing there’s a universe of infinite life available to humanity if you just follow these steps…
Things like human colonization of space, and political memes about wealth are taking advantage of the same biological quirks as religion. It’s just now we can quantify the effects rather than wave it off as mysticism.
But the human story is already set on a path of building a bridge to nowhere.
It feels like the algorithm (combined with the SEO/marketing industry) has effectively nuked discussions from organic results.
Based on the search box suggestions I get, it seems many people work around this by appending reddit to their searches. If I search for "warmest winter coat" it's a bunch of untrustworthy content marketing until you try something like "warmest winter coat reddit"
Unfortunately I prefer to avoid reddit (which also has a fair amount of astroturfing), but I haven't found a good alternative. I severely miss Google's old "discussions" (or was it forums?) filter.
What's kind of funny is there was a time where Google used to stand for --literally anything-- and would punish a site like reddit on search results for having such a user hostile interface.
For some reason though (probably because they used AMP) they basically allow them to do anything they want. Multiple popups, hijacking click events for login modals, and hiding the content with no impact to search results.
So now, in all the glory of the Internet, the person who genuinely wrote the best blog post on the "warmest winter coat" is completely unfindable on normal Google search or you force yourself via a reddit query to a completely hostile user experience unless you login.
It would be cool to just be able to do something like this "warmest winter coat --hobbyist-only".
GP was saying how reddit is actually more useful than whatever Google returns today: that's the "hobbyist-only" shortcut for today. Because that blog post is long gone from Google's indexes, since replaced with SEO-optimized or "influencer" driven sales/content marketing.
I like to compare with search.marginalia.nu results from time to time, but the restrictions it puts on the content it traverses do not make for a good daily driver.
We are saying the same thing. I reiterated it poorly.
Just also poking fun of how Google both killed the "real" blog industry and forced all the good content on reddit (which ironically does every anti-SEO thing ever and where people are trying to go).
Because most people who use Reddit on a semi-regular basis use their old desktop UI (old.reddit.com), which doesn’t have the dark patterns found on their newer / mobile site. It’s a much more usable site for the few communities I still have left on Reddit. It’s also much better supported by the individual subreddits via style sheets (most didn’t bother updating layouts when Reddit introduced a new UI because all the power users who would have updated them kept using the old interface).
GPT3 got your back:
warmest winter coat --hobbyist-only
Top 10:
1. Canada Goose Parka
2. Patagonia Down Jacket
3. Marmot Precip Jacket
4. Columbia Winter Jacket
5. North Face Thermoball Jacket
6. The North Face Nuptse Jacket
7. Rab Neutrino Endurance Jacket
8. Mountain Hardwear Ghost Whisperer Jacket
9. Black Diamond Fineline Hoodie
10. Outdoor Research Cathode Hoody
That's actually hilarious - that list isn't bad!
I suspect part of what is going on is that Google, and its employees, have forgot what Google was supposed to be. In some warped, misguided attempt to not be evil, they got confused and tried to be neutral.
Google should be opinionated. It should have a huge bias towards quality. It should not be hard for a small army of employees to be blackholing ANY crap product roundup site. Real product tests, where multiple items were actually purchased and compared, should always float to the top.
Just as much as needing to pay attention to what spam to suppress, they should be asking "what do we we want at the top" and whitelisting really great sources that always cut in front. Why should healthline ever appear before examine.com?
Instead, they have thrown their hands up, said the algorithm is in charge, and to interfere with it would be improper. Bollocks.
--hobbyist-only makes sense
when i search for something specific, i usually include a random niche tangental hyper specific keyword about the thing i want in quotes (until it gets turned into the SEO-buzzword of the day)
"impedance" for analog electronics stuff, "ring-spun" for clothes stuff, etc
This is a reply to myself... but does anyone have ideas about how this can go away? Have we lost the internet to eternal marketing and data-mining arms races? Is this why people are flocking to search-opaque places like Discord?
Sometimes I feel like that internet isn't for me anymore, and that's a little distressing.
The more I think about it the more I just don’t think it’s possible to have an internet that is supported by ad-revenue and isn’t user-hostile. The incentives are diametrically opposed.
We need a PBS/NPR of search engines.
I wonder if an argument could be made that this would actually pay for itself. Given that productive use of search is at odds with commercial use of search, maybe a publicly funded search engine could increase GWP by 0.01% which would be about $87B a year.
Yeah the internet has been turned into cable TV. You're paying for access, but the objective of 90% of the content is to get you to watch ads.
I think so, actually. New environments are refreshing until power structures form - and that's what happened with the internet.
But it is still not so dire. I went back to bookmarks, reading lists and keeping note of writers I check out. It's not bad at all as long as I keep in my interest bubble. Google or not, I still would prefer today's internet world to the decades before the internet.
We need a new protocol that will not allow that, such as gemini, which is not that good IMO and severely limited (you can't even add pics ffs).
And of course, new search engine, something distributed and in the GNU domain.
You can link to images in gemtext. It's up to the browser whether to display them inline.
Terrible UX. What if I just want to give links without wanting to have them as images?
Besides, all that can be done with img HTML tag (browsers do have 'disable images' thing already).
This is the consequence of arranging a society around infinite financial corporate growth; everything is optimized to squeeze out every dollar for the balance sheet.
Ironically we’re watching this play out now with products and techs that market themselves as “decentralized.” Maybe after this phase the tech community will consider this isn’t something we can tech our way out of.
The Eternal September always wins.
I get it, I almost included eternal september in my own post, but this feels like a reductive way to overlook an actual problem.
profit optimized is the inevitable result of any system that has profit motivation
banning advertising would help for awhile, but other profit streams would be optimised and expand to fill void, such as data collection/mining
I have thought about one way of changing the rules: banning large compannies from showing ads. I am not sure I understand the fallout this would cause, but:
- ad revenue would drammatically decrease
- buying ads would actually be affordable by local sellers
- the market would self regulate against monopolies
If ads would be banned, data collection would be unprofitable.
Recently my Google suggested search completions has started suggesting that I add "reddit" to a lot of my searches.
I can only presume that so many people have given up on the web as indexed by Google and are just searching for "<whatever> reddit" now as the only way to get any kind of content written by real people on a subject instead of SEOd filler "content".
Presumably it won't be long before Reddit itself is flooded with spam content to take advantage of this - I'm sure it's already happening to high value keywords.
That's probably my default search query for now to avoid all the marketing crap. But if this becomes the norm we will definitely see robots discussing their winter coat preferences.
You already need to be a little careful. I can't remember the specific search where I ran into it... but if you follow some user accounts recommending a specific brand you'll find that they have very shallow post histories.
This happens on Amazon reviews all the time as well.
What I end up doing is trying to find a post that isn't all-in on any specific solution... but lists pros and cons of multiple options, because it seems less likely that a content advertiser will post anything negative (or positive about a competitor).
There's been a steady increase in astroturfing and blogspam over the years on reddit to subs like /r/frugalmalefashion where small indie shops are able to pump and dumb objectively low quality wares. The mod team does the best they can but they can only do so much as the effort to surface good content costs way more then the effort to artificially boost bad content.
In a few years Reddit will be just like the generic google searches for everything but the most heroicly moderated or obscurely small subreddits.
Sadly this no longer works either. Reddit is now doing some SEO work to mark stuff as a later date than posted.
You can search "warmest winter coat site:reddit.com" and filter by past year, only to get a result from 7 years ago.
I really don't know how to search the web anymore.
Sometimes it feels the less SEO-optimized content there is for a topic, the better the search results are. Privately I de-googled a while ago (exceptions are youtube and youtube music, but then I don't have spotify). Now using DDG, and I don't miss anything. Google is better for picture search by proposing similar ones when clicking on one.
When I searched it "warmest winter coats", I get:
first result is a list from a blog by some "Emergency prep guy" it basically lists 27 coats with information.
Second result is RT online with black Friday recommendations
Third is oprah daily with recommendations and shop links
So, reddit is also, as you said, full of false information + astroturfing as well. Besides not everyone is interested in diving into reddit rabbithole to find information on warm coats.
What do we want google to do? It tries to blend whatever is available, I don't think google got worse on this particularly, but it is probably a hopeless pursuit considering the status of the web. As for forums, adding "forum" at the end sees to work, but I agree it would be nice to have the option in the toolbox.
> first result is a list from a blog by some "Emergency prep guy" it basically lists 27 coats with information.
Most of the time, this is just a list of coats someone googled and copy-pasted info from the marketing pages. This page is an affiliate-marketing site masquerading as a review site.
Not sure if specifically that page is, but that's what the majority of
"product review" results in Google are nowadays.
I know these review pages are usually affiliate links, but the problem is, when the subject is something like this (commercial interest), it is extremely hard to find anything that is objective.
Emergency prep guy seems like at least owns or knows about the products he lists, however he also discloses his site is owned by a company which is a participant in the Amazon Services LLC Associates Program.
So maybe Google should stop feeding this affiliate spam loop and try to create algorithms that try to actually understand which information is based on personal experiences and not created by financial interest. This is a very hard task to do and also prone to manipulation as well. In the end maybe Google needs some real competition for this to happen.
> So maybe Google should stop feeding this affiliate spam loop and try to create algorithms that try to actually understand which information is based on personal experiences and not created by financial interest
As you say, this is harder to do well than it sounds. There are definitely a lot of hobbyist types who review things and make a side (or even major) income from affiliate sales. With human judgment, it's quite easy to differentiate the honest hobbyist from the spam sites. I fear that any automated attempt to punish the spam sites would end up hurting the hobbyists more.
I include 'forums' in my search, or something like 'site:coattalk.com' if I know of one
It's also funny how Google basically nuked groups and made it unsearchable, while once in a blue moon you get a search result to alt.coats.winter or something
I don't get the appending reddit thing. I don't see why reddit is considered trustworthy, marketers have been planting "organic discussion" on reddit for _years_.
Yeah, I can't confirm it, but the weirdest I think I've seen is in r/science about research that was funded by the meat industry saying that a diet with meat is healthier than vegetarianism or veganism.
The study was poorly done and there were tons of comments pushing the same message: "vegetarians/vegans are annoying hipsters who will lecture you for eating meat and they'll be so deservedly upset by this."
Found it and most of those comments are deleted now (
). Is the meat industry losing a sizable chunk of profits to more people swearing off meat for moral reasons, or ditching meat as a financial decision?
Edit: Threw that link into a website that restores deleted comments (
https://www.reveddit.com/v/science/comments/qskxol/meat_cons...
).
Mods deleted all references to fact that the study was funded by a beef company. Blatant corruption?
But our uncanny valley detector is pretty good at sensing it imo and the upvote system helps and authentic people will comment or disagree.
SEO blogs are full uncanny valley for me.
In my experience reddit comments are less likely to be marketing than Google results are. Still not entirely trustworthy, absolutely.
The trick is to find an active community where they know their stuff and will downvote/delete spam. Unfortunately those needs to be extremist for their hobby and so they will look down on perfectly fine for normal use low budget things.
Gotta use "site:reddit.com"
If I'm asking google or DDG for advice on a product, it's either going to be a reddit or Wirecutter for me. 99% of results on "best *" results in _*literally hundreds*_ of domains like "best*for2021.com" "buybest*.com" "top10*reviews.com" that are all generated by bots containing only the worst knockoff / counterfeit / Chinesium products and tons and tons of Amazon affiliate links.
E.g. I was trying to remember the name of a top-of-the-line soldering station brand (Metcal) I used back in college, so I kept trying permutations of "best professional soldering rework station" on google [0] and DDG [1]but it only comes up with low-end Chinese stations, a few mentions of Weller and Hakko, but no impartial reviewers, no forums or blogs, no discussions...nothing leading to Metcal.
Then I searched "best professional soldering rework station site:reddit.com" [2], I clocked the first 3 links, scrolled, and found Metcal on the second hit. [3]
I was surprised to see Wirecutter did a review [4], and arguably the Hakko FX-888D is the best soldering station ever made (and the X-Tronic is a fine budget runner-up) for *_MOST_* people, but it's still not a Metcal (the thermal capacity and regulation of their iron tips is just unparalleled even with nice Wellers and Hakkos - you can really feel the difference when working with THICC power ground planes and RF connectors).
[0]
https://www.google.com/search?hl=en&q=best%20soldering%20rew...
[1]
https://duckduckgo.com/?t=ffab&q=best+soldering+rework+stati...
[2]
https://duckduckgo.com/?t=ffab&q=best+professional+soldering...
[3]
https://www.reddit.com/r/electronics/comments/2c4hnl/best_so...
[4]
https://www.nytimes.com/wirecutter/reviews/best-soldering-ir...
You can set that once on you.com and then it will keep that preference and show you reddit results whenever relevant.
WireCutter has really gone downhill since they were bought out imo.
The other thing I hate is the flood of reposts and blogspam around any announcement. Company 1 will give an interview to outlet A. Blogspam outlets B,C,D,E.... will publish "articles" that don't even link to the original but will highlight several out of context quotes, add minimal commentary that reads like it was generated by OpenAI, and hit publish. Then google appears to have a difficult time deciphering what the authoritative source should be.
Damn I have been doing this exact thing to get "more organic" results in recent months and didn't even think about it, but it's pretty obvious now that there has been some major exclusion when it comes to results.
"Thread" is a good keyword to include to find discussions, even though it's a bit overloaded.
Eg.
'good cheap mountain bike' -> 10 results out of the first 10 are commercial spam and listicles.
'thread good cheap mountain bike' -> 5 human discussions on entry-level MBs, 1 link to a MB forum home page (not a specific thread), 2 commercial spam, 1 paywalled magazine article testing MBs, and 1 online shop product page for a MB that happened to mention a "73mm Threaded BB shell" multiple times.
Until the spammers figure that one out. It would be nice if google had a keyword for this.
Like discussion?
https://www.ghacks.net/2014/01/23/search-discussions-blogs-p...
I don't use them enough, but the replacements are interesting: they seem to mostly search for indicators of a forum (e.g the existence of a next page button).
For me, Discord is the new reddit.
When I want information about a product, I join the discord forum associated with that hobby and I ask for recommendations.
Since Discord is a chat service like IRC, I get replies from humans instead of shady astro-turfed websites.
This is a decent solution, but not at scale. A big reason to even have all previous discussion archived is searchability and information availability. If the question has been answered it shouldn't be hard to find it. Having to have a human do the work of explaining something _every time_ someone on the internet wonders about it is infeasible.
I use the reddit suffix for topic expansion. It helps to observe people discussing a topic and discover its facets that way as opposed to potentially getting a one sided treatment in a reference starting point like wikipedia. After I get a lay of the land, that's when I start down Wikipedia sources.
Isn't this just a defect in the corpus? There is likely no quantitative, objective information about the warmth of winter coats in articles that would include the phrase "warmest winter coat" because someone who undertook an objective analysis would be uncomfortable using that phrase, which has no real meaning. With your refinement, adding "reddit" to the end of the search, you just get a bunch of randos holding forth unscientifically and no real information. You've just gratified some bias of yours.
On the other hand, a different search for "R-values of winter coats" produces a few real gems, like
https://outdoorcrunch.com/jackets/
The first results used to be blog posts, forum discussions, etc... I'm not expecting scientific accuracy (it's a bonus if I find it)... I'm expecting personal investigations and opinions. I'm ok with that. I'm not ok with content marketing supplanting it. The entire front page of a search is often some "blogs" listing products with affiliate links and no actual experience. The information is outrageously shallow and the search results as listed try to hide this.
> "R-values of winter coats"
This is a valid alternative... but I don't want to be an expert on winter coats to be able to Google basic information. I'd have to weed through a fair amount of marketing content to even find that the phrase "r-value" exists. In the past this wasn't necessary.
The internet used to be primarily a place for people to connect and share information... and now it feels like primarily a place to be advertised to. There's also the fact that many ads have evolved beyond simple billboards to psychological manipulative clickbait.
It's completely anecdotal and tangential to this topic, but I have a sneaking suspicion that the way marketing manipulates people has created unhealthy amounts of skepticism that further fuels the affinity for conspiracy theories... which tend to be so toxic that they're almost inoculated from marketing.
Social media killed forums so there's no surprise that they don't surface in search results since they are no longer active. The discussions you're looking for are now happening pretty much exclusively on Facebook and Reddit and only Reddit is well indexed.
Specifically for search, my experience today when I search something on google:
- captcha (because vpn)
- spam results (based on location, my location was never very good for technical content)
- paywalls
- no pictures cause photos is now paid (i've signed up for unlimited forever)
And on bing basically I get shopping coupons, games and, well, and microsoft's "anything's valid, except customer sat" approach.
There's a whole class of searches that no longer have value because of SEO. Try "best exercise bike" or anything that can be similarly monetized and you will of course end up at a well-crafted page designed to monetize you.
My deepest apologies for saying this, but for any type of query that has a monetization angle, I now add "site:www.reddit.com" to the query to find actual discussion about it.
Normal Reddit disclaimers apply as much of what you find is garbage but at least if you search "best exercise bike" confined to reddit you'll get real opinion not hellbent on monetizing you.
It's staggering how much the results for any product query that isn't from Reddit or a site like Wirecutter is just unusable. I was recently looking for thin winter gloves and literally every page was an SEO referral scam.
Yeah, it's been happening for so long that it's easy to forget it wasn't always this way.
I really don't get what people are talking about, first three results for best exercise bikes:
https://www.tomsguide.com/best-picks/best-exercise-bikes
https://www.cyclingweekly.com/news/best-exercise-bikes-40742...
https://www.menshealth.com/fitness/g23064646/best-exercise-b...
Not sure if they are amazing results but it's decent for such a generic query.
These are all monetized results. Brands send bikes to these places to review. It's an advertisement. If you want an organic result like a personal blog post from someone or a niche interest forum like you might have found on google search 15 years ago or so, it's just not really there. Everything you see has been paid for to be seen by you. Once you realize that, a lot of the web feel like this uncanny potemkin internet because that's all it really is these days.
You're not going to find people doing unpaid work reviewing the best exercise bikes, or other expensive items. Very few people can afford to go out and buy 10-20 exercise bikes, and fewer still people can afford to do so every time a new bike is released. And yet fewer will then go and write up their experiences and keep their list updated over time.
For stuff like this you will only get spec comparison articles or comparison articles from big sites who get gear sent to them or make so much from affiliate sales that they can afford to rent/buy gear. It's simply not likely for anyone to put so much time and effort into it for a passion project.
However, where you can still find independent reviews is if you look up a specific item. Granted, they may still put an affiliate link in, because why not try to make some money when you've put in the work of making a good review, but at least they'll have bought the gear themselves and are using it because it's what they like best, and can speak to the pros and cons of the equipment in detail.
So the big lists are still useful - check out a couple "top 10" lists, read the specs, and get an idea of what's suitable. Then look up the promising candidates individually, and find more "real" content about them. I find this usually gives results which are better than just reading Reddit discussions, at least for higher value items.
Four years old now, but here's a good long read on how an entire industry's review sites may be acquired or strongarmed into being invisible shills:
https://www.fastcompany.com/3065928/sleepopolis-casper-blogg...
You know, I cleaned out my grandparent's old house recently and came across a Schwin exercise bike from probably the 70's. It has some sort of speedometer/dynomometer thing, a resistance adjustment, and a mechanical timer. You know what? I got up on it and went for 10 minutes and my heart was beating faster and my legs felt like they were getting a workout. What more do you need in a bike? Especially now - I had my phone to listen to a podcast and watch to monitor heartrate if I'd like - what else does a bike need to do?
I use this so often that I have it bound to a keyboard text replacement on all my devices. Recently Google has started inserting non-reddit posts to the top of my results even when I include this. Weirdly I’d be less upset if they were ads, but they aren’t. Just some articles that Google thought I wanted to see more than the website I specified.
It’s infuriating.
Do you know what infuriates me most about this? It's that Google hasn't addressed it when it's clear as fucking day that their search results have gone in the tank.
Most of those "real opinions" are not real opinions.
I've been on both sides of paid reviews on Reddit.
I still do the same thing because in some subs you'll get actual conversation in the comments, but it's definitely being manipulated.
You mean you've been paid to put fake comments about products on Reddit?
There was a very good thread on HN about this not long ago(1). Google search is getting worse because it is letting companies like Pinterest game it.
Instead of fixing the spam they are instead encouraging companies to spend more and more time on SEO and coming up with their own shenanigans like better ranking for using AMP (defunct now).
People who generally make great content (think a researcher or a great software maker) can't compete with billion dollar companies like Canva, Shutterstock and Pinterest who spend millions of dollars on SEO and have dedicated SEO employees who spend all day sending outreach emails and doing experiments. Henceforth the good content never even sees the light of the day; drowned by all this "SEO" optimized content.
FWIW i still believe it's the job of the search engine to find great relevant content and show it to the user instead of the other way round. Though I know it's much easier said than done.
(1)
https://news.ycombinator.com/item?id=25538586
It's not even just gaming or priority. As others mention, it just straight up ignores substantial search terms or interprets them only to sell mainstream crap. It will return nothing rather than obscure terms that are present on the web.
I don't understand this narrative that Google is making the search results get worse or "letting" companies game it. Search result ranking is adversarial! If the results are worse than they once were, that just means that Google is less far ahead in the arms race than they used to be
An alternative explanation is that Google stopped caring about UX.
The problem isn’t solvable by modifying search queries. The fundamental issue is that the web has filled up with content designed to trick Google into sending it visitors and to maximize ad revenue. So you have content that is thick with ranking signals and thin on useful and new data. You also have to scroll past many ads and filler to get to the answer to your search query.
The fundamental problem is that Google and the SEO spammer’s interests are aligned. Google is both the search provider and ad network. I think this makes Google tremendously vulnerable to competitors who don’t have that conflict of interest, and presents a massive opportunity to those with enough courage and cash.
There has been a cat and mouse game between google and scammers for a long time. Results get scammy and Google updates to get rid of them. I just hope they update soon to get rid of the latest version. I'm sick of so many results for things like "how to fix a clogged sink" all being long pages with stuff like "first let's talk about what a clog is" then "how they get clogged" and then "history of sinks" and then after you have scrolled and scrolled and scrolled you get the answer.
This pretty much sums it up. The amount of crap and plagiarized content in most searches is what is killing most search results.
Google makes money regardless of the quality or 'originality' of the content your search comes up with so they currently have no motive to change things.
Agree with you. A lot of news publications tend to post about anything hoping searchers would land on their page for the answer. I google times for Football games, and the first link I get (in India) is from a news website, which does not even have the answer. I got so frustrated, I switched my region to US.
The topic here says "quantitatively". Does anyone have any actual statistics or quantitative data on the quality of Google search results?
All of these threads devolve into anecdotes and reminisces about the "good old days" and complaining about Pinterest. None of which is in the least quantitative.
I'd be interested to see some actual data or research on the subject, if it exists.
Or maybe it's not Google that's gotten worse but the web itself? Again, quantitative results, please, not anecdata.
> Does anyone have any actual statistics or quantitative data on the quality of Google search results?
Google has. They use this data expertly to improve search. Common sense and technological advancement tells us that, quantitatively, Google search has become better year over year, for all their relevant metrics/cost functions.
And likely, exactly because it has become better for all its users in aggregate, it has to become a bit worse for a certain group of power users. There, we can only rely on anecdotes and personal experience, but these tell us it actually has gotten worse.
Similarly, the web can become both worse and better. The really useful articles today are better researched, multi-modal, solid web of links, internet-first. Spam has also evolved. And "top 10 ways to do X"-McContent outranks better articles, because that is what the majority of Google users wants to see and clicks on. They truly have a better experience, while others' experiences suffer. It depends on what you measure.
> Common sense and technological advancement tells us that, quantitatively, Google search has become better year over year, for all their relevant metrics/cost functions.
lmfao. so you're telling me "quantitatively" that google search results have gotten better, without citing any data at all, but with an appeal to common sense and "technological advancement"?
what if i told you that search is an adversarial problem, and that it's possible for google's tech to be getting better slower than the aggregate tech power used to game google search is getting better? is this not a patently obvious possibility? it's not some kind of gotcha impossibility for google's tech to get much worse over time, even if they weren't hamstringing themselves by lots and lots of user-hostile changes which benefit google's interests rather than their users.
> lmfao. so you're telling me "quantitatively" that google search results have gotten better, without citing any data at all, but with an appeal to common sense and "technological advancement"?
Yes. If that sounds so unacceptable or strange to you, I suggest you try it. Works really well when reasoning about unavailable data, or researching a field with slow peer-review process.
> what if i told you that search is an adversarial problem
Then I get an adversarial reaction and a downvote from you.
> it's possible for google's tech to be getting better slower than the aggregate tech power used to game google search is getting better? is this not a patently obvious possibility?
Yes, that's plausible. Should be measurable quantitively too. Can you cite some data on this? :)
We could compare to the available data on the quality of (HTML) e-mail spam filtering over the years, which all have kept up. Like pg said: Spam is solved, when skilled spammers start creating content which does not look, feel, or talk like spam. So webcontent-spammers still are on the first 2 pages of Google, but with content not classifiable as spam/content farm.
> which benefit google's interests rather than their users.
One of Google's interest is their user. But perhaps not the type of user you are. Studies have shown the value that the tech of Google is delivering its users per year. This value was in the thousandths, and this value has risen. Meanwhile, Google makes about tens of dollars per user per year, less for technical users which don't click ads or block these.
> it's not some kind of gotcha impossibility for google's tech to get much worse over time
It really is, no way to mince it. Google search is funded by Google ad tech. Google ad tech has improved ML by a ton. To say Google tech is getting worse, is to totally overlook deep learning revolution, word2vec, transformers, BERT, etc. etc. etc. To state that, is to reveal the truth that you are ignorant of major technological advances in the past decade, only looking at the issue from the viewpoint of a single atypical Google-search user. What would you even do with quantitive SEQ data?
Do you really think it is possible that Google runs an implementation test of a new ranking model, and deploys it, while all measurements, human labeling, and user tests show it is doing worse? Of course not! Only if you think you are smarter, could you think that Google search changed and has gotten worse.
If Google does ML like the rest of the industry, all model changes move up their designed levers, or these changes are not committed. So if Google was unable to improve search, then Google would have looked exactly like 2008 Google. The fact that it does not, shows either you or Google is wrong. If I had to make a bet...
I'm not buying the tribal argument, that I'm the wrong user for Google. The results are shite across the board regardless of tribe.
I think they've accepted that SEO has killed past ranking algorithms and are rebuilding rank from the ground up using ML with the entire internet as guinea pig. All of the crap results we're weeding through now is grist for the ML mill.
I am literally typing exact phrases for content I know is there and not getting the results I should.
I think they've cut the cord with past algorithms, not an incremental update, a major one.
My thinking so doesn't make it so, however, so just my two cents.
> Yes, that's plausible. Should be measurable quantitively too. Can you cite some data on this? :)
It's a bit bloody rich for you to come out with that attitude when your entire point revolves around your opinion of 'common sense'
> Do you really think it is possible that Google runs an implementation test of a new ranking model, and deploys it, while all measurements, human labeling, and user tests show it is doing worse? Of course not! Only if you think you are smarter, could you think that Google search changed and has gotten worse.
You seem to be confusing the concepts of 'Google have made their tech more profitable' and 'Google have made their search capability better'.
Of all things, I am sure that this discussion thread did not add value to this community conversation, and as such, serves as spam. Apologies, and let's hope Google bot finds ways to ignore low-information content in a threaded forum. Maybe they could even locate the exact post which caused the derail, and apply some authority penalty on its author.
No, I used common sense, but I actually have the quantitative data that poster was asking about. Right now, I am doing exact keyword matches and trying to find myself. Will get back to you when my analysis is done.
> You seem to be confusing the concepts of 'Google have made their tech more profitable' and 'Google have made their search capability better'.
No, you are confused. Try to Google the article I was talking about. It talks of perceived value to the user. What value would you lose without access to Google maps, search, youtube, gmail, etc.?
Google made their tech more valuable. That sounds like an improvement to me.
> Google have made their search capability better
If you want an explanation for this obvious statement (and not be demanded to ask in return how capitalism works), I suggest you first try to code a simple search engine. I think a 100-line Python script with some imports would do. Only this would make talking about capabilities possible.
> So webcontent-spammers still are on the first 2 pages of Google, but with content not classifiable as spam/content farm.
The exact clones of StackOverflow and Wikipedia are all over the first two pages of Google.
This presumes that the metrics they optimize for are intended to represent usefulness to actual users and not, say, ad revenue. Even if they do intend to optimize for usefulness, this doesn't mean that they have metrics that accurately represent that.
I also think you're underestimating average users. Anecdotally I've heard my parents complain repeatedly about the incoherent, auto-generated, affiliate link spam that plagues product searches.
> This presumes that the metrics they optimize for are intended to represent usefulness to actual users and not, say, ad revenue. Even if they do intend to optimize for usefulness, this doesn't mean that they have metrics that accurately represent that.
They have multiple levers, of which user search quality is a big set. There is always trade-offs and a balance that must be found, which aligns with company vision and strategy. Having these levers allows business-decision makers to direct focus top-down (on a certain set of users, on producing great ad numbers, etc.).
It is clearly hard and important to design these levels and find the right balance, given a rapidly changing company and user-base. So a lot of expertise and power is invested to measure the right things, and to find the right balance (an incorrect/risky balance should also be adjustable with other levers).
So for me: either Google is trying really hard, but essentially failing. Or they have the best of the world, with all the right context, designing these levers. While hard and sometimes wrong, I do not expect to contribute anything which may improve their lever settings. If someone does know, Google would like to hire them.
So while true, that accurately measuring things with proxies, is really hard, and sometimes done wrong at companies. I do not think Google gets this wrong, or at least, gets this to be the best of breed. If their metrics still cause long-term search engine quality loss, would show them to not know what they are doing. I think they do know very well, better than me at least.
I would agree too that the balance of levers right now is in line with Google's strong market position. Search engine quality could take a small hit, if justified with extra adsense income. But when search engine quality noticeably start going down, then _all_ other metrics will suffer. You should have teams with sole focus on improving quality. Other teams will have to realize that favoring their lever over the search-engine-quality lever _must_ lead to worse outcomes for Google in general.
About product searches, I myself was not able to do this satisfactory 15 years back. It improved. But need to stop viewing things as a single lever, a single metric. To say search has become "worse" in general, is to exactly fall into the trap of not accurately measuring and losing too much nuance/details for competing objectives.
I don't think this is necessarily true. Your standard desktop computer has gotten much easier for a casual user to navigate over time, but not any less robust for power users. This is because powerful tools for customization and building are still exposed to power users. Google has slowly stripped away many of these tools. One has to wonder why, and my guess would be that it's because they would expose either the unethical ways Google deals with results, or the failure of its search model.
To be able to quantitatively compare Google-of-old with Google-of-today on the web content today, we'd have to have access to both. And we'd have to assume that Google-of-old's algorithms have not been figured-out and abused by the content that's there today. And certainly the content has changed: I wonder if you can look into archive.org by-date and run a search engine on it for comparison (I'd like to test if proportion of commercial content has grown up).
Without access to both Googles, the best you can do is compare across different search engines: special-cased ones like search.marginalia.nu can net you a quantitative feel for what exists out there that's less likely to be content marketing, but I am not sure if you can figure out where those pages rank in Google search results for the same terms programmatically?
You can also prepare for the future: record some data today, and compare in 10 years time.
"Quantitative" doesn't mean "empirical". I read the question to mean has google search returned _fewer_ results. Not fewer good results, but fewer results full stop. Perhaps the asker made a typo and meant "qualitatively".
It'd be cool to have a tool that runs locally on my Firefox history and builds metrics like how often I have to refine my search query.
My thoughts exactly, until we have actual dat but not "feelings" this whole argument is moot.
Everytime one of these threads show up, I find myself wondering why someone doesn't launch a technical and research focused competitor. There seems to be such broad agreement that Google is now terrible for research that there's for sure a market for it.
Edit: I'd love to see a some-of-the-web search engine like this. Start just with university sites, prepress archives, quality forums, public dev Slacks, etc.
I did just that. Well not specifically aimed at research, but the types of websites I'm interested in, written by humans, blogs and such. It's got some quality issues right now because I fucked up some of the keyword extraction logic and it's taken a while to identify just what's gone wrong.
Even though it's a bit broken, it has some lucid moments. Just compare:
https://www.google.com/search?q=mechanical+keyboards
https://search.marginalia.nu/search?query=mechanical+keyboar...
The take-away I want to drive home is that it's absolutely possible to build something the scale of Google c.a. 2003 and run it on consumer hardware. I think, due to general difficulties in making these things profitable, the ideal approach is to make the operation so absurdly cheap it can be run non-profit instead. I'll gladly pay out of my own pocket to have a good search alternative.
I love Marginalia, it's such a great project. You get really weird, but interesting results. Honestly I don't use it as much as I'd like, but it's still fun to go down a rabbit hole and read actual content.
It's probably quite far off being viable as a daily driver search engine, but maybe it doesn't need to be either. I built it to pick up the slack in the areas where the big search engines are floundering, and as long as it's doing that, I think that's quite sufficient.
I want a curated search engine so badly. I feel like some enterprising developer could craft one specifically geared towards developers. Imagine a search that doesn't ignore operators like '?.' and where you can set persistent conditions on every single query without having to type it out. This would help prevent you from needing to type in the language you are searching for in every query.
Oh, and the cherry on top is completely abandoning the idea of Natural Language Processing. Go right back to keywords only.
I've considered taking a stab at it, but I didn't for likely the same reason no one else has: no idea how to monetize it. Sure you can run ads, but your conversion rates will likely stink because you'd be selecting for non-suckers.
Not trying to spam but will mention devonagent once more in this thread, they just sell it, old fashioned perpetual license. respects boolean operators.
https://www.devontechnologies.com/apps/devonagent#editions
Thanks for the recommendation. Just trying it out and it is much better than doing a direct search on google
You might be able to monetize through closer partnerships with companies like Microsoft, Atlassian, Amazon, etc.. I'm envisioning an ad system that is basically geared towards selling their cloud and DevOps services. Because the engine is geared towards programming queries, you might also entice these companies by showing that developers have an easier time troubleshooting their systems. Perhaps, eventually, the engine could be bought out and act like a loss-leader or funnel to services. I know Microsoft is trying to be very developer friendly.
Just some random thoughts. Monetization is a huge bootstrapping challenge for something like this.
I’d love to take a stab at it. I think there are ways to monetize it that don’t involve advertising (at least enough to cover salaries and infrastructure). The only reason I haven’t is I don’t have access to the initial capital.
Subscriptions. You would have to take money directly.
It can be solved without centralized server: every content site, page, or paper, will publish Bloom filter for their content discoverable via sitemap, to which you subscribe via RSS/Atom. When you need to search, you will make hashes for words in your query, then will check each Bloom filter for potential matches. When potential match is found, full page source can be downloaded and checked more carefully for ranking.
Perhaps technical users are more likely to be running an ad blocker and/or are overall less likely to actually click on ads. If you're going to have 1/10th the CPM by making this search engine vs. one geared towards everyone, why would you spend the extra time and CPU cycles powering it? Maybe a subscription model would work but you'd have to show just how better it is since you're competing against good-enough free search engines.
Content curation doesn't scale, so I believe it's out of question for the modern web. The best you can hope for is to have someone throw some quality AI at it, plus a lot of manual tweaking.
Given that the goal is only a small subsection of topics, might scalability not be as critical to the success of the project? If the engine is specifically crawling official programming language documentation, help forums like StackOverflow, and select sites/pages (think university professor with resources about how to implement xyz), might that be enough to not necessitate scaling even farther?
The fact that DMOZ closed, after 19 years, was rough. It was the original 'starting point' of the Google search engine, back in the day being in DMOZ meant your page rank was high.
There is definitely plenty of new spaces for a community effort that is similar, or at least hand-held with help from an algorithm.
Not only that, we just need one that works the way Google used to work. You know, something like the old algorithm that made Google famous and made competitors obsolete.
The hard part of what Google does is spam and abuse prevention. If you only indexed known-good sources I think the hard part would basically be done. Providing good search results, rich queries, sentiment analysis etc is all old hat by now.
"Known-good" sources have a serious habit of getting owned by malware and replaced by organized crime schemes. Anti-abuse can't be handled by whitelists, it needs to be an online system.
Garbage sources have a serious habit of getting taken over. It's not that common with quality sites, but also not impossible. A simple downvoting system would be good enough.
The problem is that even quality sites die. And when they do, their domain gets poached and suddenly it's not a quality site anymore. But according to the ranking algorithms it's the same site, since its quality is measured by how many links it gets.
Coopting quality sites is a bit of a cottage industry. There is also a significant amount of hacked wordpress sites. Their domain may be quite reputable, but they're unknowingly host to a ton of spam.
Old algorithm with todays internet content would be utterly garbage.
what you want is the internet with only a couple hundred million people connected, only a million or so content creators.
Unless you have some pretty evil supervillain scheme, no moral compass, and succeed.... that world is never coming back.
I think that if the company doing the search results wasn't also the company getting money from SEO spammers posting ads on their websites then the search results might be better.
That old algorithm would just hand you a bunch of SEO spam.
I am not sure. It may be me lacking in understanding.
Google already does that though...
Yes but the Google algorithm circa 2003 would be much, much worse. There has been an arms race for the last 20 years.
The spooks will take over that one, too.
I've pondered this, too. Why not a DMOZ 2.0 type concept, where you seed the index with high quality sites and grow from there. Freshness won't matter as much for most topics, so crawling X times per day probably won't be necessary. Maybe have user defined flags to indicate the types of results they want (large sites, small sites, high authority, technical sites-- maybe different sort by features) with backlinks/domain prominence as a small factor, but also using NLP to determine authoritativeness of said content so as to facilitate less-linked site discovery.
I think something that needs to be actively maintained like DMOZ is bound to burn out. I've been thinking about how you could passively maintain an index of good resources.
Imagine a service that provides you with a personal search engine in exchange for a list of your bookmarks. Those bookmarks provide the signal for what sites to index for the public search engine.
Neeva is worth looking at. I recently switched to it exclusively for a week and found it as good as Google for 90% of what I do. (FWIW, previous attempts doing this with Bing or DuckDuckGo last about a day before I get so mad I switch back.)
Right now Neeva seems very good at navigational queries, which I do 90% of the time. It's still not as good at deep research queries for obscure things. Probably related: Neeva is relying on Bing for a big chunk of their queries. But they are building their own index.
Neeva has the right business model (search as a product) which helps align interests. On the other hand some of their reasoning around the business execution left much to be desired:
https://www.reddit.com/r/vivaldibrowser/comments/pol41p/comm...
Just tried this, even signed up. Immediately they started force feeding me "news". Looks like they added me to some (daily!) mailing list. It looks like more of the same, but this time you pay for it.
Yeah, I tried Neeva for a few days, but it was slowing me down.
This site can help:
I just searched 'what is energy' and is see posts from Quora and Medium on the first page. Definitely not what someone doing real research would be looking for
DevonAgent may do what you want, it felt very thorough and powerful but I never got in the habit of using it, there’s a learning curve to it
https://www.devontechnologies.com/apps/devonagent#editions
mac os only. 5$ or 50$ for automation+archival
It's analogous to predatory pricing. Google has all the necessary tech to go back to doing good searches. They can easily pivot back in a few months.
A new startup would need to dump loads of money into servers and building their own tech.
So there's just no way to recoup the cost of building a competitor.
You might be surprised how much compute it takes.
Not the same thing but, entire SO website runs on like 2 door sized racks. Search engine might be a different thing but if you have funding to get started, hardware costs aren’t going to be impossibly huge. Most is labor (engineering).
I’m curious, how much compute power it takes to index the whole web? I presume queries are super fast.
The monthly snapshots of CommonCrawl contains around 64k files of HTML, metadata and text only. The text only and metadata are around 250MB each, so in total the corpus would be around 16TB of text and 16TB of metadata. I don't know how much it costs to them, but when I used my own crawler I was able to download 500k webpages in around 4 hours (using the $80/month basic DigitalOcean server... let's say a total of $1/1M web pages, just the crawling process).
IMHO, indexing the whole web is an antifeature for technical and research topics. But there are projects like CommonCrawl that crawl a good portion of the web and give it away free.
what's the business model? Those verticals are probably not great for search ads. Do you think there is a large enough population of people willing to pay $X per month for a better technical + research search engine?
> Everytime one of these threads show up, I find myself wondering why someone doesn't launch a technical and research focused competitor. There seems to be such broad agreement that Google is now terrible for research that there's for sure a market for it.
Your bubble seems to agree, but the lack of serious competition, even in niches, is a sign that outside the HN bubble Google is in fact not seen as any worse.
Seems like for a decade several times a week this question shows up on HN and 99% of comments agree. Personally for my uses I don't remember it being significantly better, I'm not sure what I'm missing.
Not really. It's a sign that you can't profitably compete with Google.
- There is way more content to sift through, including video.
- There are way more Google users, including grandmas.
- Conversations have moved from discussion boards to walled gardens and chats.
- Google relies more on neural network embeddings, so does a better job when you type full sentences and semantic similarity.
- Google relies on authority signals and incoming links to a website, so non-commercial, hobbyist, or controversial content ranks way lower.
- Websites rely on Google for income, so they start producing what Google and its readers want to see.
- Spammers rely on Google for income, so those surviving after decades, have created massively successful linking rings and spam production pipelines looking at keyword search statistics.
- You were really good at Google searching years ago, having a harder time updating and letting go of what worked for you. Easier to blame Google for this.
As for tips: Anything academic, search on specific websites or Google Scholar. Anything technical/coding, search on StackOverflow. Anything cultural/commercial you want a peer answer, instead of a salesman answer, search on Reddit. Try to join like-minded communities where you can ask expert questions, and research new things in your field. Exact keyword match still works by enclosing keyword in double quotes:
"sal dulu antasma"
>Anything academic, search on specific websites or Google Scholar. Anything technical/coding, search on StackOverflow. Anything cultural/commercial you want a peer answer, instead of a salesman answer, search on Reddit
This is a completely miserable experience, and walls off useful information into classes of people who "are in the know" about where the most relevant information exists.
And if you're that grandma searching for a birthday present for your grandson? Good luck. She's likely to be devoured by ads, if not an outright scam.
They asked for searching tips, not how to solve the problem of internet search. I have a few ideas for that too though.
Agreed on the miserable experience. Do you have any ideas on how to attack this? Perhaps Google started out with the right experience, but ads eventually toppled it. Perhaps Google never hit on the right experience. What gives?
>Anything academic, search on specific websites or Google Scholar. Anything technical/coding, search on StackOverflow. Anything cultural/commercial you want a peer answer, instead of a salesman answer, search on Reddit. Try to join like-minded communities where you can ask expert questions, and research new things in your field
This is much more like what Ye Olde Webbe was like. Sites competed to build communities that were repositories of information. Things like Reddit tried to build a generic silo so that they could silo information there, which I think is a bad thing long-term.
The biggest problem, as I see it, is sites just give up on doing their own search. Not surprising, as search is a hard problem, but it plays merry hell with the democratization of the Internet to foist the problem off onto Big Corporation Inc. to do the heavy lifting.
A related problem is that many sites simply don't have what could be called a "webmaster" anymore. Everything is contracted out, or part of a subscription service, or otherwise disconnected from the owner of the site having full control. If you're a small business that sells locally produced products, you're never going to appear in Google or Amazon searches, even if you have an Amazon store. You can't afford a full-time webmaster just for your site, and all of the various platforms, like Wordpress/Shopify/etc, deal in such volume that these small businesses will be largely ignored.
The ISV model for products like AutoCAD is possibly a good route. A team of well-versed engineers and designers can build things, but you need a direct customer representative to get at the juicy meat of what the end-user needs. Apply this sort of model to search, and you can aggregate over larger swathes of customers.
> Anything technical/coding, search on StackOverflow. Anything cultural/commercial you want a peer answer, instead of a salesman answer, search on Reddit.
Do you find this better? In my experience it’s nicer to just put stackoverflow, reddit, or (often, in my case) seriouseats in my google query. Reddit search in particular is pretty miserable.
Exact keyword match does not work as well as it once did. I question whether it's working at all on YouTube.
Yandex is the most entertaining search engine. Not always the best results but it reminds me of Google from 2003. Just look at these insane results, not a single DMCA takedown or SEO spam result to be found
https://yandex.com/search/?text=free+movies
Damn! This search gave me the fun. I felt again like in 2003 when searching for free movies give me an actual movie rather than "WHERE I CAN WATCH THIS MOVIE?"-site
It's giving me better results than Google. I'll try it for some time. Thanks.
Way, way worse. There was a time you could force Google to only return pages with a specific term by wrapping it in double quotes (or further back, prefixing with a plus). This was useful when trying to learn about a specific error message/code. Now, it's a total crapshoot, and you may not get a single useful result at all.
For many searches like that I use 'tools->all results->verbatim'. Seems to help.
I've been using the verbatim mode by default for a couple years. It does improve the results substantially.
Same here. However, comparing what verbatim mode was before and is now, it has gone much worse.
I just want barebones keywords search, with just the caveat to index just the words on the main part of a web page, not the surrounding ads, etc.
Interesting, is it any different than putting the query in quotes?
Google may ignore the quotes entirely, although it's not clear when.
Came here to say this. I have noticed this increasingly in the last few weeks, constantly getting 1st page results with the exact-match term struck through and semi-relevant-but-not-really results. Struggling to think of motivation for why this would be. Or is it a byproduct of shifts elsewhere.
There's clearly a new form of highly ranking SEO "blogspam" that I'm now seeing in the top results for many searches.
It's a long article, with multiple headings and short paragraphs for each. Your search terms will be a near-match for one of the headings. The problem is that heading is 3/4 of the way down the page, so you have to scroll past pages of introductory paragraphs and basic info about whatever you're searching for.
For example, for the search term:
how long does icloud photo upload take
Included in the top results are the pages:
-
https://9to5mac.com/2018/12/31/upload-icloud-photos-iphone-i...
-
https://www.blog.motifphotos.com/using-icloud-for-your-photo...
-
https://backlightblog.com/how-to-upload-photos-to-icloud-fro...
Which all fit the mold of blog post I'm describing. General, multi-heading pages about a particular topic that rank highly for a specific contained within.
The content, once you get to it, isn't necessarily bad. But what I want from the results is a single page / blog post with _only the specific heading and paragraph_. Or even better, the best Stack Exchange question and/or Reddit thread that fits the term.
Yes, it seems to come up on HN every week.
I like to append "reddit" to many queries, for example "best bicycle for under $500 reddit", where you can read some interesting discussions, rather than a random SEO website with affiliate links.
That's what I do too, but one should be aware that marketers are gaming reddit.
I recently tried looking in the Buy It For Life subreddit for a specific type of product, and it appeared to me that one company was taking advantage of that sub to promote their product.
That's where multiple threads across multiple subs comes in handy. If you look at one thread in one sub, then more than likely you will find astroturf.
But if you look in multiple threads and do some research, following up with searches related to the item in question, then read the comments, you should be good.
For example: Pony clamps were brought back by another company, but are now being made in China. There isn't anything inherently wrong with that, but I was concerned the quality of the new product would be much less than the old, made in the USA clamps. So, I searched for the specific clamp style from Pony, read about it, then followed up with searches related to ancillary brands that were talked about in the threads. There were some out-and-out product endorsements from the new company, and assurances that the new clamps would be quality. There were also 'natural' comments that sounded suspiciously similar to what the company owner was saying. . . .
I finally settled on a company that is made in Canada.
Man, great anecdote I discovered yesterday: So "Cyrus" and "Joseph" are part of my grandfathers full name.
If you Duck Duck Go his full name, no quotes, the front page is all him and relatives; ancestry dot com and whatever.
If you Google it, no quotes? The front page is ENTIRELY "Miley Cyrus on Joe Rogan." Moreover, with quotes? Only 2 links total on the front page.
So I'd say YES.
I typed "Korean traditional architecture" into YouTube and got a link to a Mr Beast video.
Google products are being strangled so as to make them utterly useless, but useful to Google's modus operandi.
(1) It's a long term trend and (2) It's no accident.
If Google Search was perfect you'd never have a reason to click on an ad.
If you email me, I will send an invite to Kagi Search beta which deals with the problems you mentioned, namely:
- A ranking algorithm that penalizes commercial and ad/tracking bloated pages
- Ability to "mute" or "prefer" domains
- Ability to search for discussions
- Customize type and appearance of search results
Kagi Search is a new startup in the search space and we are currently in the closed beta.
Signed up on kagi.com.
Google is a recommendation engine, not search. It recommends what you will click on, not necessarily what you want.
Perfectly stated.
For me it has, not probably Google's fault but especially coding related searches usually give me a lot of sites that just copies content from Stack Overflow. It's amazing that Google can't filter and punish those sites in the rank.
Absolutely. Google definitely seems to favor consumer stuff and the most general query result possible. It's near impossible to find specific information about a certain product or item without wading through tons of shopping sites and empty review sites. And now Google removes nearly all of the specific search terms, where I normally have to click "must include" or preempt it by wrapping every word in quotes. Sometimes, I ask myself what's the point if I search for something because Google unrefines the search by removing words to return the most results in terms of quantity and generality.
Google seems to also have a heavy recency bias.
I've tried to search for news articles that were even a month old and have had trouble locating links to stories I know happened. I don't know any tricks to working around this. And if I don't recall specifically when the article came out (say, somewhere around 6 months ago), I'll usually give up.
Any tips on finding "historical" results?
I agree. I hate that any search I do will show CNN articles from this week. Dude google, if I wanted recent news I would use the news tab.
Search tools -> Any time -> Custom range
Nice. Last I looked into this, I couldn't select a custom range, it was limited to set times like, last 6 months or something. Thanks
I've gotten so pissed off about this that I compulsively save to disk every interesting article I find.
search web history that Google has,
Use a bookmark tool such as diigo.
Yes, recently looking for "stackoverflow like" stuff has in the first results clones of SO mixed with the original source.
And they speak us about the "algorithm".
IME Bing does a much better job of filtering out SO and GH clones.
Those things are a curse. But Google had a long history of killing that kind of junk off eventually. See 'spun content' for example.
I've found that double-quotes are basically mandatory in any search now. (this is Google shorthand for "results MUST include this term", you probably know that.)
I might be looking for some sort of broad topic, like _how do a I do a particular thing in SwiftUI_, and it will return a bunch of stuff about SwiftUI that sort of skirts the topic. A search for _do "a particular thing" "SwiftUI"_ is more likely to succeed.
This is what everyone in this thread seems to be missing. Google is exactly as useful to me as it was 25 years ago, in fact much more so in fact by way of several orders of magnitude more information being available on the web.
About ten years ago I had to switch to consistently using double quotes for technical searches to raise the signal-to-noise ratio, but there are a huge number of contributing factors to that. I think we discount how much the rise of search engines as a discovery mechanism hurt the original PageRank algorithm. Things used to be easy to search for because humans were self-categorizing all the knowledge available (because we didn't have good search engines), and PageRank just made that self-categorization globally available.
Now there's so much more noise on the web, and no one is manually organizing it like we did with the old "web rings", and that's not even beginning to explore the issue of sites manually gaming SEO. The algorithm's job is much harder today, and so we need to give it a little help finding what we want.
> This is what everyone in this thread seems to be missing. Google is exactly as useful to me as it was 25 years ago, in fact much more so in fact by way of several orders of magnitude more information being available on the web.
> About ten years ago I had to switch to consistently using double quotes for technical searches to raise the signal-to-noise ratio, ...
I remember double quotes being infinitely more useful 10 years ago than they are today and much more strict (i.e. good for technical searches). You could put actual code syntax into double quotes and it would return meaningful results containing that syntax. Nowadays there is no way I'm aware of to force google to respect every single character within double quotes — it's much more fuzzy now and frequently characters like `!;.,/\|{}[]` are ignored/dropped from inside quotes and the quality of (technical) results suffers. It's the 80/20 optimization problem discussed elsewhere in this thread...
This is a decent point, I never search google for literal code samples
For me, even double quotes get synonymized/get unrelated, but similar results
Me too. Some search results are just filled with junk not matter how you qualify your search
I can't prove it, but if bet anything the reason Google is getting worse is the same reason everything gets worse: engagement. They are likey optimizing results not for relevance but for engagement more and more.
Not just that but bias, they intend to promote certain content and demote others and that introduction of human bias essentially dillutes the effectiveness of the algorithm; you don't get what you are searching for per-se but someones opinion of what you should get.
Personal anecdote from yesterday.
I was running into an issue with a docker container that I was trying to install on my NAS. I searched using just the actual error, one word, and the name of the container/service, again one word. So two words in total: “FakeUserAgentError caliber-web”
DDG gives great results, where the first result is GitHub issues related to what I’m seeing and so on.
!g gives me first two links as some crap websites that have scraped the above GitHub issues, then the GitHub issue mentioned above and some JP/CN websites which are supposedly about the error but I can’t make sense. Utterly useless results.
This is the first time I’ve seen this, never before. DDG results were miles better than Google results. Note: I generally use DDG so not sure if this is recent development or not.
I think that another reason not mentioned here for missing on organic results, is that many such results have moved from forums to closed garden facebook groups (and of course facebook's own search is even worse, possibly to encourage people to ""engage"" rather than find what they're looking for).
I think I might be stuck in some old habits of googling and I've lost touch with modern google.
I'm with you on that 100%. When did we arrive at the point where relevant keywords don't return relevant results? Is training an AI on natural language processing more important than keeping your flagship product reliable? And the blog spam...
I think google has gotten _qualitatively_ worse. It's no longer a search engine, it is a recommendation engine. I flat out don't use it anymore for anything other than addresses. This wouldn't be so bad if most of the major competitors weren't just metasearch engines relying on google. Bing is no better.
It's gotten to be what it was like in 1999, you can't rely on a single engine for anything. So "default search" in browsers doesn't serve as useful a purpose anymore. I find myself using an assortment of engines, including Brave (IMO currently providing the best experience, although that doesn't say much right now) and obscure engines like gigablast.
I have stopped exclusively using Google search for 95/100 queries. Duckduckgo does the job and sometimes I bang to Google.
Yeah, exactly what I did. I find that duckduckgo is much more similar to the google of the early 2000s much more than the current google.
It doesn't work too well for ambiguous keywords or vague questions, but you can always add '!g' for those queries.
I'm reading _The Age of AI_ whose authors include Eric Schmidt. There is a part where they mention that in 2015 Google moved from "human developed" search algos to "Machine Learning". I don't know what that means in practical terms, but when I saw it a light went on for why their search had become so bad.
I'd just add that I think it's bad in the sense that many others have mentioned- lots of crap at the top, ads, very hard to find more obscure stuff. Otoh, I generally still find it better than bing / ddg for most mainstream searches I would do.
There are so many clones of Stack Exchange.
Or how GeeksForGeeks shows up higher than the pages for the official Python docs.
Or how searching for anything seemes to find an auto-generated page with that phrase that outranks any useful info.
The authentic source of the content should be the first hit, not someone talking about it, or a clone, or an SEO page linking to the authentic content. That is a colossal failure.
It's actually surprising that for all Googles talents, they can't tell that site X is just cloning StackExchange or that official documentation should probably rank higher than some random site. Either they can't or they don't care.
These sites are easily detected as clones. How Google reacts is part of the adversarial game theory.
For StackExchange clones, their tactic seems to be to push them to a secondary index. Hellban them, but keep them visible to the creators. You never start over, and try again with smarter duplicate evasion. You just see your site wither to insignificance, with sometimes a temporary bump to confuse you/annoy you/keep you uncertain about which changes helped. But sometimes this strategy can make it seem Google can't detect this, especially when using very specific keywords only found on StackExchange, there just may not be a better 18 pages than a duplicate page with a different "related questions" section.
Official documentation underranking some random site is nearly always a temporary anomaly (or makes some sense, in the case of very verbose documentation, like W3C docs). If structural, nudge Google along with some reports. If malicious, these are the sites that Google is likely to completely nuke. All authority and investment gone. Any part of the spammer's link network contributing to artificial authority also exposed. Makes more economic sense to pick softer targets and spend more energy on staying hidden/not overdoing it, never sure of the threshold.
How about a company providing user agents which can undertake searches on one’s behalf (e.g. to Google, DDG, etc) and filter those results? There may not be a need to reinvent search but instead to filter search results. Such agents may preemptively download result pages and use machine learning and other users feedback to assess their relevance. It could also curate content more than ad blockers and Readability by, for example, hiding less relevant content (e.g. all recipes seem to follow some SEO-optimized format of posing pages of gibberish before presenting the actual recipe). This sounds like a browser or browser extension coupled with servers to maintain the relevance models. This may be coupled with a VPN so pages could be downloaded as that user and analyzed in the cloud and sent to the user only if/when needed.
I do think it's a bit of a mixed bag of causes.
Google has mixed incentives since their main income is the ads they sell both on the search results page and in the pages linked from the search results. Looking after the interests of the users is a tricky balancing act, and it would be really easy for them to start cannibalizing themselves to eke out a bigger profit margin. In the absence of serious competition to keep them honest, it is in their economic interest to show mediocre search results with a painful amount of ads.
Google also appears to have pivoted toward a few particular use cases of search engines, finding where to buy things, and answering questions. I think a few uses fall in the cracks between those patterns.
They seem to be optimizing for the results that most people will click, which is fine if you are most people, but will worsen the experience for the long tail consisting of all the users looking for something specific off the beaten track. The algorithms seem to actively resist attempts at refining the query away from the most popular results, which is a peculiar behavior that I don't understand the motivation behind. I think this is a big mistake, as it makes what was already easy easier to find, and what's already difficult even more difficult to find.
I can't tell you quantitatively
But googling programming language problems, there's way more blogspam. A lot of crappy websites that seem auto-generated often rank higher than useful stackoverflow results. There spammy sites often have my search verbatem in the title, so they get clicked on...
It's pretty sad as IMO this is harming developers, blogging, and Google. I hope Google can fix the issue and reward useful content, not just content that's made itself look useful to generate clicks.
Yes, Google has become worse because the problem is now much harder. I believe that Google's time is coming, because the fundamental idea of a single centralized global search engine won't work anymore in a world that produces petabytes of new content every second. PageRank (or any other ranking algorithm) cannot compete with the massive economic incentives of SEO spammers and content writers in every niche and every language on the planet. Machine learning is even _more_ susceptible to adversarial input.
The time has come for a distributed, decentralized search engine powered by the immensely powerful devices in our pockets, with rankings determined by our peers and local network, not by a single global ranking. I'm not saying a distributed search engine with localized results will replace Google (you still want the global standardized search results in some cases), but rather we need a better alternative to simply asking your friends/communities like Reddit. The combined spare computing power of every smartphone on the planet is probably greater than AWS or Google Cloud could ever be. If people were incentivized to sell their spare battery/CPU/network capacity for web crawling, perhaps by earning money, this would solve the advertising/privacy problem with Google's business model as well.
Yes, feels like websites that consist of 3 sentences of actual information spreed across 5 paragraphs are winning.
Also, I don't like how google tries to demote controversial "wrong think".
Why doesn't google show stuff like
or
https://www.stroustrup.com/bs_faq2.html
higher?
I think it would be great if in the settings you could upload e.g. some python code that modifies some parameters. I would e.g. write something like
change_score("pinterest.com", -7)
change_score("based.cooking", +8)
change_score("stroustrup.com", +4)
Than you could share your code on e.g. hackernews. That way you could actually punish clickbait.
Yes, since a long time:
https://news.ycombinator.com/item?id=22107823
.
Any pro tips on how to google
Pro tip: try DuckDuckGo,
.
Google's search quality has definitely gotten worse, but DuckDuckGo is still worse than Google. I have DuckDuckGo as my default search engine but occasionally will Google something that DuckDuckGo can't find and Google has it on the first page.
I have this exact same usage pattern. I try to use Firefox and DDG exclusively but when a site borks or the search results are unusable in DDG, it's back into the Google ecosystem with me.
Yeah, ditto for search.brave.com. It's fine but sometimes Google is just more creative or accurate in what it returns.
ddg is bing. So why would I bother with it?
It's not "just Bing", but that also doesn't matter if people are happy with the search results. Honestly Bing has gotten pretty good, and nobody really noticed.
It comes up every time DDG is mentioned that it's "Just Bing". It could be made using an early beta version of AltaVista and rejected NetBSD patches for all I care. The results are pretty good regardless of how they're produced.
1. It hides personal information from Bing.
2. DDG has its own search crawler (and it's getting larger, the more people use it).
You can also try SearX.
Main search is powered by bing, lets not kid ourselves. ddg is not answering complex queries by using its own crawled index.
Yes, not yet. I hope it will, when it has enough resources.
Google has gone downhill since they started pushing searches towards approved sources.
Sadly your best bet when google is failing is to try Bing or other search engines.
Perhaps unrelated, but generally when companies establish monopolies and are valued primarily on their profits rather than their user growth their incentive to build the best possible product will diminish rapidly. For the mega-cap tech companies today their incentives are to add friction to prevent user migration and to aggressively monotonise their existing user base.
This is why today sites like Reddit mostly focused on adding app nag messages to their site and appending posts with distracting badges for additional revenue. It's also why companies like Google don't care about your thoughts when it comes to removing dislikes from YouTube and continue to promote paid search results and their own search content every year.
Usually when the product becomes so bad that people forgot why they use it in the first place a competitor will come along with a user-first approach and disrupt the market, but this is a process that is likely to take decades rather than years or months. In the meantime you can expect Google and other tech monopolies to continue making their products worse from a user's perspective.
I find myself using a mix of search engines these days. Google is still my primary when working (code-related searches still preform better imo), but typically use DuckDuckGo on non-work devices and very occasionally I'll use Yandex when I'm searching for things which might be influenced by political bias.
In the case of this specific question though I'm not sure whether Google consciously changed their search algorithm in a way they knew would result in poorer results. What it might suggest is that Google isn't putting as much resource into optimising the quality of their search results as they have previously. I know in recent years I've found myself increasingly using the "site:" modifier to return results from specific high-quality forums and sites which I trust. I find the vast majority of results for general queries these days link to ad-riddled, low-quality content farms.
What do you mean by "worse" - it's hard to assess something defined in a qualitative manor quantitatively.
For me its the batting average of "do I find what I'm looking for before giving up". I don't think my queries have gotten more esoteric over time and yet I've been striking out more and more in the last 3 years. Maybe I should start a personal log and actually quantify this so I can post it here in 5 years.
I find all of Google products are just constantly iterated upon for the sake of keeping staff busy and just seem to get worse as time goes on.
Google News used to be an easy to read html list that just worked. Now if you leave it open in a tab for more than 5mins you get a warning that it needs to be reloaded because of a software update.
Similarly, Google maps has begun giving me really bad routes and I live only 30 minutes outside a major tech city.
It coincided with the little green leaf that promises to route me on 'eco friendly routes'. Don't see how sitting in the car longer can be more eco friendly
Yeah, the eco mode is really annoying. I don't understand why they added that, especially now that the types of cars we are driving are increasing in diversity. What's eco for a big truck might not be the same as a small EV.
Oh my god there's an eco mode for driving now? That's hilarious... and scary.
The goals of Google have changed. Or rather, they haven't.
At one time, the premise was that good results would bring people back. Getting the exact right answer every time was the go-to means to get customers to return, and see more Ads. More ads meant more income, which meant a successful company.
But now, Google has the dominant market power. Customers are going to Google by default, never considering an alternative. Everyone who wants to have their results above the fold have to pay to play- because everything above the fold is a paid ad. And below the fold, sure, we can have something the algorithm dug up for you too. But that isn't the point anymore.
Google is a business. They don't optimize for accuracy, they optimize for money. It used to be that accuracy made money, so they did that. But that was never the end goal, and it's become far less important.
People used to call that kind of stuff “being evil”
Haha yes! From pagerank to the "attention direction engine." Google's starting to resemble that fictional company in the Silicon Valley show, Hooli.
My take: Github Copilot and similar products will replace many google queries in the near future, for code, but also for other specific domains. When I type a query, I'm essentially trying to summarize my context, and I can't compete with an advanced deep learning model that reduces my code to a 300-dimensional context vector. The same is true for a lawyer writing a legal document or a scientist writing an academic paper. Very curious to see the developments in the next few years.
Yeah, but Google does nearly the same thing with search personalization and context sensitive ranking.
Almost all personalized information retrieval systems are some variant of this type of recipe since the late 90s no matter what mathematical model they use.
Perhaps the problem for Google is that there is strong incentive for the content providers to play a cat and mouse game (aka SEO) with google. Depending on business model, anyone sufficiently popular in this space will probably have similar problems once companies have to either pay for content or revenue share. Github is a situation where people (open source developers) "donated" their code to Microsoft, but most of the world is not that egalitarian.
Would it be bad if we made advertising... illegal? In small part because it would reduce the principal-agent disparity between what you want to do and what Google wants you to do, but more importantly because it would remove the incentive to create blogspam and game the results.
Advertising serves a positive purpose by informing people about products that they will like and otherwise would not find. It also serves a negative purpose by attempting to cause people to make decisions which are not in their best interests.
I am genuinely curious what proportion of ads seen serve each of those two purposes.
I want to be informed as good and as neutral as possible.
Ad spaces are sold to the highest bidder.
Query: "Jeans"
Result: Lots of "Best quality/value jeans for 2021" SEO optimised pages with Amazon/affiliate links and not a lot of useful information.
Search engines seem to have become entirely online retail focussed. Even things like searching a location is now productised in "Top 10 hotels in location for 2021".
My tips: use also the image search to quickly identify promising pages (charts, tables for example).
Use the tools dropdown to specify the time period, for example last week or last 24h. If your WhatsApp started crashing today, relevant content is likely fresh.
When searching for articles, try to write the search like journalist would write the headline.
Use the double quotes aggressively to filter out unwanted pages (by hinting you want a certain phrase).
I don’t think I usually go past the first page. Instead I usually refine the search.
Can’t say about the quality. I still keep finding stuff and I don’t really have any way to measure how it was year or 5 ago.
When it comes to reaching actual content that isn't just an entry in an API doc somewhere, Google is basically just a vehicle for me to get to StackOverflow/Reddit at this point. For almost every search I've done without appending "reddit", the first page is full of bullshit zero-effort listicles. Like the other day I wanted to see what web frameworks Go has to offer, but when I search "golang web frameworks" I get a million links like "Top 10 Best Go Web Framework 2021!!" that are filled with obviously scraped or auto-generated content.
I get that SEO is kind of a race-to-the-bottom scenario, but I always wonder - how are these random sites in business? I understand that they get a good amount of traffic from said SEO and "advertising" pays good money for that traffic, but do advertisers really not grasp that people spend all of five seconds on these pages before going somewhere else?
Well, by one measure, the quesiton's been asked on HN for at least the past 12 years:
"google search quality"
https://hn.algolia.com/?q=google+search+quality
"google search worse"
https://hn.algolia.com/?q=google+search+worse
Notably, from 12 years ago:
https://news.ycombinator.com/item?id=902999
(There are many other results.)
If you want to have a nice, clean search experience like google did in the good old days (before everything was page-rank-gamed and hyper-individuialized) just switch to duckduckgo. Google search has become insufferable, it tries to 'guess' the context of your queries and it fails miserably in my opinion. DDG is not per-se better, but at least it is not riddled with SEO spam...
Ive made multiple 4-5 word queries looking for specific information ive found before and will be presented with 1-3 pages of results. What happened? I remember always getting g o o o o o o o o o o o o o o o g l e pages and pretty much always being able to go to the 30th "O". I find it hard to believe the internet is now smaller than it was in mid 2000s
Plenty of technical topics where i get better results from duck duck go (and i dont even like it)
They're censoring searches. If Google were a tool like a saw, it'd have 2 or 3 of it's 200 cutting grooves left. People would say, oh it's just poorly maintained! But the reality is they shaved those grooves off because they were scared you'd finish cutting.
Of course it's worse.
Anyway, I haven't used Google for many years at this point, so whatever. However very recently I'm now having issues with DDG -- seeming to ignore when I quote things, returning only tangentially related pages, including my location(!) when I didn't ask it to, etc. Does anyone know if something notable changed at DDG in the past month or so?
Filter out any Pinterest links. Those will pollute your results.
I have a feeling Pinterest results replaced the original pictures in results.
They must have been pinned from somewhere, and similar image search won't show me where.
pinterest is practically made to hack pagerank: get a thousand people to add a pin to their own board and now you have a thousand links pointing to that pin. I would think same domain linking would be downranked but what do i know
sometimes i have good luck with tineye.com, but sometimes no dice. instagram for instance doesn’t get crawled - I found a good source of info once because someone had made an entire clone of all of instagram, with all the same usernames so it was easy to find the original after finding the clone via google image search
… and terribly so.
Yes, each time I'm looking for something specific I'm only getting results from more popular "similar requests", and of course sponsored ads, this is very infuriating.
On a similar note, but this time for reasons that completely elude my reasoning, GPS navigation with google maps has become worse and worse over the years. It went from surprisingly good at avoiding closed routes and accidents at a time when there was likely a lot less real-time data available, to guiding me over and over again to the same closed roads that sometimes have been for weeks, even though they have troves of user data that clearly show everybody having to turn back and change course. WTF google ?
Also, please implement something like traffic prediction, when I leave around 5pm when it's still fluid, I get an ETA like 40 minutes later, even though I know for sure that it's going to get way worse while I'm on the road and those 40 minutes become more like 90. If I can predict it, why can't google ? That's just crazy stupid.
I recently wrote to somebody talking about a specific type of WiFi interference from idle wifi modems not transmitting traffic. No matter what you search Google for regarding the specific type of interference we were discussing, google dumbs it down for you and provides exclusively results for "Di'ja wanna know how 'ta choose the best wifi channel?" completely ignoring my technical terms even when I add plus signs and double quotes around them.
Good quantitative data is hard to find in this space. I'd welcome it, both as a user and out of interest in the data science. How would you quantify things? What search corpus bearing in mind the algorithm is possibly influenced by what terms are being used worldwide, weighting caches and the like.
Yes oh god, anytime i search for something simple in my own language i get tons of auto translated articles (likely with google translate, heh) which are completely botched with no useful info, just clickbait and jumbled sentences using as many keywords as possible. The spam content is killing the web.
I only use news.Google.com to search news articles, which it's somewhat okay at as long as it isn't a political related search (which it is extremely biased at). Otherwise I find google pretty much sucks at search anymore and don't even bother with it.
People still use Google to search the web? JK. I've been using DDG for _years_ now so I haven't kept up as much with what Google is doing these days. I used to !g when DDG turned up empty-handed and I have noticed that's become more and more pointless. Is Dogpile still around? I haven't used them in years either.
I indeed had some troubles today actually. We're finally moving from JDK7 to 8 and I was searching for recommended JVM settings.. but I only got old results and those results where biased on old info and their old presumptions. ( e.g. people meanwhile found stuff out so its good to have more relevant info on those findings, let alone newer patch versions can fix things that where broken previously)
Adding "last year" to the filters didnt help either; I just got general docs from Oracle, but no developer blogs with best practices, benchmarks or experiences from fellow devs.
I have tried to use DDG for technical queries but Google still trumps them in this regard. However, I have noticed that DDG has gotten considerably better in general non-technical queries over the past year.
Yes Google is following Facebook and YouTube business model of building echo chambers. You only get stuff that confirm your opinion so it's useless if you want the truth about any particular topic. DDG is better but it has problems too
The only thing I noticed is that I almost have to scroll "2 pages" until I get to actual results and not just ads. If it keeps going like this actually having to click on page two isn't so far fetched anymore ...
Not only that, the alternatives have vanished and have also become worse as they adopt the same algorithmic changes google has made over the years (or just outright scrape google behind the scenes).
This is shocking given the existence (now) of publicly available, open source indexing databases. You can literally build a search engine without indexing infrastructure, but everything out there is still crap.
> publicly available, open source indexing databases
Where?
For example, google seems to want full sentences instead of just keywords now.
Are we really shaming Google for having superior NLP? If you want the answer to a question, ask the question! I see no problem with this progress.
Yes. It seems it has somehow stopped returning search results.
It tries to answer my question, it shows summaries of Wikipedia articles, it has a host of sponsored links, it can point to things on the map.
But I'm using a Web search engine and I was trying to find Web pages containing my search terms. That idea seems to have been lost somewhat.
(yes, I guess if you scroll down far enough. But it's probably a matter of months before they're all gone)
Agreed, google search became a PoS for tech users. I can't find anymore things I know exists - that means I won't ever be able to find new stuff there.
Google will frequently try to sell me something unrelated when I do a specific query. It'll throw up links to local stores that match the text somehow. It's obviously about getting me to spend money.
These days, if I want to search on something highly controversial, Yandex usually gives the best results. Google has perfected the "twiddler"[1] so it's impossible to find anything off narrative for many topics.
[1]
https://youtubecensorship.com/2019-08-18-google-youtube-rigg...
Similar experience, unfortunately, and this is common to other search engines like Bing or DuckDuckGo.
I find it more difficult to find what I am looking for, and I spend more and more time searching directly on Reddit or Youtube.
Search results for products have become worse, they are basically useless. I don't think that's primarily Google's fault, though, aggregation blogs and "list sites" have basically outsmarted Google's algorithms.
I still find what I want - honest opinions and unpaid reviews - by adding "reddit" to the query. But without that the normal results on the first page are complete rubbish.
Do you mean 'measurably?'
Well i don't know what you're talking about, i can always find the ads i m looking for. I mean the most serious business would pay the most to have their ad on top of results right? Works for meee
I also get SEO lessons from sites below the ads in the list, it's quite a sight to observe how they manage to all come up with the same content slightly rearranged, and still be clogging the frontpage.
I am sure it is possible to surprise with a search service that is as much better as Google were to it's competition when it launched. I have no idea how to build it, but given that Google eqrch as of today is not very good, it must be possible.
This seems like much more of a qualitative statement than a quantitative one.
Imo YouTube search is really bad. First 5 results are related to your query, rest are just the most popular.
I see others have mentioned DuckDuckGo. Allow me to suggest swisscows:
I've been using it as my default search engine the last couple of months and it seems to do a pretty good job.
Seems to be based on Bing, see
https://en.wikipedia.org/wiki/Swisscows
They are pretty centralized this days and always try to show results from well-know sources: there services like youtube + facebook / twitter / instagram / wikipedia and other well-know sources and news portals.
I can't believe how much straight up spam/malware sites get into the first and second page of results. Sites that go to something like chickenaffairs.com, they seem to scrape content from other sites that happens to contain your search terms, get indexed in google, and then stay there somehow
You are right, I'm one of the people who is frustrated by this. I have been researching this for a very long time. To my understanding, page rank algorithm is dead. Current definition of the Web is not just for public websites, it's now including a good majority of paywall websites and smart device apps. So, if we really wanted to organize world's data, we sure need an entirely new algorithm that will give equal opportunity to all these information holders which Google's current business model can't deliver. Again, to make things right, the next disrupter is going to be a neutral algorithm that will make all these closed information searchable in the public without infecting any business's individual interests.
I'm seeing an interim era of Boutique search engines (powered by better search & people curation) - just like the old Yahoo (yet another hierarchal * *) era. I also believe that a unified search algorithm will unfold following this "yahoo like interim" again, to reorganize the Web once again.
I'm sharing a document that I'm updating on this exact issue, with the information I have collected so far:
https://docs.google.com/document/d/1cSMY5wXSKhJdMxeJEvTUJ21e...
I'm working on a project as well, to try solving this:
- with an open, neutral search protocol. Anyways, some web standards organization can do the same thing better than me, with more impact. If they do, I'm more than happy to see that as well..
Any product search now is to a list of Amazon referral links, product search on Google has become horrible
That is why you are seeing way more Search Engine startup. No one at Googles understand anything about quality, nor the intuition to understand what is good and what is bad. So they decide to AB test the heck out of everything.
It is a lot like Microsoft in the 90s.
Or when I use the date filter for just the past year, and it's still pages from 2013 on the first page of results.
I mean yesterday their result page for “twitter new CEO” looked like this:
So… yeah. The funny bit is that the actual search algo got it right.
I’m with others on appending Reddit to your search.
Check it today? It is full of pictures and information about new Ceo.
It works if you put it in "Verbatim" mode for that particular search, but I don't know how to make it permanent. Does anyone know if there's a way to always force it to use "verbatim" mode?
I know of a way and have been using it for years. It's a bit too complex to describe here. Reach out to me via email if interested. My email is in my profile.
I think Google just seems to be trying their best against an army of SEO/content marketing parasites constantly trying to game their algorithm, look up a recipe website for an example of what the internet will look like if those guys win.
Yes. Google is noticeably worse year on year for technical queries in particular, but everything else as well.
My father has been saying this for years. I find Google to be useful by using search operators. I know for a fact that the YouTube recommendations algorithm is “worse” than before. It’s infuriating
YouTube is fucking horrible. I can't fucking stand the "People also watched" results 5 results into your search results. I don't give a fuck what other people watch, I want what I searched for (which is already fucking bad)!!!
One thing I have noticed, not sure if this is recent or well treaded ground here on HN no doubt, but if you Hover over the "I'm feeling luck" button then leave and hover again, it gives a bunch of different options like "I'm feeling adventurous", I'm Feeling Puzzled, I'm Feeling Curious. I am not sure what changes to the algorithm is made when you use one of those, they do seem to give different results. I am sure it is probably documented someplace but I think it would be nice if they had a drop down menu there instead with perhaps a bit of elaboration. I have tried using different ones when the regular search is giving me pop culture type results due to perhaps ambiguity with the more interesting topic I am searching on.
I always thought a good one would be "And now for something completely Different" that would give you random low ranked results just for fun :)
Yandex destroys Google when it comes to 'search by image'. The AI recognition is incredible and it can pull results from some very weird places.
Bing video search is obviously the best choice when it comes to porn.
Brave Search is getting very close to Google for almost 90% of text searches.
Yes the Internet needs several viable search competitors but since the cost of maintaining all the data and creating a good algorithm are so high the barrier to entry seems almost insurmountable without some really clever innovations.
Yes. There is more and more sponsored content dominate searches. Technical searches often return 15 XXXExtremeProgrammingAdvice.com sites with scrapped junk.
I switched to DuckDuckGo a few months back and I am very happy with the change.
There are many comments along the lines of "curated search, yes, please, so much".
Right now search is "free" (user is the product). How much would you be willing to pay for a non-ad-driven, curated search site?
tips: do prompt engineering like you're programming VQGAN
GAN image generators can be tricked into giving results with phrases like 'unreal engine', 'artstation', 'junjo ito', 'van gogh'
https://minimaxir.com/2021/08/vqgan-clip/
same for google. 'gist' gives you a specific category of technical results. seems like the default is some combination of medium, SO + github issues
There's a whole legion of (underpaid) Search Engine Results Analysts out there that can tell you horror stories of why this is happening.
I feel like they're so much worse. It used to be I could be precise about what I wanted, but now if it's not in the initial results, it's almost pointless to try to refine it at all.
I think people stopped writing and started making videos because a revenue stream (sponsorships, patrons, ads) exist there. Nobody pays people to write anymore, so people stopped doing it.
It may be worse for some of us but is it getting better for the average/common person?
I’ve been using DDG for sometime now and can attest, it’s results are very good for technical queries. It’s fast and private.
Long story short: They replaced carefully crafted human readable rules with black box AI.
After ranking results by profit it wasn't even necessary anymore to index or present other results.
Equally unnecessary it then became to maintain or create any such websites.
The future, if you ask me, is carefully crafted invitation-only websites. Get back to information exchange just for the sake of sharing, discovering and learning.
I find it so offensive, the search engine is a fountain of knowledge and any small mod has such a profound impact on our well being
Who are these people at google that feel entitled to deprive the human race of it's most precious resource? DDG has worked well for me
For me, very definitely. But. I've also turned off all search history etc so am probably shooting myself in the foot...
There are a few factors at work.
Primarily, quality content and discussions are now all paywalled (e.g. journals, magazines, newspapers, comment threads, and original content).
Secondarily, remnant content is trivial search engine marketing content. This content is low quality, low information, often false, with a low-grade reading level.
Third, google is actively down-ranking & removing “controversial” content in the name of ML fairness & integrity. In many cases this may be just, but it’s also going to lower the overall content quality because almost everything innovative is going to be controversial .
So you have at least three massive forces preventing quality content & discussions from being found on web search.
it's a SEO / blog spammers paradise, with terrible results, designed to drive usage of AdWords
Now that YouTube has gotten rid of dislikes expect YouTube to go down the same path. There will be more trash content and you'll only know once you watch the video (comments can be deleted).
YT injects ads in front of unmonetized videos now, so they'll have bigger metrics for engagement and make more $ on pre-roll ads.
They are. I've solved the problem for myself by setting the "Verbatim" option to true in search engines' settings google url of browser.
Where can I find the "Verbatim" option?
Yes. Google and Duck Duck Go are both horrible for finding information these days.
I am actively experimenting with alternate options.
Yes. It’s broken. I’d pay $20/month for search that worked at least as well as Google used to back in the day.
Looked at
?
These days for most queries I append a " reddit" or " hacker news" to circumvent the SEO bs pages.
I don't think Google got worse, there are always a few issues with an argument "Google got worse"
Which search result is worse compared to "the time Google was good"? We need solid examples, which is usually extremely hard to find on HN threads such as these. Even the parent does not have a single good example.
Yes. Extremely, much, worse. The same goes for YouTube.
Does anyone have a solution though? Even DuckDuckGo is bad.
they talked so much about how important iphone for them, well.... i have google chrome on ios, go ahead and try to edit your query... good luck with that, it's just plain bad.
google seems to want full sentences instead of just keywords now. "How do I do X?" seems to get me better(?) results then "X + some relevant keyword"
Correct. Google will give you better results with full-sentence-like input. After several years of refining results, Google concluded that there's more benefit in teaching the machines to understand how humans ask questions than to teach humanity how to keyword like a computer expects (especially when you factor in that they get as many queries via voice these days as via text, and voice recognition in general _always_ benefits from more information to disambiguate on). There's an entire semantic-analysis layer in front of the keywording layer these days to determine some semantics of the query to try and guess what category of thing you're looking for.
I generally have no problem with a few keywords for software engineering searches. I usually go general-to-specific (for example, `react unit test useState`).
You can drop the video results by adding `-youtube` to the query.
"Sal dulu a" would both recommend "Sal dulu antasma" and list it but now unless i search for that particularly, it doesn't show up.
I'm not sure, but it's possible Google dropped 'a' as a signifier because of the semantic query support (as a single particle, it doesn't add signal to a sentence-like query). `sal dulu songs` gives me a list where Antasma shows up as item 3.
In general, my advice for Googling these days would be "don't try to keyword it out." Think more like how you'd ask another human for a random fact they might barely remember.
There's also still some symbols that are specifically understood by Google for tuning queries, listed here (
https://support.google.com/websearch/answer/2466433?hl=en
). Worth noting: the `+` modifier got killed when Google+ came and went. To force a word or phrase to be part of the results instead of "fuzzy-matched," put it in "quotes". Quotes these days do double-duty as both "I want this literally matched" and "results _must_ include this token."
> Correct. Google will give you better results with full-sentence-like input.
Please show how to use a full-sentence natural-language query to search for images of a shirt without stripes. Thanks.
The results here are interesting.
`shirt without stripes` definitely fails (and I see it is a natural-language meme because that's what is returned most prominently against that phrase. Hilarious.)
`shirt -stripes` as an image search is also not foolproof. It does a better job, but some leak through (I would assume for two reasons: if the text just doesn't mention it, the negative key filter has nothing to key on, and we have different words for "stripes" that Google doesn't consider synonyms. Is a plaid shirt striped? What about a checked shirt?).
But hilariously, all of the ad results are stripe-heavy. Looks like there's a bug: negative keywords in image search aren't passed as negative keywords to the ad engine, but instead as positive keywords. Wasted money for those advertisers; that's definitely not what the user wants.
For this sort of search, I'd actually use Shopping... Except that while there is a "Striped" category, I know of no way to do a _negative_ category search on the Shopping UI, so I get the opposite of what I want. I can get all the striped shirts ever, but non-striped would be harder.
I suppose I should amend my previous comment to say "If you don't try natural language queries on the fringe of known unsolved problems, it works better than keywords." ;) And since Google is forever targeting the common use-case, perfect may be the enemy of good here for getting most people what they're looking for most of the time.
i get only spam websites, that redirect to Aliexpress, in top positions lately
empirically yes, for a number of things. But it's still better than the rest.
Google results are 80% paywalls, subscription blackouts and mailing list nags.
I use google to search for results on websites I trust, it's basically useless otherwise.
Without any doubt both YouTube and google search have gotten way worse compared to ten years ago. It was crazy when I realized that I may have lived through a brief window in history when search was at its historical peak before being corrupted by politics and money or whatever it is that’s caused this change.
Amazing suggestions from commenters on better search engines.
Why hasn't a group of passionate developers gotten together to build an amazing, open source, ad-free, tamper-proof search engine?
Wouldn't that instantly be like a mega impactful, economy-changing kind of project?
Good question.
Do you know of any projects in that direction?
Why is it that all the biggest search engines on the planet are now practically useless?
Google, Amazon, eBay... practically worthless.
There are plenty of good comments in this thread about how Google search results aren't as good because of SEO and companies like Pinterest, "gaming the system". However Google itself is not blameless in making their search quantitatively worse through their injection of censorship and politics into their searches. Google manipulates searches on a whole variety of subjects - current, political and historical - on a regular basis. For example, Google, "Syria gas attack" and compare the top 10 results you get when you go to Duck Duck Go and search for the exact same term. Google will tell you its because it ranks "authoritative" sources higher. What metrics Google uses to determine how "authoritative" a source are not revealed. Unsurprisingly, Google's choice of "authoritative" sources always aligns exactly with the DC/Pentagon/State Department blob. Ironically, these attempts at censorship by Google just serve to further undermine what little trust many people have in "authoritative" sources all together (aside from making their search function much less useful).
Recently (last few months) Google results got REALLY BAD for me due to its attempts to provide region-specific results.
I live in Argentina, and sometimes I search for Argentinian things, but also more often I search in English for things related to work, travel, hobbies, etc. It's reasonable and OK that when I search for certain things Google provides region-specific results (e.g. "servicio meteorolĂłgico nacional" [National meteorological service] brings up the Argentinian agency and other local results instead of any from other Spanish speaking countries). This has been like that for many years now, and is OK.
The new thing is that more and more often I've been getting results that include argentine or Spanish-language results that have NO relevance to the search, and are a super big stretch.
I haven't saved examples, but I tested for 3 minutes and just came up with one... trust me that I've seen much worse than this.
Let's say I search for "french tv cat puppet" (I watched an old show called Telechat when I was a kid). I get the following results:
1 & 2 - Kitty Cats - Wikipedia (a French Canadian TV show... OK...)
3 - Pacha et les chats @ imdb (the same show)
4 - YouTube (???? "Missing: french ‎cat ‎puppet", warns Google)
5 - "Carmel: Who Killed Maria Marta?" @ Netflix (this is a true crime show based on a very famous murder in my country)
None of those results were ads. I have no idea why Google would show results 4 and 5 except to think that their index for my country is very very broken/corrupt.
Another "not as bad but still very bad" example: If I search for "mccartney bear song" (see Rupert and the Frog Song, and the "We all stand together" song) the first two results are spot on, but the third one is a Spanish language result from a local paper titled "These is the full list of songs that The Beatles played in the Get Back documentary by Peter Jackson", the snippet has McCartney and the word "song" from "Song Of Love" in bold.
Final one. I search for "retro toys with water inside"... the search terms are super poor, but the image results show me that Google got me (e.g. the Tomy Waterful toys).
Image results - OK
Video results - OK
1 - Handheld water game @ Amazon (OK...)
2 - Pinterest results (groan...)
3 - A hacked home decor Argentine site that redirects me to a page that sells a doll, no water.
4 & 5 - Relevant retro toy pages
6 - A hacked Argentinian government page (under the .gov.ar TLD, which is for government site) with the "classic fashion CHATHAM ELECTRONICS JAN-CAHG-1Z2" title. Apparently has been fixed so now it has a 404 error.
I rest my case. It's a disaster.
It's really bad if you don't use uBlock Origin, that's for sure. The amount of ad
and sponsored results has gotten really bad in recent years for a lot of queries.
Even if you use it, "organic" results seem to be just links to e-commerce websites.
google,youtube search is trash now. it's adds search now
When I can't find videos using YouTube's search that I've watched on YouTube before, I use DDG and it is usually the first result.
Yes.
yes but competition is even worse
Google is no longer a search engine - it's a behavioral management software. Stop using it unless you like being manipulated.
I submitted a post about this some time ago.
https://news.ycombinator.com/item?id=28113007
It's not just an issue of refining results to fit a majority of the population. Something is seriously wrong with it.
It is incredibly worse than it used to be. They hate organic content that is not under their control. Think forums, blogs, etc. you have to add stuff to your query to get it to bubble the organic content to the top. The worst is when you are searching for enthusiast information that is typically found on forums, newsgroups, etc. I will often include the word “forum” or “blog” on my queries to find what I am looking for.
From a freedom of speech perspective unfortunately. Google does a very good job of crushing wrong think.
I'm not even talking about the last 2 years when they've been openly policing I mean stuff that's critical of employees that google has had to fire for committing felonies on their infrastructure.
This constant "refinement" is getting to the point that I regularly end up hitting page 4 or 5 on technical matters because google introduced a broken system to counter abuse from people claiming to be best friends with bigfoot in his UFO...
I with they'd trust tech blogs more than they do... Esp from authors sticking to the subject matter not conspiracies or opinion pieces. A decent tech blog is worth 1000x more than musk has ever tweeted for instance imo.
Google has probably gotten better at its business, better search results in the ads than the search results themselves.
By owning demand for search the attention can be guided.
i noticed the exact same thing..
it's almost impossible to google game dev programming things... IMPOSSIBLE to find good resources anymore, it almost as if old (and valuable) resources are not referenced anymore!
Not sure why anyone would think an answer to such a question lies here. There are teams of PhDs working on search inside Google with high levels of secrecy. We are to somehow deduce based on just the output of some searches what quantitative effects those works have? I'm not even sure Google itself can answer that question.
Absolutely, to the point I no longer use it.
For me it was the pinterestizing of image search that finally made me realize how pointless google was. Why use a service that is just going to feed me ads in response to every query?
What do you use now?
DDG for new searches, but mostly just bookmarks and in-site search when available.
Searching the whole web isn’t nearly as useful as curating useful sites yourself.
Yes, it seems like google (and search in general) has slowly been losing the war with SEO as more and more queries return pages of SEO affiliate blogspam and less and less organic, actually useful content.
On one hand, google has been shown several time to quite literally censor certain keywords/topics out of their results, so in that regard yes.
As for the quality of results overall, depends if you value "what the crowd thinks". I personally don't, but i can see the value of it, especially when searching something very niche where the google search right now can semi-comprehend the context and give you the ~best answer.
For me, I haven't daily-drove google search since 2015 (ddg, startpage,customized searx all work fine for me), so i can't say that i know the nitty-gritty besides using google to compare results between engines.It gave me the impression of becoming an echo-chamber due to the "fact-checking" role google assumed.
Walled garden. It will converge as the great reset converges. It is like closing the fence around your cattle.
Who cares. Stop using their spy machine and use duckduckgo
Google now tells us what we're searching for.
Yes, I switched to DDG and its good enough for my needs
You've asked ~4 different questions.
Close to none of the replies acknowledge this.
Yes, HN quantifiably is close to useless. They can't even understand the problem/questions let alone the answer.
To answer some
No, Google is not worse, it's just different. But I can't quantify it. I haven't kept old data.
For Dorking check out
https://en.m.wikipedia.org/wiki/Shodan_(website)
Google Dorking still works fine. Almost by definition it's up to you to find new Dorks as the old ones get closed.
The fact Sal dulu a used to autocomplete to Sal dulu Antasma might have been Personal results, did it work for anyone else? Have you turned this off?
I search using DDG by default. Sometimes by habit I'll throw a !g on the search to tell it to use Google, but more and more often I have to think twice. If my search touches politics at all, Google is a no-go. Joe Rogan mentioned this phenomenon recently. Searching for a specific event, he explained, it was easily found on non-Google search engines, but Google hid it behind dozens of Ministry of Truth results.
This is not exactly what you're asking, I know. My point is, if they're willing to screw around with some results, I have lost confidence in all of their results. Long gone are the days where the contract with Google was that you would provide some words, and that it would provide a list of pages with those words in them.