2024-01-14
In 2020, tired of every search engine seemingly having suboptimal results and missing the instant answers I wanted, I decided to make a search engine for myself.
I knew making a general-purpose web search engine from scratch by myself was infeasible, so instead I opted to make a meta-search engine, which aggregates results from other web search engines.
First I tried forking Searx, but it was slow and the old Python codebase was annoying to work with.
So instead of forking an existing project, I made my own (but with several ideas borrowed from Searx) in NodeJS which I called simply ”metasearch” (very unique name).
I used it as my primary search engine for over a year, but it was slow (mostly due to it being hosted on Replit and being written in JS) and brittle to the point where at the time of writing the only working search engine left is Bing.
A few weeks ago I decided to rewrite metasearch as (brace for it) *metasearch2metasearch2* (my project names only continue to get more original).
In this rewrite I implemented several of the things I wish I would’ve done when writing my first metasearch engine, including writing it in a blazingly fast 🚀🚀🚀 language.
There’s a hosted demo at s.matdoes.dev, but I’d much rather you host it yourself so I don’t start getting captcha’d and ratelimited.
This blog post will explain what you should know if you want to make a metasearch engine for yourself.
