Post

Eight years ago I added client-side search to this blog with Lunr.js . It creates an inverted index at build time, ships it as JSON, and matches strings in your browser. No server-side engine required. It has worked fine ever since, in the sense that it finds a post if you type a word that is actually in it.

Earlier this year I wrote a semantic search engine in ±250 lines of Python (the kind that lets you find the “London Beer Flood” when you search for “alcoholic beverage disaster in England,” because it understands that beer is alcoholic and a flood is a disaster). That one needs a machine with sentence-transformers installed and a few hundred megabytes of PyTorch; to serve something like that in a production server requires beefy machines with expensive RAM. Not something you run in a browser tab.

So this blog had keyword search that runs anywhere and understands nothing, but no semantic search that understands things but can’t run anywhere near a static site. This post is about closing that gap: semantic search that runs entirely in your browser, with no server and no API, where the entire model is essentially a 4 MB lookup table. You can try all three of the models I benchmarked, further down , running live on your own hardware (I can’t afford GPU clusters).

Doing this surfaced a couple things, chief among which that the keyword search had a bug for a couple of years. TUrns out that doing proper evals is important.

The obvious approach costs 23 megabytes #

The models that power the Python post (sentence-transformers like all-MiniLM-L6-v2 ) can, in fact, run in a browser. Transformers.js will download an ONNX build of one and run it on WebAssembly. I benchmarked it, and on my laptop, the quantized model plus its runtime is 23.45 MB over the wire, takes about two seconds to load and become available, and then embeds a query in ±18 ms.