Beating GPT-5.6 Sol on retrieval with 100x cheaper open models
Ranked #5 on Hacker News with 206 points and 38 comments.
A 4B open-source model post-trained with Castform retrieved search results as accurately as GPT-5.6 Sol, while costing 100x less
Receive only our latest updates. No spam, guaranteed.
“Most teams' best training data is just sitting in their databases. The problem is that turning raw data into something usable is hard, and letting agents read, search, and mutate data cheaply at scale requires advanced infra. Pointing Castform at Neon skips both.”
A "good agent" needs to be strong in 2 areas:
Neon (Lakebase Postgres) and their new Search extensions solve the first; Castform solves the second.
In ~2022, the industry was going all in on embedding search. Every database provider added one, and pgvector was Neon's most downloaded extension. To provide context to LLMs, engineers handcrafted RAG pipelines, which in essence, is some form of embedding similarity search.