Why BM25 queries with more terms can be faster (and other scaling surprises)
I analyzed how BM25 query latencies scale with document count and top_k. Longer queries scale less efficiently, and essential terms impact performance in some surprising ways.
Why BM25 queries with more terms can be faster (and other scaling surprises) NEW: Instant namespace branching NEW: Branching for instant, copy-on-write namespaces Why BM25 queries with more terms can be faster (and other scaling surprises) January 07, 2026 • Adrien Grand (Engineer) BM25 full-text search has very different scaling characteristics than vector search. Vector search latency is generally a function of vector dimensions, top-k, the size of the dataset, and the presence of filters. BM25 latency, on the other hand, also varies a lot by query, and in some surprising ways: Sometimes add
saved by
related reading
- Okapi BM25 - Wikipediaen.wikipedia.org
- Transformer Inference Arithmetic | kipply's blogkipp.ly
- Building a web search engine from scratch in two months with 3 billion neural embeddingsblog.wilsonl.in
- LLM Inference Performance Engineering: Best Practices | Databricks Blogdatabricks.com
- All About Transformer Inferencejax-ml.github.io
- Static search trees: 40x faster than binary search · CuriousCodingcuriouscoding.nl
- turbopuffer: fast search on object storageturbopuffer.com
- Not All Vector Databases Are Made Equal | Towards Data Sciencetowardsdatascience.com
- Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
- Spending Inference Time - Kevin Lukevinlu.ai
- All the Transformer Math You Need to Know | How To Scale Your Modeljax-ml.github.io
- Training a Specialist Code Search Agent with turbopufferappliedcompute.com