Improving document retrieval with sparse semantic encoders · OpenSearch
OpenSearch 2.11 introduced neural sparse search—a new efficient method of semantic retrieval. In this blog post, you’ll learn about using sparse encoders for semantic search. You’ll find that neural sparse search reduces costs, performs faster, and improves search relevance. We’re excited to share benchmarking results and show how neural sparse search outperforms other search methods. You can even try it out by building your own search engine in just five steps. To skip straight to the results, see Benchmarking results.
OpenSearch 2.11 introduced neural sparse search—a new efficient method of semantic retrieval. In this blog post, you’ll learn about using sparse encoders for semantic search. You’ll find that neural sparse search reduces costs, performs faster, and improves search relevance. We’re excited to share benchmarking results and show how neural sparse search outperforms other search methods. You can even try it out by building your own search engine in just five steps. To skip straight to the results, see Benchmarking results . What are dense and sparse vector embeddings? When
Explore this link on the map →related reading
- Building a web search engine from scratch in two months with 3 billion neural embeddingsblog.wilsonl.in
- Semantic Search - Sentence Transformers documentationsbert.net
- Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
- Announcing ScaNN: Efficient Vector Similarity Searchai.googleblog.com
- From grep to SPLADE: A Journey Through Semantic Search - Elicitblog.elicit.com
- Introducing text and code embeddings | OpenAIopenai.com
- Vector embeddings | OpenAI APIdevelopers.openai.com
- GitHub - deepset-ai/haystack: Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, andgithub.com
- Faiss: A library for efficient similarity search - Engineering at Metaengineering.fb.com
- Okapi BM25 - Wikipediaen.wikipedia.org
- [2004.04906] Dense Passage Retrieval for Open-Domain Question Answeringarxiv.org
- Rerankers and Two-Stage Retrieval | Pineconepinecone.io