Postgres as a search engine — any blockers?

Build a retrieval system with semantic, full-text, and fuzzy search in Postgres to be used as a backbone in RAG pipelines.

Back Postgres as a search engine August 19, 2024 / 15 minute read Search is hard . It’s a critical part of many apps, but getting it right isn’t easy. This is especially true for RAG-pipelines where the quality of retrieval can make or break the entire process. While semantic search is trendy, good old lexical search is still the backbone. Semantic techniques can improve results, but they work best when added to a solid text-based search foundation. In this post, we’ll explore how to use Postgres to create a robust search engine. We’ll combine three techniques: Full-text search with tsvector S

related reading

• Create an advanced search engine with PostgreSQL | xataxata.io
• Full Text Search | Supabase Docssupabase.com
• Building a web search engine from scratch in two months with 3 billion neural embeddingsblog.wilsonl.in
• Full-text search engine with PostgreSQL (part 2): Postgres vs Elasticsearch | xataxata.io
• Optimizing Postgres Text Search with Trigrams - Alex Klibiszalexklibisz.com
• turbopufferturbopuffer.com
• Rerankers and Two-Stage Retrieval | Pineconepinecone.io
• Not All Vector Databases Are Made Equal | Towards Data Sciencetowardsdatascience.com
• Better RAG 3: The text is your friendolickel.com
• Optimizing RAG: A Guide to Choosing the Right Vector Database | by Mutahar Ali | Mediummedium.com
• From grep to SPLADE: A Journey Through Semantic Search - Elicitblog.elicit.com
• Introducing Hybrid Search and Rerank to Improve the Retrieval Accuracy of the RAG System - Dify Blogdify.ai