Slide 1
static.googleusercontent.com · 3,823 words · saved by 1 readers
N/A
Challenges in Building Large-Scale Information Retrieval Systems Jeff Dean Google Fellow jeff@google.com Why Work on Retrieval Systems? • Challenging blend of science and engineering – Many interesting, unsolved problems – Spans many areas of CS: • architecture, distributed systems, algorithms, compression, information retrieval, machine learning, UI, etc. – Scale far larger than most other systems • Small teams can create systems used by hundreds of millions Retrieval System Dimensions •…
related reading
- The Anatomy of a Search Engineinfolab.stanford.edu
- Building a web search engine from scratch in two months with 3 billion neural embeddingsblog.wilsonl.in
- Hermes: Algorithm-System Co-design for Efficient Retrieval-Augmented Generation At-Scalemichaeltshen.github.io
- GitHub - donnemartin/system-design-primer: Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.github.com
- Indexing 1T vectors · facebookresearch/faiss Wiki · GitHubgithub.com
- Faiss: A library for efficient similarity search - Engineering at Metaengineering.fb.com
- Build Your Own Databasenan.fyi
- Building and operating a pretty big storage system called S3 | All Things Distributedallthingsdistributed.com
- How We Store and Search 30 Billion Facesclearview.ai
- RETRO Is Blazingly Fast | Mitchell A. Gordonmitchgordon.me
- Why BM25 queries with more terms can be faster (and other scaling surprises)turbopuffer.com
- Information Retrievaljzhao.xyz