Fast regex search: indexing text for agent tools · Cursor

How we're building indexes for regular expression search so agents can find text in large monorepos without the 15-second ripgrep waits.

Blog / research Time is a flat circle. When the first version of grep was released in 1973, it was a basic utility for matching regular expressions over text files in a filesystem. Over the years, as developer tools became more advanced, it was gradually superseded by more specialized tools. First, by roughly syntactic indexes such as ctags . Later on, many developers moved to specialized IDEs for specific programming languages that allowed them to navigate codebases very efficiently by parsing and building syntactical indexes, often augmented with type-level information. Eventually this was s

saved by

• Yudhister Joel Kumar
• Kushal Thaman
• Tracy Tsoi

related reading

• Regular Expression Matching with a Trigram Indexswtch.com
• The technology behind GitHub’s new code search - The GitHub Bloggithub.blog
• The technology behind GitHub’s new code search - The GitHub Bloggithub.blog
• Optimizing Postgres Text Search with Trigrams - Alex Klibiszalexklibisz.com
• string - Data structure for fast full text search - Stack Overflowstackoverflow.com
• Building a web search engine from scratch in two months with 3 billion neural embeddingsblog.wilsonl.in
• Introducing SWE-grep and SWE-grep-mini: RL for Multi-Turn, Fast Context Retrieval | Cognitioncognition.ai
• Create an advanced search engine with PostgreSQL | xataxata.io
• The Anatomy of a Search Engineinfolab.stanford.edu
• Implementing Regular Expressionsswtch.com
• How Cursor Indexes Codebases Fast - by Engineer's Codexread.engineerscodex.com
• Pattern Searching using Suffix Tree - GeeksforGeeksgeeksforgeeks.org