flâneur

Character Prefix Conditioning | Cursor - The AI Code Editor

cursor.com · 242 words · saved by 2 readers

A clever algorithm for more accurate code completion sampling.

The first in a series of problems that give a glimpse into the work we do at Cursor. Setup When using a language model for code completion, we typically want the model to produce a completion that begins with what the user has typed. However, modern language models operate on sequences of tokens, not characters, so naively tokenizing the user's input and sending it to the model produces wrong results if the user's cursor doesn't happen to lie on a token boundary. Instead, we need an algorithm that samples a sequence of tokens conditional on a prefix of characters, rather than the more…

saved by

related reading