Fill-in-the-Middle: The Magic Behind Smart Code Completion

How your favorite code editor suggests exactly what you were thinking of

Fill-in-the-Middle (FIM) is a technique in code completion that allows language models to generate code based on context from both before and after the cursor. Unlike traditional left-to-right autocompletion, which only sees past code, FIM provides a "peek into the future," enabling significantly more accurate and contextually aware suggestions. The technique was popularized by OpenAI's 2022 paper, "Efficient Training of Language Models to Fill in the Middle", which detailed an effective method for training models on this task. This approach has since become a cornerstone for many…

saved by

• Varun Goyal

related reading

• Prompts are code, .json/.md files are statemariozechner.at
• Efficient Training of Language Models to Fill in the Middle | PDFarxiv.org
• GitHub - brexhq/prompt-engineering: Tips and tricks for working with Large Language Models like OpenAI's GPT-4.github.com
• Coding Models Are Doing Too Much | whnrehiew.github.io
• GPT in 60 Lines of NumPy | Jay Modyjaykmody.com
• Here’s how I use LLMs to help me write codesimonwillison.net
• ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
• Improving Cursor Tab with online RL · Cursorcursor.com
• copilot-explorer | Hacky repo to see what the Copilot extension sends to the serverthakkarparth007.github.io
• Extending Context is Hard | kaiokendevkaiokendev.github.io
• More problemscursor.com
• The Future of Programming: Copilots vs. Agents (Part I)eastwind.substack.com