flâneur

Scaling LLMs to larger codebases - Kieran Gill

blog.kierangill.xyz · 2,582 words · saved by 1 readers

First we learned what LLMs and genetics have in common. (part 1) LLMs don't improve all facets of engineering. So, understanding which areas LLMs do improve (part 2) is important for knowing how to focus our investments. (part 3) How do we scale LLMs to larger codebases? Nobody knows yet. But by understanding how LLMs contribute to engineering, we realize that investments in guidance and oversight are worthwhile. When an LLM can generate a working high-quality implementation in a single try, that is called one-shotting. This is the most efficient form of LLM programming. The opposite of one-shotting is rework. This is when you fail to get a usable output from the LLM and must manually intervene.2 This often takes longer than just doing the work yourself. So how do we create more opportunities for one-shotting? Better guidance. LLMs are choice generators. Every set of tokens is a choice added to your codebase: how a variable is named, where to organize a function, whether to reuse/exten

This is the third part of a series on LLMs in software engineering. First we learned what LLMs and genetics have in common . (part 1) LLMs don't improve all facets of engineering. So, understanding which areas LLMs do improve (part 2) is important for knowing how to focus our investments . (part 3) How do we scale LLMs to larger codebases? Nobody knows yet. But by understanding how LLMs contribute to engineering , we realize that investments in guidance and oversight are worthwhile. Guidance : The context, the environment. Oversight : The skill set needed to guide, validate, and verify the imp

saved by

related reading