Lean Software Scaling Laws · Gwern.net
Research proposal for measuring how coding LLM perplexity scales with codebase context size, using Lean as a test case for whether formal languages have better predictability exponents and could lead to safer, more secure software worldwide.
--- title: "Lean Software Scaling Laws" description: "Research proposal for measuring how coding LLM perplexity scales with codebase context size, using Lean as a test case for whether formal languages have better predictability exponents and could lead to safer, more secure software worldwide." created: 2025-09-01 modified: 2026-06-23 status: finished confidence: possible importance: 7 css-extension: dropcaps-yinit ... Research idea: empirically measure the scaling of coding LLM perplexity over codebase size to estimate the scaling laws of 'predictability' by programming language or other fac
saved by
related reading
- lf-lean: The frontier of verified software engineering | Theoremtheorem.dev
- Scaling LLMs to larger codebases - Kieran Gillblog.kierangill.xyz
- Composer2.pdfcursor.com
- Code is cheap. Show me the talk. - nadh.innadh.in
- 2025: The year in LLMssimonwillison.net
- Here’s how I use LLMs to help me write codesimonwillison.net
- crawshaw - 2025-01-06crawshaw.io
- If coding is solved, what now?: Measuring the sloppiness of code | EARENDILearendil.com
- Time to Wall - by Julian Warszawski - Hundred Mondayshundredmondays.substack.com
- [2605.02421] AOCI: Symbolic-Semantic Indexing for Practical Repository-Scale Code Understanding with LLMsarxiv.org
- LiCoEval: Evaluating LLMs on License Compliance in Code Generationarxiv.org
- Cheating is all you need | Sourcegraphabout.sourcegraph.com