Lean Software Scaling Laws · Gwern.net
Research proposal for measuring how coding LLM perplexity scales with codebase context size, using Lean as a test case for whether formal languages have better predictability exponents and could lead to safer, more secure software worldwide.
--- title: "Lean Software Scaling Laws" description: "Research proposal for measuring how coding LLM perplexity scales with codebase context size, using Lean as a test case for whether formal languages have better predictability exponents and could lead to safer, more secure software worldwide." created: 2025-09-01 modified: 2026-06-23 status: finished confidence: possible importance: 7 css-extension: dropcaps-yinit ... Research idea: empirically measure the scaling of coding LLM perplexity over codebase size to estimate the scaling laws of 'predictability' by programming language or other fac
saved by
related reading
- Guardian Angels: LLM Personalization for Productivity and Security · Gwern.netgwern.net
- lf-lean: The frontier of verified software engineering | Theoremtheorem.dev
- When AI Writes the World's Software, Who Verifies It? — Leonardo de Mouraleodemoura.github.io
- [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
- Scaling Laws, Carefully | Lil'Loglilianweng.github.io
- Patterns for Building LLM-based Systems & Productseugeneyan.com
- Malleable software in the age of LLMsgeoffreylitt.com
- Scaling LLMs to larger codebases - Kieran Gillblog.kierangill.xyz
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
- Taking LLMs Seriously (As Language Models) — LessWronglesswrong.com
- No, LLM is not going to replace software engineers, here's why – Fang-Pen's coding notefangpenlin.com
- Thoughts on the Alignment Implications of Scaling Language Models | Leo Gaobmk.sh