The Ground Truth - Vibe Coding Tips and Eval-Aware AI

This week I will be sharing some tips on using AI coding tools like Cursor, and some interesting AI news, including AI becoming “eval-aware”, i.e. it knows that it is being evalauted and try to behave in a certain way to pass (cheat) the test. The first tip is from illyism on X. Software engineers have long known the benefits of keeping file size small to promote proper abstraction, reduce coupling and enforce single responsibility principle on different modules. Refactoring, basically. Given the effective context length limits on LLMs, this practice is more important in the age of AI coding for managing context. For example, if you have 1 file with 2000 lines of code, the AI has to go through the file to find the lines that are relevant. However, if you spilit that file into 10 files with 200 lines each, the AI can check the file names to determine the files that are relevant, and only load the relevant ones to its context window. Having smaller files allows models to handle more comp

This week I will be sharing some tips on using AI coding tools like Cursor, and some interesting AI news, including AI becoming “eval-aware”, i.e. it knows that it is being evalauted and try to behave in a certain way to pass (cheat) the test. The first tip is from illyism on X. Software engineers have long known the benefits of keeping file size small to promote proper abstraction, reduce coupling and enforce single responsibility principle on different modules. Refactoring, basically. Given the effective context length limits on LLMs, this practice is more important in the age of AI coding f