Weight-Space Programming from scratch on simple tasks
Mechanistic interpretability usually works backwards: train a network, then try to work out what its weights do. Rikiya Takehi's recent post on Weight-Space Programming argues that the forward direction is now within reach, with language models reading and writing network weights directly instead of learning them by gradient descent. We've been testing the same idea on small tasks with known solutions, using Claude. Claude, running as an autonomous coding agent, was given a blank transformer and one rule: set every weight by hand, with no training. Across 13 tasks Claude wrote 959 weight-space programs. All ten algorithmic tasks, including addition, multiplication, sorting, parity and GCD, reached 100% accuracy within three attempts, and Claude spent the rest of each run simplifying its solutions. Because each program is written rather than learned, it comes with an account of what every head and neuron does, which makes these models a source of ground truth for interpretability. Sorry