flâneur

[2608.21664] Measuring Activation Control in Large Language Models

arxiv.org · 4,453 words · saved by 1 readers

Abstract:Safe deployment of increasingly capable models will likely come to rely on latent-space monitoring as a complement to behavioral evaluations, especially when evaluation-aware models exhibit scheming or deception. However, if models can also control their own activations, deception could extend into the latent space itself. With this in mind, we introduce the Activation Controllability Benchmark to quantify the extent to which models can modulate their residual stream via natural-language instruction. Across model families and capability levels, we find that most LLMs can control the direction and magnitude of their residual stream activations with some degree of temporal resolution, though performance varies considerably across models. In simple tasks, this level of control can evade activation-based monitoring methods (including linear probes, natural language autoencoders, activation oracles, and the Jacobian lens), albeit imperfectly. These results suggest that control over the activation space itself could become a confound for monitoring as introspective capabilities increase; therefore, we recommend that frontier labs and evaluators track activation controllability in future models.

Measuring Activation Control in Large Language Models Marek Mateusz Kowalski∗ , Joshua Fonseca Rivera, Uzay Macar, David Demitri Africa† Abstract make faithful self-explanation a native model capability (Li arXiv:2608.21664v1 [cs.AI] 21 Aug 2026 Safe deployment of increasingly capable models will likely 2026; Guo et al. 2026), but write access is also a risk. A…

saved by

related reading