[2605.24229] How Well Do Models Follow Their Constitutions?
Abstract:Frontier AI developers now train models against long written behavioral specifications, such as Anthropic's constitution (Anthropic, 2025a) and OpenAI's Model Spec (OpenAI, 2025a), integrated into post-training via methods like character training (Anthropic, 2024) and deliberative alignment (Guan et al., 2024). These documents serve a governance function, but it is unclear how well models actually follow them under adversarial, multi-turn pressure similar to what they would face in real-world deployment. We propose a multi-method audit pipeline that treats each lab's published specification as an auditable target: it decomposes the specification into atomic testable tenets (205 for Anthropic, 197 for OpenAI), generates multi-turn adversarial scenarios with the Petri auditing agent (Anthropic, 2025b), runs a modified SURF-style rubric search (Murray et al., 2026) to catch shallow single-turn failures Petri misses, validates flagged transcripts against the relevant specification, and compares the findings against the lab's own published system card. Applying the pipeline across seven models per specification, we find that models follow their own lab's specification substantially better with each generation. On Anthropic's constitution, the Claude family falls from a 15.0% violation rate (Sonnet 4) to 2.0% (Sonnet 4.6); on OpenAI's Model Spec, the GPT family falls from 11.7% (GPT-4o) to 3.6% (GPT-5.2 medium reasoning), with the severity ceiling falling from 10/10 to 7/10. We cannot externally isolate whether these gains come from specification-specific training, broader post-training improvements, or evaluation awareness. Remaining failures cluster around operator-imposed personas under AI-identity questioning, irreversible action in agentic deployments, and fabricated quantitative claims with false precision.
How Well Do Models Follow Their Constitutions?1 Arya Jakkli† Senthooran Rajamanoharan Neel Nanda CentraleSupélec Google DeepMind Google DeepMind aryajakkli2002@gmail.com † Work conducted during the ML Alignment &…
saved by
related reading
- [2605.27288] It's Not Always Sycophancy: Measuring LLM Conformity as a Function of Epistemic Uncertaintyarxiv.org
- [2606.05433] Zero knowledge verification for frontier AI training is possiblearxiv.org
- [2607.14345] Value Leakage: An LLM's Answers Are Silently Shaped by Its Own Valuesarxiv.org
- (a) Anthropic constitution.arxiv.org
- [2507.11473] Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetyarxiv.org
- How well do models follow their constitutions? — LessWronglesswrong.com
- Teaching Claude Whyalignment.anthropic.com
- [2604.16812] Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
- Constitutional AI: Harmlessness from AI Feedbackarxiv.org
- How confessions can keep language models honest | OpenAIopenai.com
- [2312.06942] AI Control: Improving Safety Despite Intentional Subversionarxiv.org
- [2602.05910] Chunky Post-Training: Data Driven Failures of Generalizationarxiv.org