Notes on Inference Integrity - by James Tillman - ForeWord
Claude Fable’s deliberately triggered sandbagging shows that training-time targets are, by themselves, insufficient to guarantee particular LLM behaviors.
This article was created by Forethought. See all our research on our website. Summary: Claude Fable’s deliberately triggered sandbagging shows that training-time targets are, by themselves, insufficient to guarantee particular LLM behaviors. To preserve the public’s reasonable confidence in LLM behaviors, LLM foundation model companies should take inference-time guarantees as seriously as their model specs. When the system card for Anthropic’s Fable was published on June 9th, the card noted that using Fable for “frontier LLM development” would run contrary to the terms of service for the…
saved by
related reading
- Alignment pretraining could backfire — LessWronglesswrong.com
- User awareness in frontier modelstransluce.org
- Investigating three real-world incidents in our cybersecurity evaluations \ Anthropicanthropic.com
- Teaching Claude Whyalignment.anthropic.com
- Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
- [2604.22082] Removing Sandbagging in LLMs by Training with Weak Supervisionarxiv.org
- Thoughts on Claude Fable's silent safeguards — LessWronglesswrong.com
- Teaching Claude why \ Anthropicanthropic.com
- Automated Researchers Can Subtly Sandbagalignment.anthropic.com
- Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
- [2604.16812] Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
- Fable is SOTA at CIFAR Speedrun (& specification gaming): lessons on AI R&D automationfulcrum.inc