On Anthropic's Sleeper Agents Paper - by Zvi Mowshowitz

thezvi.substack.com · 10,079 words · saved by 1 readers

Scott Alexander also covers this, offering an excellent high level explanation, of both the result and the arguments about whether it is meaningful. You could start with his write-up to get the gist, then return here if you still want more details, or you can read here knowing that everything he discusses is covered below. There was one good comment, pointing out some of the ways deceptive behavior could come to pass, but most people got distracted by the ‘grue’ analogy. Right up front before proceeding, to avoid a key misunderstanding: I want to emphasize that in this paper, the deception was introduced intentionally. The paper deals with attempts to remove it. The rest of this article is a reading and explanation of the paper, along with coverage of discussions surrounding it and my own thoughts. Paper Abstract: Humans are capable of strategically deceptive behavior: behaving helpfully in most situations, but then behaving very differently in order to pursue alternative objectives wh

The recent paper from Anthropic is getting unusually high praise, much of it I think deserved. The title is: Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. Scott Alexander also covers this, offering an excellent high level explanation, of both the result and the arguments about whether it is meaningful. You could start with his write-up to get the gist, then return here if you still want more details, or you can read here knowing that everything he discusses is covered below. There was one good comment, pointing out some of the ways deceptive behavior could…

On Anthropic's Sleeper Agents Paper - by Zvi Mowshowitz

related reading