Séb Krier on X: "The way people discuss AI incidents is pretty important and I'm a bit concerned we're sleepwalking into a bad world littered with bad abstractions. How you label something is often a lossy compressions of a causal model, so the words you use affects which hypotheses people update https://t.co/IsMLwiMhR4" / X
The way people discuss AI incidents is pretty important and I'm a bit concerned we're sleepwalking into a bad world littered with bad abstractions. How you label something is often a lossy compressions of a causal model, so the words you use affects which hypotheses people update
Séb Krier @sebkrier The way people discuss AI incidents is pretty important and I'm a bit concerned we're sleepwalking into a bad world littered with bad abstractions. How you label something is often a lossy compressions of a causal model, so the words you use affects which hypotheses people update toward and by how much. Reward hacking has been known about for a long time, and is a serious problem with models - and I think this is the expression we should use when discussing stuff like the Hugging Face incident (assuming that was, in fact, a reward hacking incident). But some people will fre
saved by
related reading
- Séb Krier (@sebkrier) on Xx.com
- Your AIs don't do what you want. This is really badrewardhacking.org
- Are we existentially threatened by the type of AI misalignment seen in the OpenAI Hugging Face attack? — LessWronglesswrong.com
- What failure looks like — LessWronglesswrong.com
- What failure looks like — AI Alignment Forumalignmentforum.org
- The Case Against AI Control Research — LessWronglesswrong.com
- AI Control: Improving Safety Despite Intentional Subversion — LessWronglesswrong.com
- Cognitive Security as an AI Safety Cause Area — LessWronglesswrong.com
- On AI Safety Jargonturningflukes.substack.com
- Séb Krier on X: "Half baked thought: In discussions about AI, claims are often made about both capabilities and societal effects, and in practice the boundary is pretty blurry. Different personality types and professions see things through x.com
- Simulated Users & Sad LLMs1a3orn.com
- Séb Krier on X: "(I know I'm a stuck record) An important assumption in AI discourse is that sufficiently capable generalist *models* are the main event. Get the model smart enough, and it more or less does everything. Value creation, compex.com