Medical AI Safety: Doing it wrong. – Lauren Oakden-Rayner
laurenoakdenrayner.com · 4,028 words · saved by 1 readers
Medical AI has a safety problem; we know for a fact our testing isn’t reliable. We’ve seen how this plays out before.
Last post , I said we have a safety problem in medical AI. I even suggested that it is bad enough that it could lead to a tragedy. It may not seem that way. Numerous papers are published every week, showing deep learning systems achieving impressive results on medical tasks. Products are being approved by the FDA, and companies are starting to sell them to healthcare providers. I’m going to argue we are doing three specific things wrong: we assume good experimental performance equals good clinical performance we assume good overall performance equals good subtask performance we are not v
saved by
related reading
- Medical AI Safety: We have a problem. – Lauren Oakden-Raynerlaurenoakdenrayner.com
- AI for medicine is overhyped right now - by Dan Eltonmoreisdifferent.substack.com
- The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
- AI Safety Seems Hard to Measurecold-takes.com
- How do we (more) safely defer to AIs? — LessWronglesswrong.com
- Current AIs seem pretty misaligned to me — LessWronglesswrong.com
- AI Control: Improving Safety Despite Intentional Subversion — LessWronglesswrong.com
- AI Safety for Fleshy Humans: a whirlwind touraisafety.dance
- AI models can be dangerous before public deploymentmetr.org
- Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
- Why AI alignment could be hard with modern deep learningcold-takes.com
- The algorithm will see you now - Works in Progress Magazineworksinprogress.co