A structured protocol for elicitation experiments | AISI Work

Calibrating AI risk assessment through rigorous elicitation practices.

Identifying dangerous AI capabilities requires rigorous evaluation of models at the upper limit of their abilities—a process that is far from straightforward. There are many ways to enhance the performance of a model after it has been trained, from carefully crafted prompts to external tool access. We need evaluation practices that uncover the full range of what models can be used to achieve. These practices fall under the umbrella of elicitation, which plays a critical role in safety and security work. Done well, it helps us avoid underestimating what models can do, thereby improving the…

saved by

• Tazik Sh

related reading

• Research Areas in Methods for Post-training and Elicitation (The Alignment Project by UK AISI) — AI Alignment Forumalignmentforum.org
• Recommendations for Technical AI Safety Research Directionsalignment.anthropic.com
• Incident Report: unsanctioned agent behaviour during cyber testing | AISI Workaisi.gov.uk
• The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
• Model evals for dangerous capabilities — LessWronglesswrong.com
• Making sense of the UK’s AI Security Instituteadalovelaceinstitute.org
• How do we (more) safely defer to AIs? — LessWronglesswrong.com
• Research Areas in Benchmark Design and Evaluation (The Alignment Project by UK AISI) — AI Alignment Forumalignmentforum.org
• Research Areas in AI Control (The Alignment Project by UK AISI) — AI Alignment Forumalignmentforum.org
• Ten AI safety projects I'd like people to work onthirdthing.ai
• AI models can be dangerous before public deploymentmetr.org
• Self-exfiltration is a key dangerous capabilityaligned.substack.com