Cecilia Wood - Beyond vNM Self-modification and Reflective Stability - PIBBSS Symposium '23 - YouTube

Follow along using the transcript. Follow along using the transcript.

This is one of the talks given by a PIBBSS Fellow at the PIBBSS '23 Symposium. Abstract: Any tool we use to increase AI safety should be robust to self-modification. I present a formalism of an agent able to self-modify and argue this captures a wide range of goals or preferences beyond standard von Neumann-Morgenstern utility. In particular, I address mild or soft optimization approaches, such as quantilizers or maximising worse case over credal sets, by applying existing preference axiomatisations from economic theory which are more general than the axiomatisation given in the von…

saved by

• Bhagyesh Kumar

related reading

• Martin Soto - Constructing Logically Updateless Decision Theory - PIBBSS Symposium '23youtube.com
• Stanford CS25: Transformers United V6 I From Representation Learning to World Modelingyoutube.com
• Open-minded updatelessness — LessWronglesswrong.com
• What will be left for us to work on?normaltech.ai
• The Artificial Selftheartificialself.ai
• Neel Nanda on Mechanistic Interpretability: Progress, Limits, and Paths to Safer AI — EA Forumforum.effectivealtruism.org
• Inference, Diffusion, World Models, and More | YC Paper Clubyoutube.com
• Focus Dario Amodei (Google Brain)youtube.com
• Sparks of AGI: early experiments with GPT-4youtube.com
• My personal cruxes for working on AI safety — EA Forumforum.effectivealtruism.org
• AI Risk & Safety Concepts with Dario Amodei & Daniel Deweyyoutube.com
• The Two Cultures of AI Safety: Catastrophists vs Uniformitarians | Jesse Hoogland (Timaeus)youtube.com