the trouble with making predictions as an agent
substack.com · 660 words · saved by 1 readers
a forecast probability is an endorsement of the event staying at that likelihood
This piece has been replaced by Deep Anti-Determinism. Go read that instead! Say there’s some event you care about, E. You want E to happen. Initially, your life plan is π_0, and you estimate P(E | π_0) < 1. But since you want E to happen, you come up with a new plan, π_1, to make E more likely. Now you calculate P(E | π_1) and check: are you satisfied? If not, you devise a new plan, π_2, and calculate P(E | π_2). If each plan is better than the last, you’ll obtain a monotone increasing sequence of probability forecasts, bounded above by 1. By Monotone Convergence Theorem, this sequence…
saved by
related reading
- Deep Anti-Determinismlydianottingham.substack.com
- Towards a scale-free theory of intelligent agencymindthefuture.info
- The AI Superforecasters Are Here - by Scott Alexanderastralcodexten.com
- Embedded Agency (full-text version) — LessWronglesswrong.com
- Everything is Predictionblog.cassi-ai.com
- Optimality is the tiger, and agents are its teeth — LessWronglesswrong.com
- Towards a scale-free theory of intelligent agency — AI Alignment Forumalignmentforum.org
- Reinforcement Learning in Newcomblike Problemsproceedings.neurips.cc
- Recursive forecasting: Eliciting long-term forecasts from myopic fitness-seekers — AI Alignment Forumalignmentforum.org
- 1406.2148arxiv.org
- Open-minded updatelessness — LessWronglesswrong.com
- [1902.09469] Embedded Agencyarxiv.org