[2002.06673] Performative Prediction

Abstract:When predictions support decisions they may influence the outcome they aim to predict. We call such predictions performative; the prediction influences the target. Performativity is a well-studied phenomenon in policy-making that has so far been neglected in supervised learning. When ignored, performativity surfaces as undesirable distribution shift, routinely addressed with retraining. We develop a risk minimization framework for performative prediction bringing together concepts from statistics, game theory, and causality. A conceptual novelty is an equilibrium notion we call performative stability. Performative stability implies that the predictions are calibrated not against past outcomes, but against the future outcomes that manifest from acting on the prediction. Our main results are necessary and sufficient conditions for the convergence of retraining to a performatively stable point of nearly minimal loss. In full generality, performative prediction strictly subsumes the setting known as strategic classification. We thus also give the first sufficient conditions for retraining to overcome strategic feedback effects.

Abstract:When predictions support decisions they may influence the outcome they aim to predict. We call such predictions performative; the prediction influences the target. Performativity is a well-studied phenomenon in policy-making that has so far been neglected in supervised learning. When ignored, performativity surfaces as undesirable distribution shift, routinely addressed with retraining. We develop a risk minimization framework for performative prediction bringing together concepts from statistics, game theory, and causality. A conceptual novelty is an equilibrium notion we call perfor

related reading

• GitHub - google-research/tuning_playbook: A playbook for systematically maximizing the performance of deep learning models. · GitHubgithub.com
• [2603.02202] Frontier Models Can Take Actions at Low Probabilitiesarxiv.org
• Forecasting: Principles and Practice (2nd ed)otexts.com
• Is Capability a Liability? More Capable Language Models Make Worse Forecasts When It Matters Mostarxiv.org
• [2605.11134] Spurious Correlation Learning in Preference Optimization: Mechanisms, Consequences, and Mitigation via Tie Trainingarxiv.org
• Learning to be Bayesian without Supervisionpapers.nips.cc
• The Parable of Predict-O-Matic — LessWronglesswrong.com
• Fundamentals of predictionmlstory.org
• Recursive forecasting: Eliciting long-term forecasts from myopic fitness-seekers — AI Alignment Forumalignmentforum.org
• Paper: Prompt Optimization Makes Misalignment Legible — LessWronglesswrong.com
• Pitfalls in Evaluating Language Model Forecastersarxiv.org
• Researchbactra.org