Data on forecasting accuracy across different time horizons and levels of forecaster experience — Rethink Priorities

rethinkpriorities.org · 4,783 words · saved by 1 readers

This post draws a lot on niplav's Range and Forecasting Accuracy, not least for much of the code used to extract the PredictionBook forecasts, and also in identifying the most promising sources of useful data. I think that this post is probably most useful to individuals making forecasts being aware of common failure modes and attempting to learn from them, and informing decision makers about these failure modes also, rather than attempting to provide those looking to use forecasts with e.g. a transform they should apply to long term forecasts. There has been a great deal of interest in forecasting in the EA community in recent years, particularly with the prominence of longtermist thinking. It is clearly of great interest that we be well equipped to make predictions about future events, and to understand the accuracy and failure modes of such predictions. Additionally, many of the questions we care most about will have long time horizons, therefore any evidence we can gain which helps

Share Written by: Charles Dillon Key Points Forecasting well is a valuable skill for many purposes and people, including for EA organisations aiming to identify which areas they should focus on and what the outcomes of various initiatives would be. There is a limited public record of people making scored forecasts over time horizons greater than ~1 year. Here I use data from PredictionBook and Metaculus to study performance of predictions over different time horizons. I also looked at performance between users with different levels of forecasting practice. When looking at individual predictors

Data on forecasting accuracy across different time horizons and levels of forecaster experience — Rethink Priorities

related reading