✳flâneur — a map of the web's best reading

Automated Prompt Engineering for Forecasting

metaculus.com · 35 words · saved by 1 readers

As part of a grant for the Foresight Institute, Metaculus has been running some experiments to make open-source research tools for forecasting bots and generally improve forecasting bot performance. This last sprint, I created an automated prompt optimizer to test whether some prompts do better than others. These are preliminary findings that we think can be useful to other researchers and bot makers. We plan to further test and improve this approach and ideally make the optimizer a publicly usable tool. Below is a snapshot of early findings. In the graphs below, the first bar represents the "perfect predictor" or the score of the community prediction (this is only perfect in the sense that it maximizes expected baseline score). The next bar is the Control Group (using the control prompt), and the final two bars are the top two scoring prompts using the training question set (collected when the prompt optimizer was run). Error bars denote 90% confidence intervals calculated using a t-t

Automated Prompt Engineering for Forecasting Analysis of Automated Prompt Engineering for Forecasting by BenWilson • Jun 9, 2025 • Edited on Oct 13, 2025 • 10 min read 12 Follow Research News Artificial Intelligence Technology

Explore this link on the map →

related reading