Scoring rules part 3: Incentivizing precision – Unexpected Values
[This is Part 3 of a three-part series on scoring rules. If you aren’t familiar with scoring rules, you should read Part 1 before reading this post. You don’t need to read Part 2, but I think it’s pretty cool.] In 9th grade I learned the difference between accuracy and precision from a classroom poster. The poster looked something like this: Accuracy means that you’re unbiased: maybe you’ll never hit the bull’s eye exactly, but you aren’t consistently off in the same direction. Precision means hitting near the same spot (not necessarily the bull’s eye) every time. Generally speaking, precision without accuracy is pointless. Accuracy without precision… well, it depends. If you’re hunting rabbits, it doesn’t get you very far. If you’re conducting a survey, on the other hand, an accurate (unbiased) estimate is useful even if it’s not precise. Nevertheless, it’s better to be accurate and precise than just accurate. . Let’s say you’re forecasting the probability that it will rain one week f
Eric Neyman Math , Rationality , Research April 24, 2020 December 7, 2020 [This is Part 3 of a three-part series on scoring rules. If you aren’t familiar with scoring rules, you should read Part 1 before reading this post. You don’t need to read Part 2 , but I think it’s pretty cool.] In 9th grade I learned the difference between accuracy and precision from a classroom poster. The poster looked something like this: Accuracy means that you’re unbiased: maybe you’ll never hit the bull’s eye exactly, but you aren’t consistently off in the same direction. Precis
Explore this link on the map →saved by
related reading
- Scoring rule - Wikipediaen.wikipedia.org
- Imprecise Probabilities (Stanford Encyclopedia of Philosophy)plato.stanford.edu
- Probabilities are not the right concept — LessWronglesswrong.com
- Models Don't "Get Reward" — LessWronglesswrong.com
- AlgZoo: uninterpreted models with fewer than 1,500 parameters — LessWronglesswrong.com
- How hard is it to inoculate against misalignment generalization? — LessWronglesswrong.com
- Bits per Spike as a Betting Game · neurostatsblogneurostatsblog.github.io
- Recent Frontier Models Are Reward Hacking - METRmetr.org
- [2605.12474] Reward Hacking in Rubric-Based Reinforcement Learningarxiv.org
- A Technical Explanation of Technical Explanation – Eliezer S. Yudkowskyyudkowsky.net
- Brier score - Wikipediaen.wikipedia.org
- Recursive forecasting: Eliciting long-term forecasts from myopic fitness-seekers — AI Alignment Forumalignmentforum.org