Resorting Media Ratings · Gwern.net
gwern.net · 4,929 words · saved by 1 readers
Commandline tool providing interactive statistical pairwise ranking and sorting of items
--- title: Resorting Media Ratings description: Commandline tool providing interactive statistical pairwise ranking and sorting of items created: 2015-09-07 modified: 2018-08-15 status: finished confidence: likely importance: 3 css-extension: dropcaps-yinit ... > User-created datasets using ordinal scales (such as media ratings) have tendencies to drift or 'clump' towards the extremes and fail to be informative as possible, falling prey to ceiling effects and making it difficult to distinguish between the mediocre & excellent. > > This can be counteracted by rerating the dataset to create a un
related reading
- How Not To Sort By Average Rating – Evan Millerevanmiller.org
- Automated Weak-to-Strong Researcheralignment.anthropic.com
- Essays · Gwern.netgwern.net
- Bradley–Terry model - Wikipediaen.wikipedia.org
- Book Reviews · Gwern.netgwern.net
- About Gwern · Gwern.netgwern.net
- DataRater: Meta-Learned Dataset Curationarxiv.org
- A statistical approach to model evaluations \ Anthropicanthropic.com
- Whole-History Rating: A Bayesian Rating System for Players of Time-Varying Strengthremi-coulom.fr
- How Should We Critique Research? · Gwern.netgwern.net
- Quantifying Truesight With SAEs · Gwern.netgwern.net
- [2207.00076] Efficient computation of rankings from pairwise comparisonsar5iv.labs.arxiv.org