You Are Not Measuring What You Think You Are Measuring — LessWrong
Two laws of experiment design: First, you are not measuring what you think you are measuring. Second, if you measure enough different stuff, you migh…
x You Are Not Measuring What You Think You Are Measuring — LessWrong Best of LessWrong 2022 Practice & Philosophy of Science World Modeling Curated 443 You Are Not Measuring What You Think You Are Measuring by johnswentworth 20th Sep 2022 10 min read 45 443 Eight years ago, I worked as a data scientist at a startup, and we wanted to optimize our sign-up flow. We A/B tested lots of different changes, and occasionally found something which would boost (or reduce) click-through rates by 10% or so. Then one week I was puzzling over a discrepancy in the variance of our daily signups. Eventually I s
Explore this link on the map →related reading
- 2. A brief introduction to research design — Learning Statistics with Pythonethanweed.github.io
- How to Measure Anything — LessWronglesswrong.com
- Research Designs | Nobanobaproject.com
- Cargo Cult Sciencecalteches.library.caltech.edu
- Goodhart's law - Wikipediaen.wikipedia.org
- Goodhart's Law and Why Measurement is Hard — Ribbonfarmribbonfarm.com
- Self-experiment risk-taking interview, by Elizabeth van Nostrand, Gwern · Gwern.netgwern.net
- Science in a High-Dimensional World — LessWronglesswrong.com
- 1 Introduction – <span style='font-weight: 700'>Causal Inference</span><br/> <i style='color: #00b7ff'>The Mixtape</i>mixtape.scunning.com
- 2 Theories – Experimentologyexperimentology.io
- There are no statistics in the Kingdom of Godexperimental-history.com
- How Should We Critique Research? · Gwern.netgwern.net