flâneur

p < 0.05 considered harmful - by Richard Demsyn-Jones

simplicityissota.substack.com · 1,596 words · saved by 1 readers

The unreasonable persistence of an old standard

p < 0.05 considered harmful The unreasonable persistence of an old standard Richard Demsyn-Jones Apr 10, 2023 2 Share I spend a lot of time thinking about decision criteria for shipping experimental features. The topic could use a little more attention. Maybe it’s higher up the Maslow’s hierarchy of experimentation, somewhere above the prerequisites of actually running randomized experiments, constructing them optimally, and analyzing them correctly. Yet it is still important to actually ask: under what conditions do we ship? When it comes to this question, there’s a magic number that breaks t

related reading