flâneur

How Should We Critique Research? · Gwern.net

gwern.net · 8,285 words · saved by 1 readers

Criticizing studies and statistics is hard in part because so many criticisms are possible, rendering them meaningless. What makes a good criticism is the chance of being a ‘difference which makes a difference’ to our ultimate actions. epistemology, Bayes, scientific bias, causality, decision theory Sci­en­tific and sta­tis­ti­cal re­search must be read with a crit­i­cal eye to un­der­stand how cred­i­ble the claims are. The Re­pro­ducibil­ity Cri­sis and the growth of meta-science have demon­strated that much re­search is of low qual­ity and often false. But there are so many pos­si­ble things any given study could be crit­i­cized for, falling short of an un­ob­tain­able ideal, that it be­comes un­clear which pos­si­ble crit­i­cism is im­por­tant, and they may de­gen­er­ate into mere rhetoric. How do we sep­a­rate fatal flaws from un­for­tu­nate caveats from spe­cious quib­bling? I offer a prag­matic cri­te­rion: what makes a crit­i­cism im­por­tant is how much it could change a re­su

--- title: How Should We Critique Research? description: Criticizing studies and statistics is hard in part because so many criticisms are possible, rendering them meaningless. What makes a good criticism is the chance of being a 'difference which makes a difference' to our ultimate actions. created: 2019-05-19 modified: 2019-07-07 status: finished confidence: highly likely importance: 8 css-extension: dropcaps-kanzlei ... > Scientific and statistical research must be read with a critical eye to understand how credible the claims are. The Reproducibility Crisis and the growth of meta-science h

related reading