✳flâneur — a map of the web's best reading

A/B testing mistakes I learned the hard way - PostHog

posthog.com · 1,818 words · saved by 1 readers

Running experiments is equal parts powerful and terrifying. Powerful because you can validate changes that will transform your product for the better…

A/B testing mistakes I learned the hard way Lior Neu-ner Aug 06, 2024 Product engineers , Experiments Contents 1. Including unaffected users in your experiment 2. Only viewing results in aggregate (aka Simpson's paradox) 3. Conducting an experiment without a predetermined duration 4. Running an experiment without testing it first 5. Neglecting counter metrics 6. Not accounting for seasonality 7. Testing an unclear hypothesis 8. Relying too much on A/B tests for decision-making Conclusion Running experiments is equal parts powerful and terrifying. Powerful because you can validate changes that

Explore this link on the map →

related reading