Detecting interference: An A/B test of A/B tests

At LinkedIn, most decisions are made using experiments. When we want to decide between two features, we test them against each other in the real world: we give feature A to a random set of members, feature B to another set, and we compare the results. Are users of feature A more engaged? Do they have a better experience with our products? If so, feature A wins.

Detecting interference: An A/B test of A/B tests Skip to main content LinkedIn respects your privacy LinkedIn and 3rd parties use essential and non-essential cookies to provide, secure, analyze and improve our Services, and to show you relevant ads (including professional and job ads ) on and off LinkedIn. Learn more in our Cookie Policy . Select Accept to consent or Reject to decline non-essential cookies for this use. You can update your choices at any time in your settings . Accept Reject A/B Testing/Experimentation Detecting interference: An A/B test of A/B tests Authored by Guillaume Sain

saved by

• Amos You

related reading

• A/B testing mistakes I learned the hard way - PostHogposthog.com
• The A/B Test: Inside the Technology That's Changing the Rules of Business | WIREDwired.com
• Internet Research Labirl.computer
• Mediumnir-boger.medium.com
• Berkson's paradox - Wikipediaen.wikipedia.org
• Spotify’s New Experimentation Platform (Part 1) | Spotify Engineeringengineering.atspotify.com
• The top 3 mistakes that make your A/B test results invalid | Conversionconversion.com
• What The Heck is... A Business Experiment?linkedin.com
• One-tailed vs Two-tailed Tests of Significance in A/B Testing | Analytics-Toolkit.comblog.analytics-toolkit.com
• 02 - Randomised Experiments — Causal Inference for the Brave and Truematheusfacure.github.io
• Research Designs | Nobanobaproject.com
• Networks, Crowds, and Markets: A Book by David Easley and Jon Kleinbergcs.cornell.edu