A/B test
Two-variant comparison trial
An A/B test is an experiment comparing two versions of something—a webpage, email, advertisement—to see which performs better.
Users are randomly shown version A or version B, and their responses are measured: We ran an A/B test on the signup button color.
The method originated in direct marketing but now dominates digital product development.
Also called split testing.
- p-valueProbability under null hypothesis
- hypothesisTestable proposed explanation
- control groupUntreated subjects for comparison
- dependent variableOutcome being measured
- heatmapsColor-coded data visualizations
randomization · click-through rate · variance · statistical significance · personalization · segmentation · sample size · confidence interval · confounding variables · landing page · replication · standard deviation · lead generation · customer retention · trend analysis · conversion rate · experiment · winner takes all