A/B testing
Also called split testing, A/B test, split test
What is A/B testing?
A/B testing, also called split testing, is a method of comparing two versions of something, such as an ad, an email or a landing page, that differ in one element. The audience is split at random, each half sees one version, and you measure which gets more clicks, sign-ups or sales.
Changing only one element is what makes an A/B test useful: if version B wins, you know why. Tests with more than two versions are sometimes called A/B/n tests, and tests that change several elements at once in combinations are multivariate tests, which need much more traffic.
Results need enough data to be trusted. A version that leads after 20 clicks may lose after 2,000. Most ad platforms and testing tools report a confidence level or a "winner" once the difference is unlikely to be chance.
Why A/B testing matters
A/B testing turns marketing guesses into evidence. Over time, small proven improvements to headlines, images and offers add up to a lower CPA and a higher conversion rate.
It also settles debates inside a team. Instead of arguing over which headline is better, you run both and let customers decide.
How to run an A/B test
- Choose one metric that matters, such as cost per purchase or sign-up rate.
- Pick one element to change: the headline, the image, the offer or the call to action.
- Split the audience randomly so each version reaches similar people at the same time.
- Run the test until you have enough results for the difference to be reliable.
- Keep the winner, record what you learned, and test the next element.
A/B testing example
Crumb & Co, a bakery, tests two Instagram ad headlines with the same photo: "Fresh sourdough every morning" versus "Pre-order sourdough, skip the line". After a week, the second gets noticeably more orders at a lower cost per order, so it becomes the new control.
A/B testing FAQ
What is the difference between A/B testing and split testing?
They usually mean the same thing. Some people use "split testing" for comparing two whole different pages or campaigns, and "A/B testing" for changing one element, but most use them interchangeably.
How much traffic do I need for an A/B test?
Enough for each version to collect a meaningful number of conversions, not just views. The smaller the difference between versions, the more data you need to tell them apart.
Related terms
Creative testing is running several versions of an ad against each other to learn which image, video, hook or copy brings results most cheaply.
Conversion rate is the percentage of people who take a desired action, like buying or signing up, out of everyone who could have.
Click-through rate (CTR) is the percentage of people who click a link, ad or button after seeing it: clicks divided by impressions.
Ad copy is the written text of an ad: the headline, primary text, description and call to action that persuade people to click or buy.
A landing page is a web page built for one goal, like a signup or booking, that visitors reach from an ad, post or email.
Dynamic creative is an ad feature that mixes your images, videos, headlines and text into combinations and shows each person the ones likely to work.