What is A/B Testing?
A/B testing shows two versions of something to comparable groups and measures which performs better. Half your visitors see version A, half see B, and the difference in conversion tells you which to keep. Straightforward in principle. In practice most tests get called far too early, on far too little data, to mean very much.
Split Testing
How A/B testing actually works
Traffic is split randomly between variants, and because assignment is random the two groups should be alike in every respect except the change you made, which is what lets you attribute any difference in outcome to that change rather than to luck.
Decide three things before you start: the single metric you are judging on, the sample size needed to detect an effect worth having, and the date you will stop. Writing those down in advance is what separates a test from a story told afterwards. Then run full weeks, because weekday and weekend behaviour differ.
Where teams get A/B testing wrong
Stopping when the result looks good. Variant B leads on day three, somebody declares victory, and the difference has evaporated by day ten, because early leads in split tests are overwhelmingly noise. Set the end date first.
Second, testing several things at once, since changing the headline, the image and the button together means a win tells you nothing about which element caused it and you cannot repeat the success.
Third, running tests on traffic that cannot support them. A page seeing four hundred visitors a month cannot detect a small improvement in any sensible timeframe, so make the change you believe in and spend your effort on qualitative research instead.
What good looks like in India
Most Indian D2C brands lack the volume for continuous testing on every page, which is fine. Concentrate tests where traffic is heaviest, usually the main product page and checkout, and make variants meaningfully different so a real effect is large enough to actually see.
For ad testing, Meta’s built-in experiments handle the split cleanly and avoid the audience overlap you get from duplicating campaigns by hand. Test one variable there too: the hook, the format, or the offer. Not all three in one week.
Related terms: AEO (Answer Engine Optimisation) · AI Overviews · Anchor Text.
Where this shows up in the work: AI & Data Analytics · Full glossary.
A/B Testing — questions, answered.
At least two full weeks, and long enough to reach the sample size you calculated in advance. Always run complete weeks so weekday and weekend behaviour are represented equally across both variants rather than skewing one.
Not reliably for small effects. With limited traffic, test only large changes, accept longer run times, and lean harder on session recordings, surveys and support conversations to decide what is worth changing in the first place.
Last updated 2026-08-08
Ready to replace guesswork with a growth engine?
Book a 30-minute strategy call. We’ll show you exactly where your funnel is leaking, before you spend a dollar.