Who is right: the director or the marketer
“Let’s make the button bigger and red” — “no, that looks cheap”. Such arguments in the meeting room are endless because everyone has their own intuition. An A/B test closes the question with facts: we show half of visitors variant A, the other half variant B, and see where there are more leads. The winner is not the loudest but the numbers.
What an A/B test is at its core
You take a page, make a second variant with one change (heading, button, form, offer) and show the variants to random visitors in parallel. The tool measures the conversion of each and tells you whether there is a statistically significant difference. The key phrase is “one change”: if you change the heading, image and button at once, you will not understand what exactly worked.
Start with a hypothesis, not a button
A bad test: “let’s repaint the button and see”. A good one starts with a hypothesis based on data. First look at where the funnel leaks — in Metrica via Session Replay and click maps — and only then formulate:
“Visitors do not reach the form because it is at the very bottom. If we raise it above the fold, conversion will grow.”
Such a hypothesis can be tested and confirmed or refuted. This is part of systematic work on site conversion.
How much traffic you need
The main mistake of beginners is to stop a test after two days with the conclusion “B won by 30%”. On small numbers that is chance. Guidelines:
- You need at least several hundred conversions per variant, not visitors.
- At a 2% conversion and 100 leads a week, a test for a noticeable improvement runs 2–4 weeks.
- Under ~1,000 visits a week per page, A/B tests are almost pointless — the statistics will not accumulate. In that case do not test but implement proven practices and watch the week-over-week trend.
How not to fool yourself
- Wait for significance. Tools show “confidence” — do not stop before 95%.
- Test full weeks. Behavior on weekdays and weekends differs; a test cut off mid-week lies.
- One change at a time. Otherwise you will not understand the cause.
- Do not peek and do not stop at the first lead — this is the classic trap, the numbers are still jumping.
What to test first
The biggest effect comes from large elements, not button shades:
- the heading and offer on the first screen;
- the length and fields of the form (a short one almost always wins);
- the presence and format of the price;
- the main call to action;
- the page structure — what to show higher up.
Tools
To start: Google Optimize is shut down, but there are Yandex experiments (via Metrica and Variocube), VWO, AB Tasty, Optimizely. For simple tests of headings and buttons, the built-in experiments in site builders and ad accounts are enough. Choose by traffic and budget — at the start free options are sufficient.
Related articles
Conclusion
An A/B test turns marketing from the art of arguing into the science of measuring. But it only works with enough traffic and discipline: a hypothesis from data, one change, full weeks, 95% significance. Low traffic — do not test, implement the proven. We will help you build a data-driven conversion-growth process — get in touch.
