A/B Test Calculator
Calculate statistical significance and determine the winner of an A/B test using Z-score and normal approximation.
Did this tool work for you?
How to use this calculator
Conversion rates for A and B are calculated, then pooled. The standard error accounts for both sample sizes. A Z-score above 1.645 gives 95% one-tail significance; above 1.96 gives 97.5%.
- 1
Enter the number of visitors and conversions for your control (Variant A) and test (Variant B).
- 2
The calculator computes conversion rates, relative uplift, and Z-score automatically.
- 3
Check Statistical Significance — 95% or higher means the result is unlikely due to chance.
- 4
Only call a winner once significance is 95%+; end the test early and you risk a false positive.
Frequently asked questions
What does 95% statistical significance mean?
It means there is only a 5% probability that the observed difference in conversion rates is due to random chance. Most teams use 95% as the minimum bar before declaring a winner.
How long should I run an A/B test?
Run the test for at least one full business cycle (usually 1–2 weeks) and until you reach the required sample size for 95% significance. Stopping early inflates false-positive rates significantly.
What is a good relative uplift?
Any positive, statistically significant uplift is worth acting on, but in practice most A/B tests that reach significance show 5–30% uplift. Very large uplifts (>50%) from small samples are usually noise.
One-tail or two-tail test?
This calculator uses a one-tail test (is B better than A?). If you need to detect differences in either direction, use a two-tail threshold of 97.5% significance instead of 95%.
A/B Test Calculator — Statistical Significance & Winner
How A/B Test Statistical Significance Works
Statistical significance tells you whether the conversion rate difference between two variants is real or just random noise. The Z-score measures how many standard deviations apart the two proportions are. A Z above 1.645 corresponds to 95% one-tail confidence — the conventional threshold for calling a test. Below 95% you should keep running the experiment; above it you can act on the result. The pooled standard error accounts for both sample sizes, which is why larger samples produce tighter confidence intervals and easier-to-reach significance.
Common A/B Testing Mistakes to Avoid
Peeking at results daily and stopping the moment you see 95% significance is one of the most common errors — it dramatically inflates the false-positive rate. Run your test for a predetermined sample size calculated before the experiment starts. Also avoid running more than one change at a time unless you use a full factorial design. Segment your results by device, traffic source, and user type to catch interaction effects that the overall numbers hide.
Learn more from an authoritative source:
WikipediaCPM Calculator
Calculate cost per thousand impressions (CPM), total ad cost, or total impressions for your campaign.
CPC & CTR Calculator
Calculate cost per click (CPC), click-through rate (CTR), and total clicks for PPC campaigns.
ROAS Calculator
Calculate Return on Ad Spend (ROAS) and determine how much revenue you earn per dollar spent on advertising.
Email Marketing Calculator
Calculate open rate, click-through rate, conversion rate, and revenue from email campaigns.
Results are estimates for informational purposes only and do not constitute professional financial, medical, legal, or technical advice. Read full disclaimer →