A/B Testing
Create

A/B testing guide

How Long Should an A/B Test Run?

An A/B test should run long enough to collect the sample required for the effect you care about and long enough to avoid obvious timing bias. The answer depends on traffic, conversion rate and minimum detectable effect.

Start with sample size, not calendar days

Duration is the output of the calculation. First estimate how many observations each variant needs, then divide by the traffic that actually enters the experiment.

Smaller lifts take longer

Detecting a 5% relative improvement is much harder than detecting a 30% improvement. Teams often underestimate this and stop tests before they are informative.

Account for traffic allocation

If only half of site visitors enter the experiment, use that eligible traffic rather than total pageviews when estimating duration.

Avoid peeking-based decisions

Repeatedly checking a conventional fixed-horizon test and stopping as soon as the result looks favorable raises the chance of a false positive. Define the plan before starting.

When the duration is impractical

If the required run time is measured in months, test a larger change, choose a higher-traffic decision point, or use pre-launch preference research to narrow alternatives first.

Related tools and guides

Test duration calculatorSample size calculatorLow-traffic A/B testing

Have two options to compare?

Create a focused preference test and collect evidence before the decision gets expensive.

Create a survey