Why Do I Need a Minimum Number of Tested Page Views to Compute the Results? How Many Are Required?

A/B testing requires enough data before its results can be evaluated reliably. With too few visitors, differences between variants may simply reflect normal random variation rather than a meaningful difference in performance.

Why Is a Minimum Sample Size Needed?

Imagine this scenario:

• Only two visitors see Variant B
• Both convert, resulting in a 100% conversion rate.

Can we conclude that Variant B will convert at 100% for all visitors? No. There simply isn’t enough data to draw a reliable conclusion.

Collecting more data helps Nelio A/B Testing determine whether the differences observed between variants provide enough statistical evidence to identify a winner.

With too little data:

• Confidence remains low.
• Results may change substantially as new data arrives.
• A reliable winner can’t yet be identified.

Minimum Requirements in Nelio A/B Testing

Nelio A/B Testing uses minimum sample size requirements to avoid drawing conclusions from tests with too little data.

1. Minimum Tested Pageviews to Compute Results

By default, Nelio A/B Testing requires 100 tested page views before it starts computing the statistical results of a test.

Until this minimum is reached, there isn’t enough data to perform a meaningful statistical analysis.

2. Minimum Tested Pageviews To Declare a Winner

Even after Nelio starts computing the results, it won’t declare a winning variant until the test has collected at least 500 tested page views.

Reaching 500 tested page views doesn’t automatically mean that your test has a winner. Nelio still needs enough statistical evidence to determine that one variant performs better than the others.

How Many Tested Page Views Will My Test Need?

There isn’t a fixed number of page views that guarantees a conclusive result.

The amount of data required depends on factors such as:

• The conversion rate of your variants.
• The difference in performance between them.
• The number of variants being tested.

The Results page provides information to help you estimate how much data your test may need. In Nelio A/B Testing 8.5 and later, this includes an estimate of the page views required to reach 90% confidence.

This estimate is intended as a guide and may change as new data is collected. Reaching that estimate doesn’t necessarily mean Nelio will declare a winner.

Can I Change These Thresholds?

Yes. In Nelio A/B Testing → Settings you can adjust the minimum required sample size according to your needs.

Keep in mind that lowering the minimum allows Nelio to evaluate tests with less data, but results based on smaller samples should be interpreted with greater caution.