How Does Nelio A/B Testing Decide on a Winning Variant?

Nelio A/B Testing identifies a winning variant when the collected data provides enough statistical evidence that one variant performs better than the others for the selected conversion goal.

How Nelio Determines the Winner

Nelio uses the G-Test to compare:

• The number of visitors each variant received
• The number of conversions each variant generated

TThe G-Test evaluates how different the observed results are from what you would expect if all variants performed the same. From this, Nelio calculates a confidence value, which indicates the strength of the statistical evidence that one variant performs better than the others.

What Confidence Is Needed to Declare a Winner?

There’s no universal confidence threshold that applies to every A/B testing scenario. By default, Nelio A/B Testing declares a statistically significant winning variant when the confidence level reaches 95%.

As more data is collected, the confidence value may change depending on the performance of the different variants.

Can I Stop a Test Before or After It Reaches 95% Confidence?

Yes. The statistical result helps you evaluate the performance of your variants, but the final decision about when to stop a test is yours.

You may choose to continue collecting data even after Nelio identifies a winner, or stop a test earlier if you have other reasons for doing so. However, results with lower confidence should be interpreted with greater caution.