Nelio A/B Testing uses the G-test statistic for computing the statistical significance of the results of a test. The G-test statistic is a measure of how much overall variant there is from an ideal prediction that you would expect if all versions were the same. It produces a confidence value that tells you how “trustable” is the fact that one alternative is better than the other/s.
A high confidence value tells you that you can be certain that choosing the best variant is the best option to improve your conversion rate. A low confidence value means that with the current results, even if one variant has so far a higher conversion rate than another, this could still be due to random effects so you have to interpret the results with caution.
We recommend waiting for at least a 90% confidence value in order to be sure to choose the best variant.
The more visitors you have and the bigger is the observed difference between the variants, the higher is the confidence value.