The duration for which you need to run your A/B testing tests depends on several factors such as follows:
- The complexity of the test. The more complex the test (more variants, with less differences between them, …), the longer you will need to run the test to get significant results.
- The number of visitors who participate in the test. The more number of people you test, higher is the reliability of the test results.
Stopping a test too early may result in not getting meaningful results, while keeping a test running for too long could make you lose conversions and sales due to poor performing variant in the test.
We’ve investigated the amount of time the test of our customers have been running. In average, all the test were running for less than a month and most of them didn’t surpass two weeks. In our opinion, that’s not too much time regarding the benefits you can get in return! Also, if you fail to find a winning combination within this amount of time, you may conclude that the variants do not stand a chance to beat the original version.