Today I bring you a new A/B test that we have been running for a few months. As always within this series of posts, we are going to see in detail the improvement hypothesis that we have made, its application by creating variations of a page on our website, and the results obtained.
It has been a while since we designed the landing of our company using our current WordPress theme. And this landing is a page that we have not changed since then.
Today we are going to see an alternative version of our landing with many changes compared to the current one. In addition, we will check if this new version really works better or not looking at real data.
Testing Hypothesis
After studying the different sections included in our landing page, we have decided to give the page a complete facelift to see if this improves the results. Its goal is that the traffic that reaches this page ends up visiting the landing pages of our products. With this in mind, we are going to design an alternative version that is more visual, more concise, and does a better job at highlighting our products.
The hypothesis we want to validate is that a simpler, more visual, and more direct landing should work better to increase conversion. The alternative version that we propose is the one you can see on the right in the following gallery:
Nelio’s Landing. Simplified version of Nelio’s landing.
Among the list of changes we are going to test, it stands out that the first section of the page will now occupy the first fold. Then there is a section with the two main plugins we sell, accompanied by an illustration and a call to action.
After that, we have reduced the amount of text we display in our testimonials and we have kept the list of logos of some of the clients that we have.
Regarding the sections that show our blog posts, we believe that now they get too much relevance, so in the alternative version we have reduced them.
Finally, we end with another image following the same style but showing our company team and a last call to action to visit the landing page of Nelio Content.
As you can appreciate, the version that we are going to test is simpler and more visual. We love its design better than what we have now. But that doesn’t matter at all. What really matters is to see if it really works better or not.
Definition of the A/B test
To check if the alternative version that we have created for our page works better or not, we have to use an A/B test.
Access the WordPress Dashboard and go to the Nelio A/B Testing menu. Then go to the Tests menu and create a new A/B test of pages. In the edit view of the new test select the original page and create a variation, which you can then edit to implement the new design.
The goal we want to measure is the amount of visitors from this page that go to the pages of our products. Every time a visitor sees our landing and visits one of the product pages, we will count a conversion.
Once we have everything ready, we start the test. From that moment on, our A/B testing tool is in charge of dividing the traffic between the original version and the alternative version so that half of the visitors see one and the other half see the other.
We just have to wait for the results to accumulate to finally decide if our hypothesis was valid or not.
Nelio A/B Testing
I was very impressed by the quality of this plugin, how easy it was to set up, and the outstanding support Nelio provides. I highly recommend Nelio A/B Testing.
Josette Millar
Analysis of the A/B testing results
After a month and ten days, the A/B test tells us that it has found a version that is clearly better than the other. You can see the results in the following screenshot:
Surprisingly for us, the version that has worked best is the one we have been using on our page so far. The alternative version, although we liked it better, works almost 11% worse than the current one. And statistical confidence is high enough to trust that the results obtained are not the result of chance.
This shows that personal opinions are useless when it comes to web design. A page that we thought was better is clearly not when we take a look at the real data of our audience.
What has gone wrong here? Perhaps it was the first section occupying the entire first fold of the web, or the arrangement of the new content with the images, or the colors, … There are many variants in the alternative version. Which leads us to think that since this radical change has not worked, we could continue with smaller changes, like the ones we tested in this other test.
If we had not done the A/B test we would have changed the page directly without testing and our conversions would have dropped by 11%. And this turns into fewer visitors who see our products and end up buying on our website.
Therefore, we will continue using the current page. But we will also keep looking for variations that may work better than the current one. Because that’s what it’s all about: seeking constant improvement is what we must do with our website.
Leave a Reply