Today we present the new A/B test we’ve been running on our website to see if reducing the text load on one of our pricing pages gets us better results.
As always within this series of posts, we are going to see in detail the improvement hypothesis we made, its application creating variations of a page on our website, and the results we obtained.
Testing Hypothesis
We’ve been mulling over the pricing pages for our premium WordPress plugins for a while now. It is clear that if you want to improve the results of your website to increase the benefits you get from it, you have to test changes to see if any of them improve what you have.
We recently tested variations in the colors and sizes of the action buttons. And today we are going to continue the tests on the pricing page of our premium plugin Nelio A/B Testing to see if simplifying it makes it more efficient.
Our original pricing page has the pricing table in the first section of the page. In this pricing table (which you can see here), each of the three pricing plans has a small section with a summary of the features in a list with small icons to the left of each item.
Next, we have a more detailed section with the main features of our plugin. But this is a bit repetitive, since the pricing table itself already has a list of characteristics. Finally, we have a section with testimonials. You can see it all in the following screenshot:
After thinking about these initial blocks a few times, we believe that simplifying them can be good. There is a lot of text in these feature and testimonial blocks and some concepts are repeated.
The current visitor to the web usually does not read everything. There is too much text that will only be partially scanned by the visitor’s eyes. Therefore, if we simplify these blocks, will we improve the results of the pricing page?
Our alternative proposal for this pricing page is more visual. We have removed the first block of features to use an image together with a message that is more aimed at implying the benefits you can get with our plugin.
As for the list of testimonials, we’ve kept it but we’ve reduced the text of each of them to make it easier to read. In addition, we have added a header highlighting that our plugin now allows visitor segmentation, one of the most demanded features that also was under-promoted. The visual result can be seen in the following screenshot:
So, our hypothesis is that by reducing the text that appears on a pricing page and changing it for more visual sections, the results will be better.
We are going to actually test whether this is so and, therefore, our hypothesis is valid or if, on the contrary, we’re wrong.
Definition of the A/B test
To test the changes correctly, we are going to create an A/B test to test the version of the current page against the version of the page with the simplified sections.
We go to the Nelio A/B Testing menu on our WordPress dashboard and create a new A/B test of pages. The following screenshot shows the user interface for defining such a test:
We select the pricing page we want to test. Then we create a variant and we edit it to modify the content of the page according to our previous hypothesis.
Regarding the goals we want to measure in our test, we defined five:
- Clicks on the purchase button of any plan.
- Clicks on the purchase button of the basic plan.
- Clicks on the purchase button of the professional plan.
- Clicks on the purchase button of the enterprise plan.
- Actual purchases of any plan.
With all this, we are ready to start the test and see the results. Remember that you do not have to do anything else on your part. The tool you use to do A/B testing (in our case, Nelio A/B Testing) will be in charge of dividing the traffic that reaches your page so that one half sees the original version and the other half sees the variation with the changes.
Nelio A/B Testing
I was very impressed by the quality of this plugin, how easy it was to set up, and the outstanding support Nelio provides. I highly recommend Nelio A/B Testing.
Josette Millar
Analysis of the A/B testing results
We started this test after the summer and we have had it running until the end of the year. Let’s see the results of each of the goals in detail.
First of all, if we focus on the clicks that visitors made on the purchase buttons of any plan that appears on the pricing page, we see that the simplified version of the page performs 17.5% better than the current one.
However, the statistical confidence level has not yet reached the desired level (above 90%), although it’s quite high. We will have to see the results of the other goals to make a decision about it.
If we look at the second goal, which only takes into account the clicks to the cheapest plan, we see the improvement is way clearer. The simplified variant is 50% better than the current one, with high statistical confidence. Therefore, there are no doubts here: this version is way better than the one we had before.
If we look at the results in terms of clicks to the intermediate plan, the results are not conclusive as there is little statistical variation in the analyzed sample of our visitors.
And this happens even more if you look at the clicks to the most expensive plan that we have for sale on the pricing page. The data is too small to draw any conclusions.
Finally, if we focus on the sales that we have had on this page, the result is quite tight and we didn’t find a clear winner. However, the alternative is slightly better…
After viewing the results and analyzing them from each specific goal, we have concluded that the simplified version of the pricing page is better in terms of getting more clicks on the purchase button of the basic plan (which is also the most sold plan, as we know from the data provided by our payment system).
As for the other goals, we cannot draw further conclusions from any of them. Therefore, we have decided to use the simplified variant as the final version of the pricing page from now on. We want more clicks on the best-selling plan without making the rest worse, and statistics tell us that this variant gives us that.
We will continue to analyze the data on this page in the coming months to verify that the trend continues. And to you, dear reader, we recommend that if you manage a web page, you put it under test using A/B testing. Only by doing this you will find its best version.
Featured Image by James Wainscoat on Unsplash.
Leave a Reply