Today we present a new A/B test that we have been running on our website to see which page layout gets more subscriptions to our premium plugin Nelio A/B Testing.
As usual within the series of post of the test of the month, we are going to see in detail the hypothesis for improvement that we have made, its application by creating variations of a page on our website, and the results obtained.
Hypothesis to Test
A/B testing your pricing page can provide valuable information about the behavior of your website visitors, as well as improve your conversion rate and revenue. In our case, we wanted to work on Nelio A/B Testing’s pricing page and, in particular, to highlight its features.
In the original version, there’s a block with the different Nelio A/B Testing plans and their basic features, followed by a block with a promotional image and message, and then another block of three testimonials.
Then comes a detailed table comparing the features included in each of the plans we offer:
And, finally, there’s a section where we emphasize that we are WordPress VIP technological partners and that you are guaranteed the return of your subscription during the first 30 days. Finally, the page ends with a section on FAQs and the possibility to contact us.
After thinking about this page several times, we hypothesized the following: one of the most important things to subscribe to one plan or the other is to have a clear view of the features each one offers. Therefore, if we make these features clearer and more prominent, we may get more conversions.
With this in mind, here’s what we did.
On the one hand, we slightly increased the font size of the features shown in the price table (from 0.8rem to 1rem).
In addition, we also changed the order of some sections within the page: we moved the table with the detail of the functionalities higher up and moved the promotion and testimonials section down to the bottom of the page.
So let’s put this hypothesis to test and see whether we were right or not.
Definition of the A/B Test
To test the changes properly, let’s create an A/B test to test the current page against a new version with the changes we just described. To do so, let’s go to the Nelio A/B Testing menu on our WordPress dashboard and create a new A/B test of page:
We select the page we want to test and create a variant, which we can then edit to match our proposed layout. One thing that’s great about Nelio A/B Testing is that you’ll be able to edit alternative content using your page builder (in our case, it’s Gutenberg).
Regarding the conversion goals to be measured, we defined eight goals:
- Subscription completed : we want to know how many purchases have been made.
To do this, we have defined the conversion action as a visit to the “Thanks for your subscription” page to which the visitor always lands after completing the purchase.
- Three goals to measure the time that the user has been on the page:
- 10 seconds
- 30 seconds
- 60 seconds
- Four goals to measure the scroll that the user has made on the page:
- 30% scroll
- 50% scroll
- 75% scroll
- 100% scroll
For these 7 goals we have created seven custom events as conversion actions: time-10s, time-30s, time-60s, scroll-30, scroll-50, scroll-75, and scroll-100.
And added the small JavaScript code snippets that will launch the corresponding conversions (here you will find more details) in the two variants. In our case, we have customized our theme to be able to add the code in the page directly from the editor, as you can see in the following image.
With all this, we are now ready to start the test and let the tool you use for A/B testing (in our case, Nelio A/B Testing) collect the data.
Nelio A/B Testing
Native Tests for WordPress
Use your WordPress page editor to create variants and run powerful tests with just a few clicks. No coding skills required.
Analysis of A/B Test Results
We started this test at the end of January and have had it running for just over a month.
First of all, we will focus on the results from a performance point of view. Which variant achieved the most subscriptions?
As you can see in the image above, we see that the new version with the highlighted features is clearly a winner with a statistical confidence of 96.84%. With a conversion rate of 5.93% versus 3.14% in the original version, variant B performs 89.2% better.
If we look at the results in terms of time spent by users on the page, the results are inconclusive as there is little statistical variation in the analyzed samples of our visitors.
It may seem that, indeed, by making features more visible and higher up on the page, users need less time to decide to make a purchase.
In fact, if you look at the confetti map in which you can filter, among others, by the time before clicking, we see how in variant B, for example, there are 231 visitors who click (anywhere) in 5 -10 seconds. On the other hand, in variant A there are only 167. This may also reinforce the idea that the visitor who lands on variant B can make the decision to buy more quickly, as they have the relevant info at hand.
If we look at the results in terms of page scrolling performed by visitors, we also find inconclusive results.
Here it may also seem that in variant B, by having the features detailed higher up, visitors possibly scroll less to see the details.
Conclusions
Given the results we have obtained, we can safely conclude that variant B is better from the point of view of economic performance. Therefore, it is best to apply it as a definitive version on our website.
As you can see, with a single click we can apply the winning variation and show it as default from now onmake the final version. What better way to increase our revenue, don’t you think?
A/B tests help us identify, with real data, what works and what doesn’t. What are you waiting for to create your A/B test to increase conversion on your website?
Featured image by Unseen Studio on Unsplash.
Leave a Reply