Foto de dos pantallas de una oficina

Today we present a new A/B test that we have been running on our website to see which page layout gets more subscriptions to our premium plugin Nelio A/B Testing.

As usual within the series of post of the test of the month, we are going to see in detail the hypothesis for improvement that we have made, its application by creating variations of a page on our website, and the results obtained.

Hypothesis to Test

A/B testing your pricing page can provide valuable information about the behavior of your website visitors, as well as improve your conversion rate and revenue. In our case, we wanted to work on Nelio A/B Testing’s pricing page and, in particular, to highlight its features.

In the original version, there’s a block with the different Nelio A/B Testing plans and their basic features, followed by a block with a promotional image and message, and then another block of three testimonials.

First section of the Nelio A/B Testing pricing page
First section of the Nelio A/B Testing pricing page.

Then comes a detailed table comparing the features included in each of the plans we offer:

Second section of the Nelio A/B Testing pricing page
Second section of the Nelio A/B Testing pricing page.

And, finally, there’s a section where we emphasize that we are WordPress VIP technological partners and that you are guaranteed the return of your subscription during the first 30 days. Finally, the page ends with a section on FAQs and the possibility to contact us.

Third section of the Nelio A/B Testing pricing page
Third section of the Nelio A/B Testing pricing page.

After thinking about this page several times, we hypothesized the following: one of the most important things to subscribe to one plan or the other is to have a clear view of the features each one offers. Therefore, if we make these features clearer and more prominent, we may get more conversions.

With this in mind, here’s what we did.

On the one hand, we slightly increased the font size of the features shown in the price table (from 0.8rem to 1rem).

In addition, we also changed the order of some sections within the page: we moved the table with the detail of the functionalities higher up and moved the promotion and testimonials section down to the bottom of the page.

Section one of the proposed alternative page of the Nelio A/B Testing pricing page
Top of the proposed alternative page of the Nelio A/B Testing pricing page.
Section two of the proposed alternative page of the Nelio A/B Testing pricing page
End of the proposed alternative page of the Nelio A/B Testing pricing page.

So let’s put this hypothesis to test and see whether we were right or not.

Definition of the A/B Test

To test the changes properly, let’s create an A/B test to test the current page against a new version with the changes we just described. To do so, let’s go to the Nelio A/B Testing menu on our WordPress dashboard and create a new A/B test of page:

Editing the A/B test on the Nelio A/B Testing pricing page
Editing the A/B test on the Nelio A/B Testing pricing page.

We select the page we want to test and create a variant, which we can then edit to match our proposed layout. One thing that’s great about Nelio A/B Testing is that you’ll be able to edit alternative content using your page builder (in our case, it’s Gutenberg).

Regarding the conversion goals to be measured, we defined eight goals:

  • Subscription completed : we want to know how many purchases have been made.
Defining the Subscription Completeness Goal
Defining the goal of subscription completness.

To do this, we have defined the conversion action as a visit to the “Thanks for your subscription” page to which the visitor always lands after completing the purchase.

  • Three goals to measure the time that the user has been on the page:
    • 10 seconds
    • 30 seconds
    • 60 seconds
  • Four goals to measure the scroll that the user has made on the page:
    • 30% scroll
    • 50% scroll
    • 75% scroll
    • 100% scroll

For these 7 goals we have created seven custom events as conversion actions: time-10s, time-30s, time-60s, scroll-30, scroll-50, scroll-75, and scroll-100.

And added the small JavaScript code snippets that will launch the corresponding conversions (here you will find more details) in the two variants. In our case, we have customized our theme to be able to add the code in the page directly from the editor, as you can see in the following image.

JavaScript code added to each of the test variants
JavaScript code added to each of the test variants.

With all this, we are now ready to start the test and let the tool you use for A/B testing (in our case, Nelio A/B Testing) collect the data.

Nelio A/B Testing

Native Tests for WordPress

Use your WordPress page editor to create variants and run powerful tests with just a few clicks. No coding skills required.

Analysis of A/B Test Results

We started this test at the end of January and have had it running for just over a month.

First of all, we will focus on the results from a performance point of view. Which variant achieved the most subscriptions?

A/B test results from the point of view of completed subscriptions
Results of the A/B test from the point of view of completed subscriptions.

As you can see in the image above, we see that the new version with the highlighted features is clearly a winner with a statistical confidence of 96.84%. With a conversion rate of 5.93% versus 3.14% in the original version, variant B performs 89.2% better.

If we look at the results in terms of time spent by users on the page, the results are inconclusive as there is little statistical variation in the analyzed samples of our visitors.

It may seem that, indeed, by making features more visible and higher up on the page, users need less time to decide to make a purchase.

In fact, if you look at the confetti map in which you can filter, among others, by the time before clicking, we see how in variant B, for example, there are 231 visitors who click (anywhere) in 5 -10 seconds. On the other hand, in variant A there are only 167. This may also reinforce the idea that the visitor who lands on variant B can make the decision to buy more quickly, as they have the relevant info at hand.

If we look at the results in terms of page scrolling performed by visitors, we also find inconclusive results.

Here it may also seem that in variant B, by having the features detailed higher up, visitors possibly scroll less to see the details.

Conclusions

Given the results we have obtained, we can safely conclude that variant B is better from the point of view of economic performance. Therefore, it is best to apply it as a definitive version on our website.

Application of variant B as final in an A/B test
Application of variant B as final in an A/B test.

As you can see, with a single click we can apply the winning variation and show it as default from now onmake the final version. What better way to increase our revenue, don’t you think?

A/B tests help us identify, with real data, what works and what doesn’t. What are you waiting for to create your A/B test to increase conversion on your website?

Featured image by Unseen Studio on Unsplash.

Leave a Reply

Your email address will not be published. Required fields are marked *

I have read and agree to the Nelio Software Privacy Policy

Your personal data will be located on SiteGround and will be treated by Nelio Software with the sole purpose of publishing this comment here. The legitimation is carried out through your express consent. Contact us to access, rectify, limit, or delete your data.