When it comes to selling online, how you show prices can make a big difference. Do you show the full price upfront, including all taxes and fees, or do you start with a lower price, show discounts and add the extras at checkout? Both ways have their pros and cons, and figuring out which one works best for your customers can be tricky.
It is this dilemma that has inspired us to create this month’s test.
Running an A/B test with these two pricing strategies will help us gather data-driven insights on which approach works best for our audience. Do they appreciate the honesty of upfront pricing, or are they drawn to the initial lower prices even if it means seeing additional fees at checkout? These results can help us optimize our pricing strategy to boost sales and enhance customer satisfaction. So, let’s dive into A/B testing and find out which pricing approach really clicks with our audience!
Testing Hypothesis
On our pricing page, by default we show a price that looks attractive, i.e. the equivalent monthly price for annual plans (which is 20% cheaper than if you subscribe monthly) with the Session Recordings addon pre-selected. Prices are shown without VAT applied. We indicate that billing is annual and VAT is not included.
At any time, the customer can select between the monthly and/or annual plan and activate or deactivate the inclusion of the Session Recordings addon to the subscription.
For example, as shown in the image above, the amount for the yearly subscription to Basic plan plus the Session Recordings addon has an equivalent monthly price of 38€ plus without taxes.
At the time of checkout, after clicking the subscribe button, the customer can see the resulting price billed annually and including VAT.
Displaying prices in this way can provide a marketing advantage, as lower base prices can be more attractive to our visitors. A lower starting price may attract more customers at the beginning, as it appears more competitive.
On the other hand, lower prices can lead to customer frustration when they see a higher annual amount with taxes added and there is an increased risk of cart abandonment. Customers might feel deceived, which can damage trust and loyalty.
Therefore, we decided to create an A/B test based on the hypothesis that showing the full price upfront provides transparency, as customers know exactly what they are going to pay from the outset, which builds trust. This approach also simplifies decision-making, as there are no surprises at checkout, making it easier for customers to make their decision. In addition, there are fewer abandoned carts, as customers are less surprised by the final price.
Our A/B test introduces a different pricing display as shown below:
Note that in this case, in order not to show too high figures, we show by default the monthly plans without the Session Recordings addon selected and including VAT.
At checkout, the prospect views exactly the same price shown in the previous page.
Here, when the user selects to view the annual plans, the totals to be paid annually including taxes are shown.
Definition of the A/B test
To test the changes properly, we create an A/B test to test the current page against a new version with the changes we just described. To do so, let’s go to the Nelio A/B Testing menu on our WordPress dashboard and create a new A/B test of pages:
First, we select the page we want to test and create a variant, which we can then edit to match our proposed layout. One thing that’s great about Nelio A/B Testing is that you’ll be able to edit alternative content using your page builder (in our case, it’s Gutenberg). So we edited the pricing page, made all the changes we needed to display prices correctly, including changes to the JavaScript code to integrate with FastSpring and display prices with VAT.
Note, as you can see in the screenshot above, that the main difference between editing a variant of an A/B test versus editing your original site pages is that editing the test pages includes Nelio’s A/B Testing sidebar that allows you to go back to editing the A/B test.
After editing the variant, we proceed to create our conversion goals. This is, what we want our visitors to do on this page and we decided to create five conversions goals:
- Purchase: we want to know how many purchases have been made.
To do this, we have defined the conversion action as a visit to the thank you page to which the visitor always lands after completing the purchase.
Since for the customers to purchase, must first proceed to click on a subscription button for one of the plans shown, we have also defined:
- Click on any plan: we want to know the total number of clicks we get on any of the plans offered.
As you can see in the A/B testing creation screenshot, for this goal we have defined as conversion action that the visitor clicks on an element matching de CSS selector a[href^="#purchase-nab-"] which corresponds to the SUBSCRIBE buttons of any plan.
And finally:
- Three goals to know the exact clicks on each plan: Click Basic, Click Pro and Click Enterprise.
In these three cases, we have defined as conversion actions the clicks on the elements matching the CSS selector, a[href="#purchase-nab-basic"] for the Basic plan, a[href="#purchase-nab-pro"] for the Professional plan and a[href="#purchase-nab-enterprise"] for the Enterprise plan, respectively.
With all this, we are now ready to start the test and let the A/B testing tool (in our case, Nelio A/B Testing) collect the data.
Analysis of the A/B testing results
We started this test the June 12 and have had it running for exactly two months.
First of all, we will focus on the results from a performance point of view. Which variant achieved the most subscriptions?
As you can see in the screenshot above, the graph shows a conversion ratio of 3,38% for Variant A and a conversion ratio of 4,13% for Variant B. This is a 22.2% improvement in the conversion rate of Variant B over the original version. You might think: WOW, that’s awesome!
But don’t be too hasty, this result is worthless unless it is statistically significant. This is, that there is enough evidence from the statistics point of view that the new variant likely has a higher conversion rate than the original one. Unfortunately, the results show that “No variant seemed better than the rest”.
How is this possible? In this case, the difference between conversions in either version may not be considered sufficient to know with certainty that it will have a real impact in the future. So in this case, although it may seem to you that clearly variant B is better, from a statistical point of view it may be purely coincidental.
If we look at the results in terms of user clicks to the different plans, what do we have?
It seems that, indeed, users tend to click on any plan when prices are smaller (which makes sense, right?). So, the confidence that the Control version is better is around 92,7%. In other words, it seems that people are more willing to buy and therefore click on the buy button with the lower prices, but we have already seen in the previous image that this does not mean that they buy more. In fact, more purchases have been made with version B.
If we look at the detail of clicks on each plan, for the case of the basic plan, we see that the control version is better and in this case we can be sure that users click more on the purchase button when they see the control version with a lower price. Then, it is possible that when they see the final price, they do not seem to be so encouraged to buy.
However, in the case of the Professional and Enterprise plans, although it was not conclusive, more clicks were made on alternative B.
Now, let’s take a look at the heatmaps to see if we observe any difference between the behavior of users landing at these pages.
As you can see in the two images above, there are no differences to be noted between the two variants. The hottest areas in both cases are the button to change the type of subscription (monthly vs. annual) and the button to add or remove the Session Recording addon on the Basic or Professional pla (the Enterprise plan already includes the Session Recording addon).
Nelio A/B Testing
I was very impressed by the quality of this plugin, how easy it was to set up, and the outstanding support Nelio provides. I highly recommend Nelio A/B Testing.
Josette Millar
Conclusion
The test run over the past two months has not given us conclusive results and has rather provided us with information that is quite in line with our intuition about the advantages and disadvantages of the two alternatives.
Variant A: Lower Price Up Front with Added Fees at checkout
Starting with a lower price and adding fees later has its benefits. A lower initial price can attract more customers initially, as it appears more competitive.
On the flip side, this approach can lead to customer frustration when extra costs are added later. Customers might feel deceived, which can damage trust and loyalty. There is also a higher risk of cart abandonment, as more people might abandon their carts when they see the final price.
Variant B: Full Price Up Front
Showing the full price upfront has several advantages. It provides transparency, as customers know exactly what they’re paying from the start, which builds trust. This approach also simplifies decision-making, as there are no surprises at checkout, making it easier for customers to decide. Additionally, there are fewer abandoned carts, since customers experience less shock at the final price. A clear pricing strategy can enhance the overall customer experience, leading to happier customers who are more likely to return.
However, showing the full price upfront also has its downsides. The initial price might seem higher compared to competitors who add fees later, potentially deterring some price-sensitive customers.
Our Decision?
When after running a test, the results are inconclusive, there will probably not be much change if you leave the original page as you had it at the beginning or if you decide to change it for the new version.
Considering that our priority is to get more sales of the larger plans and we like not to confuse our users with the information they see, we have decided to go with the B variant as the definitive variant.
The great advantage of using Nelio A/B Testing is that this is as simple as, after stopping the test, clicking the Apply button and all users will see this page.
Featured image by Akshay Chauhan on Unsplash.
Leave a Reply