You probably already know that the best technique to optimize the conversion on a website is to create A/B tests. But when you consider creating A/B tests on your website, many questions arise: Which pages should I test? What kind of A/B tests should I create? How do I create them? When should an A/B test end? And, once an A/B test has finished, what should I do? This post is intended as a guide to help you create A/B tests in a systematic way, answer all the above questions and make sure you get the best results on your website.
Table of Contents
What Is An A/B Test
A/B tests are design tests that allow us to make variations of the same page to compare the behavior of users in the different options and to be able to evaluate which of the versions obtains the best results.
Once you have analyzed the data, you can conclude which version responds better and when the results are more positive. For example, a few years ago, Obama, in order to raise revenue for his campaign, ran a test with different variations of his web landing page combining different images and buttons.
And the result was that with the winning version he managed to increase revenue by a total of 60 million dollars compared to the original version. Not bad at all, right?
When performing an A/B test, the elements to test on your website not only have to be pages, but also menus, widgets, themes, templates, etc. Normally, we call “original version,” “control version,” or “variant A” to the version that existed before running a test, and “variations,” “variants,” or “alternatives B, C, and so on” those that are not the original version. The differences between the different variations can range from very subtle changes with respect to the original version to variations with radical changes.
What’s more, A/B tests are created for the purpose of analyzing user behavior in order to achieve a pre-defined goal. We have already seen in the previous example that the objective was to obtain more revenue for a political campaign but it can also be to obtain more subscribers to a newsletter, more responses to a form, more sales of a product, etc.
Benefits of A/B tests
I guess the benefits of A/B testing are clear from the example above. A/B tests allow you to increase revenue in a proven way, based on real data about the behavior of our visitors. But it’s not just about revenue, the A/B tests also allow us to:
- Improve the user experience. A/B testing allow us to compare the different preferences and tastes of our visitors. With each test we perform, we will improve aspects of our website that we know are more successful and, conversely, we will not make design changes that could have a negative impact on our users.
- Improve bounce rate and reduce user friction. A/B testing helps to improve clicks to our products, articles and ads. Identify what works best and what our visitors ignore.
- Improve the conversion rate. When we talk about conversion, we are not only talking about improving revenue, but also that users take any desired action, such as consulting the details of a product or signing up for our newsletter.
- Improve our analysis. Google Analytics gives us a lot of information about what happens on our website, but an A/B test, in a very easy way, tells us if one alternative is better than another.
- Test everything on our website. One of the great benefits of A/B testing is that it allows you to test any design element and content. Not just an image and a button, you can also test font styles, change the menus, forms or any widget you have or even try different WordPress themes and see which one works best. A/B testing is the only tool that allows you to carry out this type of tests.
- Reduce the risk. And the best of all is that you know, with real and proven data, that the changes you make to your website, after performing an A/B tests, are much more reliable than anyone’s opinion. It will be precisely the opinions of your own users that will decide the changes you should make to improve your website.
Before Creating an A/B Test
If you want to optimize the results of your A/B tests on your website, you should first perform the following tasks. They will help you to identify the data on which you should act and which A/B tests you should create.
Define The Objectives, KPIs and Target Conversion Metrics For Your Website
First, identify what you want to achieve with your website, what are your conversion goals, KPIs, and the target conversion metrics for your website. For example, if you have a restaurant, one of the objectives of your website may be to increase online reservations. In this case, a KPI will be the number of reservations made through your website’s contact form and the target conversion metric may be to receive, for example, 100 online reservation forms.
Let’s see another example: we sell subscriptions to WordPress plugins and one of them is Nelio A/B Testing. One way to promote it is by publishing blog posts like the one you are reading right now, in which we try to help our readers learn more about A/B testing. In this case, one of the blog’s conversion goals is to get the reader to end up visiting the Nelio A/B Testing plans and pricing page. The KPI is the number of visits to that page and our target metric is to get 2,000 visits per month.
That is, before any type of analysis, identify the conversion funnel of your visitors: the steps they go through and the different pages they visit from the moment they land on your website until they are converted (i.e. perform those actions that will satisfy the objectives of your website).
Nelio A/B Testing
I was very impressed by the quality of this plugin, how easy it was to set up, and the outstanding support Nelio provides. I highly recommend Nelio A/B Testing.
Josette Millar
Collect Information and Analyze If You Are Achieving Your Goals
From your conversion funnel, your goals, KPIs, and target metrics, you analyze with the tools you currently have what results you are getting and the difference with respect to the expectations of target metrics you had set. Remember that we need data on which we can act. To do this, start by performing a heuristic analysis, that is, evaluate each page of your conversion funnel according to the following set of criteria and assess whether you could improve the results:
- Does the webpage meet user’s expectations in terms of content and design? How can we improve it?
- Is the content and offerings on this page as clear as possible? Can we make it clearer or simpler?
- What is it that causes hesitation on this page or makes the process difficult? Can we simplify it?
- What is on the page that does not help the user to act?
- And finally, can we increase user motivation?
Google Analytics is a good tool that provides much of the information you need to answer the above questions. For example, you can see the average time on each page, the bounce rate, where users click etc. Other tools that are very useful in this phase are heatmaps, clickmaps, and scrollmaps. These provide additional information on what catches the user’s attention on each page, as well as what they are ignoring. In short, it helps you better understand your user’s behavior.
For example, with Google Analytics we saw that almost half of the visitors to the Nelio A/B Testing page visited our pricing page, which is not bad. But the scrollmap helped us to identify that most of our visitors did not go beyond the first fold of the landing page to learn more about our product. Here we were able to find a potential area for improvement.
After this analysis, you get a detailed list of all the aspects that you think you could improve on your website. In the example above, we concluded that we should improve the Nelio A/B Testing page to try to get visitors to scroll down.
Generate Hypotheses For Improvement
Next, for each of the identified problems, generate an improvement hypothesis. For example, if in the previous point we identified that we had the problem of our visitors staying in the first fold of the Nelio A/B Testing page, perhaps if we modify the size of the first fold and make the title and text more attractive, our visitors will be encouraged to look at the rest of the page.
At the end, you will get a list of problems together with your hypotheses for improvement.
Sort The List of Items to Be Improved By Highest Impact And Lowest Cost
When you create A/B tests on your website, you cannot be testing them all at the same time because the results obtained would be mixed and it would be difficult to draw reliable conclusions. For this reason, the next task to perform is to prioritize the list obtained in the previous point in a pragmatic way: which improvements can have more impact and which can be easier to change.
The improvements that are going to have the most impact are usually those that occur on the most visited pages and, above all, on the pricing pages. Regarding the cost of the changes, it will depend on the hypotheses you have made in the previous point: it’s not the same to change the title or a color as, for example, to create new multimedia videos, etc.
Although ideally, you should end up trying all the items to improve, my recommendation is to start with those that can generate better results.
And that’s it. After all these steps, you can start the A/B test of the first item on your ordered list.
How to Create an A/B Test
If you’ve made it this far, you’ve already done the most difficult work. Creating A/B tests is very easy if you use the right tool. To do this, I recommend you read our article on the different A/B testing plugins for WordPress. It’s all advantages if you use a WordPress plugin, such as Nelio A/B Testing, to create A/B tests:
- You can create tests of any element of your website.
- You define the variants using the same predefined WordPress editors, without the need to use any external tool.
- It allows you to define a wide variety of conversion goals and actions.
- And it even allows you to segment which visitors you want to participate in a test.
In a matter of minutes, you’ll be able to create a test to validate your hypothesis.
During an A/B Test Run
Once the test is created, with a good A/B testing tool, you just have to indicate that the A/B test should start. It will divide your visitors into as many groups as you have variants and each visitor will always see the same variant. In addition, it will also be responsible for collecting conversion data so that you can know at all times in which state the test is.
The results page of an A/B testing tool should provide information about its status (if there are already enough visits to determine a winner) and different graphs about the conversion rates of the original version and the variants you have created.
How long should an A/B test be running? It depends on its complexity (the more variants, the more complex) and the number of visitors viewing the test (the more visitors, the sooner you will get reliable data on the results).
If you stop a test too soon you may not get a meaningful result, while keeping a test running for too long can cause you to lose conversions and sales due to a poorly performing test variant, as well as missing the opportunity to create new tests that can get you even better results. Our tests usually last between two and four weeks, but if you have a website with much more traffic, with less time you can achieve great results.
At The End Of An A/B Test
A test can be terminated for several reasons: there is a winning variant and the results are statistically significant, the original version is the winner and the results are also statistically significant, or you simply decide to end the test because it has already been running for too long without obtaining results that indicate a clear winner.
If there is a winning variant, congratulations! Now you just need to tell the A/B testing tool to apply the winning version as the final and only version that all your visitors should see from now on.
But it can also happen that the results obtained are not as expected. In this case, the test helped you understand that your hypothesis was incorrect. The A/B test help us validate our expectations and guided us towards the best solution possible.
Optimizing conversion on our website is a recurring process of analyzing data, creating hypotheses for improvement, creating tests and drawing conclusions about the results obtained. I strongly recommend that you take a look at the different tests that we are creating on our website and the results that we obtain. I’m sure they will help you to quickly understand what the conversion optimization process of a website consists of.
Featured image of Bannon Morrissy on Unsplash.
Leave a Reply