Let’s start “The A/B Test of the Month” with a test that is not A/B. Before you run away to another corner of the Internet, let me explain you why we will start with heatmaps.
One of the main causes of not using A/B testing on a regular basis on your website is not knowing where to start: what changes do I try, where, and why? Those are the key questions you will ask yourself. If you don’t know how to answer them, you’ll give up testing.
To overcome this, what I recommend is that the first thing you do is to choose a page from your website to start. This page must be relevant to your goals. This means that it is a page where you receive many visits, or it is a key page for your activity or business.
In today’s example, we have chosen to start with the pricing pages of our premium plugins: Nelio A/B Testing and Nelio Content. These pages are key to our business, since the income of our company comes through both of them.
Now that we have chosen the pages, to know what changes to make and why, it is better to start by running a heatmap test. We do this because heatmaps provide us with real data on how our visitors interact with our website.
When we really see how our pages perform with real user data, it is much easier to get ideas for future tests to run. This is why heatmaps are always a good start.
How to Create a Heatmap Test in WordPress
Creating a heatmap test for a WordPress page is very easy. You just have to select the chosen page and start the test.
In Nelio A/B Testing, create a new heatmap test by selecting this type of test from the dialog that opens when you click on the button to add a new test.
On the test editing screen you set a name for your test so that it is easy to identify later. You can also add a description of what you want to try in the test. Last but not least, you need to select the page on which you want to do the heatmap test:
When you have everything ready (as you see in the previous screenshot, it is very easy), click on the button in the upper-right corner of the window to start the test. You just have to wait for your visitors to come so that Nelio A/B Testing collects data from their behavior on the page and process the results.
Heatmap Results
In our case we had two heatmap tests running at the same time on our website: one for the pricing page of Nelio A/B Testing and another for the pricing page of Nelio Content.
Each test has been collecting data for about 20 days and has analyzed the behavior of about 500 visitors. This is more than enough to be able to draw certain conclusions and think about future A/B tests that we can run in these pages.
Keep in mind that these pages are not the ones with more visits on our website. They are the final stage of the conversion funnel, hence traffic is lost along the way. But remember that we chose them because of their importance to our business. In future tests we will study other pages with more traffic, for other reasons that we will explain.
Nelio A/B Testing
I was very impressed by the quality of this plugin, how easy it was to set up, and the outstanding support Nelio provides. I highly recommend Nelio A/B Testing.
Josette Millar
Pricing Page of Nelio A/B Testing
Let’s start with the results obtained for the pricing page of Nelio A/B Testing. The heatmap test shows us the data from three different perspectives: heatmap, scrollmap, and confetti. Embedding here the 3 full images is a too much, because they are quite large in terms of height in pixels. To avoid forcing you to scroll up and down all the way, I just linked them here:
- Heatmap of the pricing page of Nelio A/B Testing.
- Scrollmap of the pricing page of Nelio A/B Testing.
- Confetti from the pricing page of Nelio A/B Testing.
Let’s now take look at the most relevant fragments of the previous results:
As you can see in the previous image, a heatmap is a graphic representation that shows the interaction of the users with the elements of the page, in an aggregated way and with a color scale where a warm color indicates great relevance.
In the pricing page of Nelio A/B Testing, the first fold of the page is where there are more hot spots. If we analyze it in detail, the elements in the area of the basic plan and the professional plan get more relevance than the enterprise’s. Given this, perhaps we could reorder the pricing plans to show the enterprise plan first. Note that we are used to read from left to right, so showing the most expensive plan on the pricing page first can be a good change to try in an A/B test.
Another element that receives a lot of user interaction is the link to scroll down to the see the detailed comparison of the plans. This is good, since it means that there is interest in seeing more details about the plans.
What it is not so good is that there are many interactions in non-clickable elements. You can see it in the confetti, which is the graphic representation where you can see all the clicks made by visitors:
In the entire block of plans that you can see in the previous screenshot, only the buttons that appear in each plan are elements that can be clicked. Keep in mind that clicking on everything else has no effect. Our current interface is generating confusion.
Visitors are clicking on the texts and icons of the features of each plan, which are non-clickable elements. From here we can understand that they try to find more information about it, but instead we have this information further down the page.
Another A/B test that we could try here is to change each of the features of the plans so they have a floating element that appears when hovering over the text and provides more contextual information to the visitor. We could test this version with contextual help against the current version to see which one ends up generating more purchases.
The confetti also gives us aggregate information of the visitors, extracted from the clicks they have made. Nelio A/B Testing includes several types of filters to better understand different characteristics of visitors, as you can see here:
From these filters we extracted the following data:
- Chrome and Firefox are the most-used browsers by our visitors, by far. Therefore, we must make the web look good, at least, in these browsers.
- Half of the visitors to this page come from the United States. This matches our sales data.
- Visitors visit us more the first 3 days of the week. Knowing this information can be useful to, for example, making offers the other days in other areas of the web, and thus attract traffic on days with low-traffic.
- Almost 100% of visitors access the page from a desktop computer or a laptop. We barely have visits from mobile devices. This is something to study in the future.
- Windows 10 and Mac OS X are the operating systems from which people visit us. It matches the previous data, since we have no visitors from operating systems for mobile devices.
- We have more visitors during the workday (the test was run in the Chicago time zone). It makes sense, since our main client is from USA.
- Visits are made on screens with a minimum width greater than 1500 pixels. Another fact that confirms that visits are from desktop computers or laptops.
If you check the full versions of the heatmap and confetti, you will see that all other sections of the pricing page of Nelio A/B Testing are less relevant. With that in mind, let’s check the following scrollmap:
If you check the complete scrollmap you can see that very few visitors go beyond the first fold of the page. This makes me think that perhaps we should go with a much shorter version of the pricing page of Nelio A/B Testing, condensing the relevant information above the fold.
Sections after the first fold of the page have a lot of text, and visitors are not reading it. Less than 20% comes to see and interact with the detailed table comparing the plans, which you can find at the bottom of the web. All this data is provided by the scrollmap:
Testing with shorter versions of the page including more visual sections can help convince the potential buyer and get a pricing page much more effective and efficient. This is another idea of a possible change to try with an A/B test.
We already have a few ideas that we could test in the future, so it is clear that the information that heatmaps provide us with is very useful to get ideas that we can then apply in A/B tests.
Pricing Page of Nelio Content
As with the pricing page of Nelio A/B Testing, here you have the complete images with the heatmap results for the pricing page of Nelio Content:
- Heatmap of the pricing page of Nelio Content.
- Scrollmap of the pricing page of Nelio Content.
- Confetti of the pricing page of Nelio Content.
The pricing page of Nelio Content is simpler and shorter than the one for Nelio A/B Testing. Among other reasons, this page has only one plan, while with Nelio A/B Testing we had 3 different plans.
If you look at the complete heatmap and compare it with the one of Nelio A/B Testing you will see that in this case there are more hot spots in other parts of the page, besides the first fold.
In the previous image we see that the action buttons within the table comparing plans get special relevance. This is good, because we have them there for that reason. However, note that the Starter plan in the table, which is nothing more than the free version of Nelio Content, takes on special relevance.
Here the question is whether we should have a link to the WordPress plugin directory in our pricing page so that visitors can download the free version of Nelio Content or not. Now we have it, but the heatmap shows that probably several visitors are leaving our pricing page to go for the free version.
This topic is controversial and the source of long discussions in our meetings. On the one hand, keeping the link is not good because you lose a possible direct sale inside the pricing page. But even though visitors leave our site, it may be the case that trying Nelio Content for free is a good thing. Users may love the tool and end up buying it later once they’ve tested it. In Nelio we have discussed this several times and I suppose today will not be the last time we do it ?. As for now, the link stays. What do you think about it?
We also see that the video catches a lot of attention. We created a very cool video, and maybe we could show it earlier in the page. We could try this change with an A/B test because the section after the first fold is not receiving that much relevance either.
On the other hand, we see on the scrollmap that the loss of visitors is less abrupt as we scroll down the page. This confirms that a shorter pricing page may work better if what we want is for the visitor to see as much of the content as possible.
In addition, combined with the previous heatmap we see that the two sections after the first fold, which talk about specific features and show opinions of people, may need a redesign. They are sections with too much text and that do not attract too much attention. We could try a more visual and direct alternative version.
With respect to the graphic of the confetti of clicks, it is interesting to see the amount of incorrect clicks that are made in not-clickable areas within the first fold of the page:
Even though most of these clicks cannot be avoided (yes, assume people click on side areas, especially on the right, when scrolling), there are others that are the result of confused visitors. This is the case of the clicks on the monthly price number that you see in the previous screenshot. That price is not a clickable item; you have to click on the orange button instead. However, the yearly price number shown below is a proper link.
We must look for another less-confusing way to show the two prices, with an equal interaction for both cases. This is another A/B test that we could run on this page.
And Now What?
We started without having any idea about what A/B tests we could run on our website. We have seen that heatmaps help us get ideas easily to test in A/B tests on our pages. Once we have these testing ideas, what we have to do is prioritize them according to our needs.
Remember that the changes in the first fold of the page are faster to test since more people will see them and thus you will obtain results faster. You could start prioritizing the A/B tests that try changes in that part of the page.
And most important lesson here is: once you start testing, don’t stop. After every iteration of the testing process you will learn something new from both your website and your visitors.
See you next month with another type of test in which we will analyze real results. Remember to leave me a comment with your opinion about this post and everything I explained here. On which page of your site would you run a heatmap?
Featured image by Steve Halama on Unsplash.
Leave a Reply