We continue with this series of posts explaining one of the A/B tests we’ve been running on our website. This time our goal is to try to drive traffic from our blog to our product pages. By doing so, we can get more visitors attracted by our content and also interested in our products.
The blog is one of the strengths of our website. We have been creating content for years. And we believe such content is useful to our readers. In addition, the blog is one of the pillars in our inbound marketing strategy.
To try to drive more traffic from the blog to our product pages, we have previously seen the use of a sidebar, which we tested here. We also tried to add banners to the content of the most popular blog posts, although the result we got wasn’t the one we expected.
In the case that we are going to study today, we added an additional section in each blog post showing our products. Thanks to an A/B test we will see if this change gets better results.
Test Hypothesis
On our blog, each post follows the same content template. First we have a header with the title and featured image. Then we have to the content. And finally we have the comment section.
In the test that we are going to explain this time, the hypothesis we want to test is that “by adding a section before the comment box with links to our products, the traffic that will reach them through the blog will be higher”.
To validate this hypothesis, we propose to modify the template of our WordPress theme to include a section like the one you can see in the following screenshot:
In this new section we have two boxes with featured images and the title of our products, along with a short descriptive phrase and an action button that takes the visitor to the landing page of each of them.
To create this section we have duplicated our theme’s post template, and then included the additional HTML code before the comments block. This means we now have a second post template in our WordPress theme.
With this, we have everything ready to start the A/B test and validate the hypothesis.
Definition of the A/B test
To define the A/B test in WordPress we will use our Nelio A/B Testing plugin, which you can find in the WordPress plugin directory.
Once the plugin is installed and activated, we go to the menu Nelio A/B Testing » Tests and create a new A/B test of templates. What we get from it is a user interface like the following:
We see that the original version is the default template for posts. As an alternative version, we are going to choose the new template we’ve just created in the previous point and that includes the section with the two products.
Next, we are going to define the conversion goals and actions that we want to measure in the results. In this case we are going to define three goals. In the first, we are going to count as a conversion any visit to the landing page of either of our two products that comes from a blog post. The other two goals are going to count the visits to each product separately.
With everything ready, we only have to start the A/B test. Once the test is active, half of the visitors will see the original template and the other half will see the version with the changes automatically. Likewise, every time a visitor who has seen a version visits the page of one of our products, the A/B testing plugin will count a new conversion and add it to the results.
We just have to wait for the results to accumulate to see the final trend and decide if one of the two versions is better than the other.
Nelio A/B Testing
I was very impressed by the quality of this plugin, how easy it was to set up, and the outstanding support Nelio provides. I highly recommend Nelio A/B Testing.
Josette Millar
Analysis of the A/B testing results
We have had this test running on the web for more than three months, during which time almost 145 thousand visits to blog posts have been analyzed. However, the impact of the change we have made to the template has been very low, with less than one percent converting.
This is mainly due to the fact that very few visitors pay attention to what is after the content. This is something that we already noticed when we talked about heatmaps and scrollmaps in a previous post. The deeper vertically on the page the change you make, the less impact it will have. In this A/B test we have confirmed again that this is what happened.
But going back to the results, if we look at the first goal, both versions of the template have had a similar performance. Neither version has achieved substantially significant changes that allow us to detect that one of the two options is clearly better.
The same happens if we only look at visits to the Nelio Content landing page, our second goal in the A/B test. The results are again not significant. Neither version of the template is better than the other.
Interestingly, if we look at the third goal, the version with the added change seems to be the winner in the A/B test. With a statistical confidence greater than 90%, the template that shows our products after the content of the posts manages to direct more traffic to the Nelio A/B Testing landing page.
However, we are talking about conversion rates lower than 0.2%, so we can say that, although the statistics tell us that the template with the changes is the winner for this last goal, for practical purposes the numbers we get are not too interesting.
With today’s A/B test we have found that if we want to achieve a greater impact by making changes to our blog posts, what we have to try is that these changes appear as high as possible on the page. Otherwise, it is very difficult to achieve relevant results that help us make decisions. Although that is still another hypothesis we’ll have to validate in the future…
Featured image of Austin Chan on Unsplash.
Leave a Reply