We continue expanding the series of the test of the month of our blog with a new edition in which we want to know what is better: a video or an image?
On the main page of our website we have several sections so that the first-time visitor understands who we are and what we do. Often I visit a website and leave it without knowing what they sell or who they are. This is why it is so important that your landing is as clear as possible presenting your organization and your goals.
It is like when you are watching television and see an ad that you have no idea about what is promoting or trying to sell you. It is clear that this type of sensation is not what you want to cause to the visitors of your website.
But let’s get back to the topic that we want to discuss today. One of the most relevant blocks in our landing page is the one in which we present our products. Specifically, our premium plugins, which pay the bills every month and our salary (and also allow you, dear reader, to continue reading these articles regularly).
Currently, we use a two-column block that includes two cover blocks. In each of these blocks we have an image of the plugin in question, along with an action button with the name of the plugin that takes visitors to the specific landing page of each product.
Hypothesis And Definition of The A/B Test
The first thing we need to improve our website is a hypothesis of change that we believe can improve the performance of our page. In the specific case we are dealing with today, the hypothesis is as follows:
Using promotional videos instead of the images that present our plugins on the landing page of our site will drive more visitors to the specific landing pages of each plugin.
We can agree that the hypothesis will be true or think that it will be false, but all these are opinions that do not take us where we want: to advance in the optimization of the conversion rate of our website.
Therefore, we are going to put both versions, with images and videos, under test with a simple A/B test on the main page of our website. In the following comparison you can see the version of the original block with images, and the alternative version with the videos:
To set up the test you only need to download Nelio A/B Testing and activate it in your WordPress. Now let’s create a new A/B test of pages. The following editor lets us do it:
We set a title to the test in order to identify it easily, as well as a simple description that allows us to explain what we are going to test. In the variants section we select the main page of our website as the control version. Then we create a new variant (that will contain the block with videos) that we will edit later.
In the “conversion goals and actions” section we will create three goals to analyze the behavior of our visitors from three different points of view:
- A first goal will count visits to any of the two pages of our plugins, after the visitor sees one of the two versions of the page under test.
- A second goal will only count visits to the Nelio Content plugin page.
- A third goal will only count visits to the Nelio A/B Testing plugin page.
As you see, defining goals is very simple in Nelio A/B Testing. It is about selecting the landing pages in the test editor interface.
Now let’s go to edit the alternative page. We only change the products block so that instead of using cover blocks we will use blocks that embed our promotional videos from YouTube. And as a call to action we will use two buttons just below each video that drive visitors to the target pages:
We managed to create a simple alternative version to test if it works better (or not) than the version we are currently using. Now we just need to save the test and start it.
From now on, the traffic of visitors that arrive at our main page is divided in two parts: some see the version with the original block and the others the version with the promotional videos.
Will videos work better than static images to promote our products on the web? We just have to wait for the results to see it…
Analysis of The Results
The test has been running for 6 weeks. Enough time to get significant results. In fact, we could have stopped the test before, but I took a few days off because I’m now the father of a handsome boy.
As I mentioned before, we now have three sets of results: one for each goal. Let’s first look at the overall results. In this case, a conversion is counted if the visitor sees one of the two variants of the page under test and ends up visiting any of the pages of our plugins. You can see the result below:
The control version, which uses images to show the plugins on our main page, is the clear winner of the test, with a conversion rate of more than 35%. This means that of every 100 visitors who see that version, 35 visit a product page. Not bad, considering that the version that used the videos is over 20% worse than this.
Furthermore, the statistical confidence of the results is 99.97%, which tells us that the results are valid and we can be sure that they are not due to random effects on the sample tested.
Interestingly, what we were already using is better than the change we proposed.
Let’s see now the results individually. In this case, we separate the visit to the page of each plugin to see if the improvement occurs in both plugins or only in one of them. In the following screenshot you have the results for visits to the Nelio Content page:
The previous trend continues, although in this case statistical confidence is not as high as before (almost 84%). The original version is still better, although not as clearly.
However, if we look at the results for the other plugin, Nelio A/B Testing, here we do see that the version with images clearly beats the version with videos:
In addition, the statistical confidence here does reach a reliable value that indicates that we must keep the winning version (the one that uses images) and discard the other.
Therefore, there is no other choice but to affirm that the images have worked better than the videos in this case.
Nelio A/B Testing
Native Tests for WordPress
Use your WordPress page editor to create variants and run powerful tests with just a few clicks. No coding skills required.
Conclusions
Looking at the results we have obtained, the hypothesis we created stating that videos would work better than images to promote our plugins on the main page of our website is false. And the first one surprised here is me, who blindly believed that videos would give a better result than images.
However, we can discuss if a different design for the videos could work better. In the variant that we created the action button takes a secondary role, being located below the video. Maybe we could rethink this and test it again.
Also, I could have analyzed the click-through rate of the video as a conversion goal. Even though it is a more elaborate and complex thing, if more visitors watch the videos, maybe they will gain interest in our products and end up buying them.
What if we use animated GIFs? This is another design that could work very well, as animated GIFs grab visitors’ attention quickly. As a future test idea, we could create a couple of GIFs with highlights of our plugins and A/B test if they work better than static images.
Finally, you may think that if the results of an A/B test indicate that the proposed changes are worse, you wasted your time. Well, that’s not the case. If I had blindly followed my intuition and used the videos without measuring or testing anything, I would have ended with a worse performance on our site. But the worst of thing about it is that I wouldn’t even realize that.
Thinking that you are improving your website when you are really making it worse is something that may usually happen if you do not A/B test changes. Remember: propose changes to your website, test them and, depending on the results, apply or discard them. It is the only way to improve.
Featured image by Tim Mossholder on Unsplash.
Leave a Reply