Have Your Customers Tell You What to Test Next

We all know that improving the conversion rate of your website is the goal of an A/B test. Depending on the site you’re running, these improvements might imply getting more customers into buying more products, subscribing to your mailing list, or sharing more posts. The problem is that’s easier said than done. Where do we start? What should we focus on?

A long time ago we discussed the process of A/B Testing. Essentially, it consisted in the following five steps:

1. Identify the goal that has to be improved (sales, subscriptions, shares…).
2. Hypothesize how we can improve the conversion rate. We need to identify what’s poorly designed and thus needs to be improved, so that we increase our conversion rate. Typical examples of what can we change include the color of your calls to action, the layout in which the information is presented, or the labels we use. The hypothesis would tell us something like “if we change this call to action from green to red, then it will stand out and visitors will more likely click on it”.
3. Implement the changes. Once we know what we want to change, we have to create the experiment. This includes creating one or more alternative versions of our page and, for each of them, implement the changes we proposed in the previous step.
4. Run the test and wait for the first results to come in. When you’re in this stage, waiting is extremely important. You need to wait until the results are statistically significant.
5. Act on the results. Finally, you know whether your hypothesis is true (“the green button converted better than the red one”) and, if it is, you can change your website accordingly.

Whether you’re an experienced A/B tester or a beginner who has only run a few tests (if any!) so far, coming up with meaningful and valuable ideas for designing new A/B tests is complicated and time-consuming. Luckily for you, absolutely all the split tests have one thing in common—your customers. In fact, customers are the first and only reason for you to run A/B tests on your website. You want them to convert better, so you need to know their pain points and see how you can change your website so that it overcomes them.

So, if customers are the most important aspect of A/B tests, why don’t we use them for generating new ideas for our split tests? In this post we’re going to take a look at different data sources we can use to seed great A/B tests. In particular, we’ll focus on:

1. Customer service knowledge, including our ticketing system, e-mail, and public forums.
2. Internal search keywords.
3. Usability tests.
4. Competitor data.

1. Customer Service Knowledge

We already said that the goal of an A/B test is to improve the conversion rate of a website. We might as well say that the goal of an A/B test is to overcome your customer pain points. Customer pain points are defined as follows:

A customer pain point is anything that causes a customer discomfort or annoyance that your business can solve.

A few months ago Ruth explained us how we can optimize our website for customer acquisition. In that post, she talked about the importance of identifying the concerns of our visitors (that is, their pain points). Your customers certainly have one or more pain points. Pain points might be small but affect a lot of people (who, by the way, may talk about it on social networks), or might be a major problem for only a tiny subset of customers that don’t feel like sharing or complaining about it. The problem here is to identify and address all these pain points. But how?

The first stop in the journey of discovering your customers’ concerns is your customer service database. If you’re running a business, you’ll likely have one or more means for your customers to reach you. For instance, in Nelio we have a support platform powered by Freshdesk, the WordPress.org plugin repository, and the e-mail. Through any of these channels, our customers and prospects can reach us and send us their questions, complaints, and requests, making them one of the most fertile grounds of ideas for new split tests.

Consider, for example, the latest question a WordPress user had about our service. Gary accessed our website and saw that we offer different plans, but he didn’t know whether a subscription was required to use our plugin. Other users have contacted us by other means and have asked the same and related questions:

• how does the free trial work?
• do I need a subscription for using Nelio A/B Testing?
• can I pause my subscription at any time?
• do you have a money-back guarantee?

All these questions highlight possible reasons for prospects not subscribing to our service. If we act on them, we may improve our conversion rate. So, instead of just answering these questions over and over again, we decided to test out a new version of our plugin where we clearly stated that you need an account for using Nelio and that we offer a 30-day money-back guarantee. Unfortunately, these changes did not lead to an increase in conversion rates, even though it reduced the amount of tickets we receive asking for these topics (and time is money, right? 😉 ).

In my opinion, it’s clear that the customer service team has a lot of insight on what’s going on in your customers’ minds. Therefore, it’s important that the marketing team, responsible of generating and running A/B tests, has a tight feedback loop with the customer service team, so that they can address from the very beginning of the acquisition funnel those issues that happen regularly.

I hope that, by now, you’ll agree with me that your past customer service entries are a great source of A/B testing ideas. However, scanning  through all of them can be tedious. But that’s easy to solve: just group your tickets and e-mails in categories and ease your future scans!

2. Internal Search Keywords

If your website has a search bar, then your visitors will probably use it at some point. The keywords they use for searching stuff in your website are also a great source for designing new tests. Are there any recurring terms? If so, you should try highlighting them in your home page and making them super accessible!

By now you should be thinking: “Of course! That’s a pretty smart idea… except that I don’t have any information about which keywords are being used in WordPress. Nor it is possible!” Well, you’re partially right. Let’s see what can and what cannot be done in WordPress, shall we?

By default, WordPress does not track information about any of the searches that are triggered in your site. So it’s completely normal that you don’t have any information about previous searches. However, you can instruct Google Analytics to track your site searches. As you can read in the linked article, you simply need to tell Google Analytics to index WordPress site searches (which use the s query parameter) and GA will take care of everything else! Swift, huh?

Alternatively, there are some great plugins that improve WordPress regular search functionality and take it to the next level:

• Swiftype Search. Swiftype is an external service that provides advanced search capabilities. In particular, you’ll be able to customize the search results for any query string, and you’ll have interesting analytics about all the searches that are run in your site. Swiftype is WordPress VIP-approved and already used on large sites like TechCrunch and Mixergy.
• Relevanssi. Relevanssi replaces the standard WordPress search with a better search engine. Among others, it sorts results based on relevance and includes fuzzy matching, and it maintains a user search log.
• Search Meter. Search Meter automatically records what people are searching for in your blog and whether they find it or not. Its admin interface shows you what people have been searching and, more interestingly, which searches have been unsuccessful.

3. Usability Tests

Antonio talked about usability tests last year. A usability test gives us insight on how our visitors feel while they’re browsing our website, and it helps identify those aspects that might be stopping them from becoming our customers. In his post, Antonio mentioned a few tools for gathering real users’ feedback:

• Peek. This tool records a 5-minute video of a person browsing your website, for free!
• Try My UI. Similar to Peek, Try My UI will also give you a video of a user browsing your site. In this case, however, you may even specify the specific actions that you want them to take during the recording session.
• GhostRec. This tool records your visitors’ activity, without their knowing you’re doing so.
• Heatmaps and Clickmaps by Nelio A/B Testing. I like to think of heatmaps as previous step to A/B Testing. In this sense, heatmaps are a perfect tool for detecting what your users are not doing.

This technique takes a little longer than the previous ones, because you have to run multiple usability tests on different users. However, these tests will teach you a lot of your visitors with little to no effort. Believe me when I tell you that having actual comments on what’s working and what’s not, as opposed to speculating, can make a big difference.

4. Competitor Data

Did you know that your customers look at your competitors too? Yep, I’m sorry to be the one bringing the “bad news” (you already knew, didn’t you? 😉 ), but that’s the way it is! Customers browse your competitors’ websites, compare their prices and functionalities with yours and others’… they may even purchase their products! And that’s not necessarily a bad thing. Complaints and compliments about a competitor’s site might as well apply to yours, so you can use this feedback in your own benefit (which was not originally intended for you).

If the prospects of one of your competitors have a problem with their website which you’re solving, those prospects will become your customers once they reach you! Why? Because you’re solving their problems and your competitors don’t (or, to be fair, at least you clearly state you solve them).

Now, where do you geed this data from? That’s an easy question! We’ve already talked about where you can get this data for your own website, haven’t we? Check your competitors’ public forums and support threads and see if they have recurring problems. Or run usability tests on their websites and see what customers say about them. Or compare their search results with yours and check how similar (or different) they are. Remember that, right now, we’re only interested in creating a pool of ideas that we’ll later use to build our own A/B tests… just gather as many information as you can!

Nelio A/B Testing

I was very impressed by the quality of this plugin, how easy it was to set up, and the outstanding support Nelio provides. I highly recommend Nelio A/B Testing.

Josette Millar

WordPress.org

View Plans

OK, Let’s Do It!

As you can see, there’s plenty of sources you can use to generate new testing ideas. If you’re not already keeping track of your tickets, e-mails, and site searches, you should start right now. Public feedback and competitor data is already available, but it’ll take some time on your end to be collected and classified. Once you have this sorted out, you simply need to commit to properly classify new information as it comes in. If you do so, you’ll have and endless stream of data and insight that will lead to your new split tests.

Tell us what other sources of inspiration you use! 😀

Featured image by Amy McTigue.