WordPress Split Testing Themes

WordPress split testing themes is an experiment that tests an original theme and one or more alternative (child) themes in order to determine which version is more effective in improving your WordPress site.

The Nelio A/B Testing plugin will perform the experiment and return real data that will clearly show which version of themes will optimize conversions by attracting more visitors to your goal page or post. A/B and Multivariate testing will give you confidence that each change made to your WordPress site is generating more revenue, more subscriptions, and more clicks! Start split testing themes after you have installed, activated, and configured your A/B Testing plugin.

Instructions for Split Testing Themes

Once you have installed, activated, and configured your A/B Testing Plugin, adhere to the following directions to begin split testing!

Create a New Experiment for Split Testing Themes

  1. Under the Nelio AB Testing menu, click Experiments to view and manage your experiments. Please note that the first time you access this page, you will not see any experiments.
  2. Three Different Ways to Begin:
    • If you have not yet defined any experiments, click the Create one now!link.
    • Click the Add New button placed beside the title, Experiments, in the upper left hand corner.
    • Under the Nelio AB Testing menu, click on the Add Experiment option.
  3. You will now see a list with all the possible kind of experiments you can create. For each of them, you’ll see its name, an icon, and a description.
    • Click New A/B Theme Test.

Setting Up Basic Information

Fill out the Basic Information of an Experiment. In this particular case, the basic information includes:

  • Name: a meaningful and descriptive name for the experiment that will help you identify it in the future.
  • Description: any additional information about your experiment that will better help you identify it.

Select Alternative Themes

Under the Alternatives tab, select the Themes you want to A/B Test. As always, the current theme will be considered an “alternative” of the experiment too.

Please note that themes have to be configured before they can be used in an A/B experiment. In order to configure a theme, just select it using your WordPress dashboard (as if you were to manually change the theme), and set up the menu, widgets, and so on. The configuration will be automatically saved in your WordPress database and will be used by our plugin.

Please also note that plugins needed by any of your themes have to be enabled to (our plugin cannot determine which plugins are needed by each theme and, thus, it cannot enable or disable them automatically).

Goal Setup

In order to define and configure which actions should be counted as conversions, go to the Goals tab. Actions are organized into Goals*. You may add additional Goals by clicking the Add Additional Goal button. The first time you access this tab, you’ll see the Default, main goal. Actions are added as follows:

  1. Click on the New Action you want to create (for instance, if you want your users to view the Basic Package or the Professional Package page, add a new Page action).
  2. Configure the action properly (in our example, you’d have to select the select the Basic Package page, or the Professional Package, as well as any additional info required by the action*).
  3. Repeat the process for all the actions you need.

* For more information about how goals work and the available actions, check our Conversion Actions and Goals page.

Start Experiment

Once you have defined the experiment completely (that is, you defined the basic information and added one ore more alternatives), you may start the experiment. In order to do so, just go to the Experiments page, hover over the experiment you want to start, and click on Start to launch it.

* Experiments cannot be edited once they’re running. You may edit the contents of the alternatives, even though this is not recommended. Changing the definition of an experiment while it’s running may impact on the performance of each alternative and, thus, invalidate the statistic results obtained.

Understanding Progress Information and Results

Find the Results of Your Experiment

Running experiments are shown in the Dashboard as simple cards where relevant information is shown. For this kind of experiments, the following information is available:

  • Page Views: how many times have your visitors viewed the tested theme (either the original version or one of its alternatives).
  • Alternatives: number of alternatives defined in the experiment (original version included).
  • Original Version’s Conversion Rate: conversion rate of the original version.
  • Best Alternative’s Conversion Rate: among all the alternatives (original version excluded), it shows the highest conversion rate.

If you want to see more information about the Progress of the Experiment, either click on the experiment’s card in the Dashboard or go to the Experimentspage, hover over the experiment, and click View.

General Information

The Progress of the Experiment page is organized in three different tabs. Each tab focuses on a different aspect of the test and provides the most valuable and relevant information related to that aspect.

The first tab is the General tab. There, you’ll find:

  1. Summary section, which shows the most relevant metrics of your running experiment:
    • Big Icon. The icon helps you identify the progress of the experiment quickly. If it’s red, then there’s either not enough data or none of the alternatives is better than the rest; if it’s yellow (with a clock on it), there’s a possible winner, but we’re not confident enough about the results; and if it’s blue (with a #1 badge on it), there’s a clear winner.
    • Conversion Rate of the Original Version. How your current version of the tested element is performing. This gives you an idea of your current conversion rate.
    • Best Alternative’s Conversion Rate. The percentage of times that visitors went to the designated goal page o post when shown the best alternative* (without including the original version).
  2. Timeline of the conversion rate for the original and each alternative, as well as the total number of conversions and page views that the experiment has.
  3. Some metadata about your experiment. In particular, you can see the name and the description of the experiment, how long has it been running for, and the finalization mode (if any).
  4. Finally, you have two additional graphics showing the conversion rates of each alternatives and the improvements of each alternative with respect to the original version.

* The best alternative is defined as the alternative that has the higher conversion rate.

Details of the Experiment Alternatives

The second tab contains information about the Alternatives of your experiment. The alternatives include the original version of your tested element and all the variants you have created. For each alternative, you’ll have:

  1. Screenshot of the alternative (if available), so that you can see at a glance how it looks like.
  2. Information (both visual and textual) about the conversionspage views, and conversion rates of each alternative.
  3. A set of Quick Action Buttons:
    • View Content. Opens a preview of the selected alternative.
    • View Heatmap. It allows you to view the heatmap and clickmap for the page that is being tested.
    • Edit. It is shown when the experiment is running and allows you to modify the alternative.
    • Apply. It replaces the Edit link when the experiment is finished. It allows you to override the original page with the contents of the winning alternative.

Conversion Goals and Actions

The last tab contains information about the Conversion Actions of the selected goal. You can change the active goal using the Target icon located at the upper-right corner of the user interface, next to the Stop button.

For each goal, you’ll see the list of all the conversion actions it contains. These conversion actions are organized depending on its type.

Take a look at our analytics page for more about about conversion actions and conversion goals.

Not yet a subscriber?


The best A/B Testing plugin for WordPress