How to Create a Theme A/B Test

A Theme Test lets you compare two or more WordPress themes installed on your site to determine which one performs better in terms of your goals — such as increasing engagement, reducing bounce rate, or boosting conversions.

You can run a Theme Test directly from your WordPress dashboard — no code, no manual configuration needed.

Step 1: Create a New Theme Test

1. In your WordPress dashboard, go to Nelio A/B Testing → Tests.
2. Click Add Test.
3. In the dialog, select Theme (under the “Global” or test types list).

💡 Make sure the themes you want to compare are already installed in your WordPress site before creating the test.

Step 2: Name and Define Your Test

Give your test a descriptive name (for example, “Theme A vs Theme B – Engagement Test”).

In the right sidebar you’ll find:

• Status & Management: see whether the test is Draft, Ready, Scheduled, or Paused, and configure start/end dates.
• Description: summarize what you are testing and what you expect (e.g., “We believe the new theme layout will increase product views”).
• Test Scope — Specify which parts of your website are affected by the theme variants. By default, variants are visible across your entire site. To limit the scope to certain pages, use the URL form to define where the test should run (for example, /shop/*, /blog/, or ?utm_source=ads). Limiting the scope is useful if you want to test a theme’s impact only on specific sections (like the shop or blog) before applying it site-wide.

Step 3: Select the Variants to Test

A Theme Test compares complete themes (including the one that is currently active), meaning the design, layout, typography, and styles applied to all your site’s content and measuring which performs best for your defined goals.

Select the Variants to Test

1. In the main panel, select one or more Variant Themes (Version B, C, etc.) from the list of themes already installed on your WordPress site.
2. Each variant will represent a different theme applied to your website for a subset of visitors.

💡 Note: Before starting the test, ensure all themes you plan to use are fully configured. To do so, temporarily activate each theme in your WordPress Dashboard (as if you were switching the active theme) and set up menus, widgets, headers, and any custom settings you want preserved. These configurations are automatically saved by WordPress and will be used by Nelio A/B Testing during the test.

Keep in mind that any plugins required by your tested themes must also be activated — Nelio A/B Testing cannot detect or activate theme-specific plugins automatically.

Step 4: Define Goals and Conversion Actions

Every test needs a goal — what counts as success.

Each goal can include one or more conversion actions that describe the specific behavior you want to track.

Name your goals, then click the Add Action button to choose from the available conversion actions.

You can combine multiple actions under one goal.

🔍 Learn more: What Are the Conversion Actions and Goals? →

Step 5: Add Segmentation Rules (Optional)

Segmentation lets you decide who participates in the test. You can target visitors by conditions such as:

URL parameter · Referrer · Language · Location · Logged-in status · Visitor Type · Day/time · Browser · Device type · Operating system · Window Width · Cookie · IP address

If you skip segmentation, all visitors to the tested pages are included automatically.

🔍 Learn more What is Segmentation and How to Use It →

Step 6: Start the Test

When everything is ready:

1. Review your setup summary at the top of the page.
2. Click Start.

Nelio A/B Testing will:

• Randomly assign visitors to variants.
• Track conversions anonymously via Nelio Cloud.
• Display real-time results in your dashboard.

⏯ Pause, Resume, or Stop Your Test

• Pause a running test if you need to make changes, then resume it later.
• Stop a test manually whenever you want, or let it end automatically under certain conditions such as duration, total number of visits or confidence results (available in Professional and Enterprise plans).
• Once a test is stopped, its results remain available for review.

Step 7: View Results and Analyze Performance

While your test runs, go to Nelio A/B Testing → Tests and click View Results to monitor its progress.

Summary of the results

In the right sidebar, you’ll find a quick overview of your test:

Goal selector – Choose which goal’s results to display.

Status icon – Indicates experiment status:

🟡 Collecting data.

🕒💛 Potential winner, low confidence

🟢🏅 Clear winner found.

🔴 No improvement over control.

Unique results – Filter duplicates.

Public results – Share a public link.

Export results – Download a CSV.

Summary – Shows status, duration, start date, and end condition.

Test Progress

The main area of the results page is divided into two tabs that visualize your test’s progress and performance:

• Conversion Rates — A line chart showing conversion evolution across variants.
• Views and Conversions — Compares total visits and conversions for each variant.

Confidence indicators appear alongside these charts, updating automatically as more data accumulates.

Below the charts, you’ll find a summary table with:

• A thumbnail preview (if available) of each variant.
• Number of visits and conversions.
• Conversion rate and improvement over control.

Step 8: Improve Your Web Conversion

You can end a test manually using the Stop button in the upper-right corner of the results page, or let it end automatically based on the configured end mode.

After completing a test and to improve the conversion of your website, click on the Apply button of the winning variant (if it exists) and that theme will become the active theme of your website.