Home › Documenation

How to Create a Headline Test

A Headline Test lets you compare different titles or headlines on your WordPress site — whether on pages, posts, or custom post types — to discover which one drives more clicks, engagement, or conversions.

You can create new variants, test against existing ones, and launch everything from your WordPress dashboard.

Table of Contents

Step 1: Create a New Headline Test

  1. In your WordPress dashboard, go to Nelio A/B Testing → Tests.
  2. Click Add Test.
  3. In the dialog, select Headline (under the Page category).
  4. Give your test a clear, descriptive name, such as Hero Headline Variation, Pricing Page Title Test, or Blog Post Headline Test.
Test selector in Nelio A/B Testing

Step 2: Define Scope and Consistency

In the right sidebar, you’ll see configuration options :

  • Description — Optionally record your hypothesis or what you expect to learn.
  • Test Scope — Specify where the headline variants will appear on your site.
    • By default, variants are visible across all pages of your website where the tested headline appears (for example, home page, archives, or widgets).
    • If you want to limit the test to specific URLs, use the provided form field to define the pages where the test should be executed.
  • Status & Management — Set start and end dates or schedule the test.
Naming a headline A/B test with Nelio A/B Testing

Step 3: Select the Post to Test

First, select the post to be tested — known as the Control Version (or simply Version A).

  • Choose it from the list of suggested posts, or search by title, URL, or identifier.
  • Once selected, Nelio will display the post’s title, excerpt, and featured image — together forming the headline that visitors will see during the test.
Selecting headline to test with Nelio A/B Testing

Step 4: Create New Variants

Next, create one or more variants of your selected headline.

  • By default, each new variant is an exact copy of the control headline.
  • Update the Title, Excerpt, and Featured Image for each variant.
  • To add more versions, click the New Variant button at the end of the section.
Definition of variants of a headline A/B test with Nelio A/B Testing

Step 5: Define Conversion Goals and Actions

Unlike other test types, Headline Tests have an implicit goal — encouraging visitors to click and read the post associated with the tested headline.

  • A page visit is counted each time a visitor sees a headline under test.
  • A conversion is recorded when that visitor clicks through and visits the post linked to that headline.

In short, Headline Tests measure which title, excerpt, and featured image combination attracts more readers.

Headline test conversion goal and actions with Nelio A/B Testing

Step 6: Add Segmentation Rules (Optional)

Segmentation lets you decide who participates in the test. You can target visitors by conditions such as:

URL parameter · Referrer · Language · Location · Logged-in status · Visitor Type · Day/time · Browser · Device type · Operating system · Window Width Cookie · IP addressddress

If skipped, the test includes all visitors to the selected headline.

🔍 Learn more What is Segmentation and How to Use It →

Step 7: Start the Test

When everything is ready:

  1. Review your setup summary at the top of the page.
  2. Click Start.

Nelio A/B Testing will:

  • Randomly assign visitors to variants.
  • Track conversions anonymously via Nelio Cloud.
  • Display real-time results in your dashboard.

⏯ Pause, Resume, or Stop Your Test

  • Pause a running test if you need to make changes, then resume it later.
  • Stop a test manually whenever you want, or let it end automatically under certain conditions such as duration, total number of visits or confidence results (available in Professional and Enterprise plans).
  • Once a test is stopped, its results remain available for review.
⚠️ Important note about editing paused tests

Editing a paused test may reset or affect ongoing statistics. When in doubt, duplicate the test before resuming.

Step 8: Review Results and Apply the Winner

While your test runs, go to Nelio A/B Testing → Tests and click View Results to monitor its progress.

Summary of the results

In the right sidebar, you’ll find a quick overview of your test:

Goal selector – Choose which goal’s results to display.

Status icon – Indicates experiment status:

🟡 Collecting data.

🕒💛 Potential winner, low confidence

🟢🏅 Clear winner found.

🔴 No improvement over control.

Unique results – Filter duplicates.

Public results – Share a public link.

Export results – Download a CSV.

Summary – Shows status, duration, start date, and end condition.

Results in an A/B test with a winner variant

Test Progress

Evolution of an A/B test with one variant

The main area of the results page is divided into two tabs that visualize your test’s progress and performance:

  • Conversion Rates — A line chart showing conversion evolution across variants.
  • Views and Conversions — Compares total visits and conversions for each variant.

Confidence indicators appear alongside these charts, updating automatically as more data accumulates.

Below the charts, you’ll find a summary table with:

  • A thumbnail preview (if available) of each variant.
  • Number of visits and conversions.
  • Conversion rate and improvement over control.
Results of an A/B test of headlines with Nelio A/B Testing

Step 9: Improve Your Web Conversion

You can end a test manually at any time by clicking the Stop button in the upper-right corner of the results page. A test can also end automatically when the condition set in its end mode (e.g., duration, number of page views, or statistical confidence) is met.

Once a test has stopped, you can apply its findings to your site. If there’s a clear winner, click the Apply button next to the winning variant. Nelio will automatically replace the content of the original headline with that of the winning variant, helping you improve your website’s conversion performance in just one click.

Next Steps