Add an A/B testing clause to your next design contract

Are you a WordPress enthusiast? Whether you are a casual blogger or a professional editor, Nelio Content is the editorial calendar that makes your life easier. Take a look at it!

How much did you pay the last time you redesigned your site? At least several hundreds of dollars. Maybe a few grands. Was it worth? Most likely you don’t know. Maybe you liked the new design, you felt it looked better. Hey, maybe you even started to get more sales and attribute that to the new design (don’t worry, this is one of the most frequent mistakes you can make).

The problem with design projects

The honest truth is that you have no idea. You’re not sure whether the new design performs better than the former one and, if it does, whether the improvement justified what you pay for. And that’s because you didn’t split test the new design against the old one. Split testing (also known as A/B testing) finds out how people actually act in front of both designs rather than gambling on what you think they will do (or, even worse, what your pricey designer tells you they will do).

In its simplest form, A/B testing proposes to randomly split the traffic to the site in two groups so that 50% of the visitors see the A design while the other 50% sees the B design. By monitoring how users in each group react, we can calculate the conversion rate of each group and, in case there is a statistically significant difference between the two, declare a winner design.
In its simplest form, A/B testing proposes to randomly split the traffic to the site in two groups so that 50% of the visitors see the A design while the other 50% sees the B design. By monitoring how users in each group react, we can calculate the conversion rate of each group and, in case there is a statistically significant difference between the two, declare a winner design.

Our solution: add a split test clause in your next contract

If you’re paying a web designer / agency to revamp your website to get more sales (or subscribers, or shares,…) is only fair they prove to you their work actually achieves that, don’t you think? Word of caution: designers will not like this idea and will try to convince you they know what they are doing; this may be true but it’s your money so up to you whether you blindly believe them or ask them to put their money where their mouth is.

This requirement should be added as a clause in the contract you stablish with them (because you do plan to define some kind of formal contract with them, right???). An example of such clause (make sure you understand the meaning of the word ‘example’, i.e. do NOT try to copy & paste this text in your next contract) could be:

The new design shall function better than the current one for the following conversion goals: X, Y and Z under a standard A/B test with two alternatives (current design and new design) for a confidence level of 95% or above. If the new design fails the test you (the designer) accept to provide an alternative design able to pass the test or reimburse X% of the agreed project price

Note that this money back guarantee is not the only benefit you’ll get when starting to adopt a split test approach to make sure your website gets better over time. Following an A/B testing philosophy will also:

  • Help you formalize what are the goals you pursue with the site. What are the actions that you want visitors do when on your site? (obvious for ecommerce sites but not so much for other kinds of sites).
  • Open your eyes to how much your site sucks. You can’t imagine how bad your web pages and site design really are until you start A/B testing them. The new design may perform better than the old one but most likely it will still behave very poorly.
  • Culture of continuous improvement. Once you start with A/B testing you can’t stop. The vision that you can use a scientific approach to increase non-stop the returns you get from your site is so powerful that you’ll become a “split testingholic” until you bring your site to the next level.

Did we convince you? Will you try to include a split testing clause in your next contract? I hope you do and if your webiste is powered by WordPress remember that Nelio is the best tool to make sure designers fullfill your new contract obligations!

Featured image by Caitlin Childs.

Leave a Reply

Your email address will not be published. Required fields are marked *