A/B Testing in the Age of Privacy: Navigating Regulations and Maintaining Trust

(Note that I am not a lawyer and this post is not legal advice. Always check with your DPO or legal team first).

A/B testing has become a popular technique for testing the effectiveness of marketing campaigns, website design, and user experiences. The way A/B tests are run is that you show different variations of your website (or mail, etc.) to random users while measuring how the change affects the metrics (conversion actions) you are testing for. To make sure any specific user sees the same variant for the whole duration of the test, a cookie with user information and behavior is set.

And that’s the issue: a cookie with user information and behavior is set.

The last several years have seen a bunch of new regulations and requirements around data privacy to give individuals more control over their personal data. They are aimed to give more data transparency and agency to consumers, while limiting certain data collection practices.

If A/B testing tools are based on cookies, can I still perform A/B testing in my website in compliance with data privacy regulations? Can I collect the information I need to deliver great experiences in my website while protecting users data? That’s what we’ll discuss in this post.

What You Need to Know About Data Privacy Rules

Data privacy is important because it allows you to have control over your personal information, making sure it’s not misused or abused. Basically, it’s like having a lock on your diary so that no one can read it without your permission.

When data privacy is in place, it helps keep sensitive information like bank records, medical records, and other personal details safe from unauthorized access. This prevents identity theft, cyber-crime, and other forms of misuse that can harm you or your organization. Having data privacy also builds trust with customers, clients, and partners, because they know that their personal data is being handled in a responsible and ethical way.

The GDPR and ePrivacy Directive in Europe, CCPA in California, LGPD in Brazil, PDPA in Singapore and more, are data privacy laws that exsit to give individuals more control over their personal data. It is important for you to be aware of these laws and to take steps to comply with them. Failing to comply with data privacy laws can result in severe penalties and damage your company’s reputation.

Without entering into the details of each law, as a website owner you need to be able to:

• Clearly inform your users about what’s being done with their personal data,
• Obtain explicit consent to use their personal data,
• Provide a lawful basis for using that data,
• Allow users to retract consent at any time,
• Respond to and act on requests from them regarding their personal data.

The ePrivacy Directive, informally known as the “EU Cookie Law,” is of particular importance to A/B testing because it’s focused on cookies and includes the following requirements:

• Obtain user consent before sending them marketing emails or using cookies to track their online behavior,
• Provide users with information about their use of cookies and how they can opt-out,
• The prohibition on the interception of electronic communications without the consent of the parties involved.

Moreover, depending on the country, different regulations might apply. For example, in France, the CNIL has an exemption for cookies used for A/B Testing. Whereas in Britain, the ICO says that you can’t use an exemption for A/B testing and the  German Federal Data Protection Act (BDSG) is being regarded as one of the most stringent in the world.

1st and 3rd Party Cookies

Before I continue with the impact of privacy on A/B testing, let me clarify the difference between 1st and 3rd party cookies. Basically, the difference between these two types of cookies is based on how a user’s browsing activity is collected and where that data gets sent.

1st party cookies are directly set by the website or domain you visit, or the app you have logged into. Once your user’s data has been captured, it is sent to an internal server.

3rd party cookies, on the other hand, are created by outside domains and relayed back to an outside third-party server, like Google, Facebook, or LinkedIn. It’s because the data is collected and sent out to a 3rd party that 3rdparty cookies are so named and their main purpose is tracking user’s behavior.

And because they track and send data back to advertisers’ servers, they’re seen as intrusive and highly invasive of user privacy. For this reason, current privacy regulations want them abolished. And, as a result, 3rd party cookies will soon be blocked by most major web browsers.

Summarizing, current regulations don’t prohibit the use of cookies. However, you can only use non-operational first-time cookies if, and, only if, the visitor previously provides explicit consent. And, as a result, 3rd party cookies will soon be blocked by most major web browsers.

Many A/B testing tools use 3rd party cookies, which means they will have to adapt to this new paradigm and limit some of the personalized A/B testing they are offering. However, Nelio A/B Testing only uses 1st party cookies to run A/B tests.

With this tool, you will not be able to track information across multiple domains but you will always maintain the privacy of your users’ data.

Nelio A/B Testing

I was very impressed by the quality of this plugin, how easy it was to set up, and the outstanding support Nelio provides. I highly recommend Nelio A/B Testing.

Josette Millar

WordPress.org

View Plans

How to Perform A/B Testing While Ensuring Data Privacy

With this scenario, can I still improve conversion on my website through A/B testing while complying with data privacy regulations?

The answer is yes, A/B testing does not need to disappear. But you must navigate these rules to maintain trust with your visitors, users and customers.

Here are some tips for conducting A/B testing in the age of privacy.

Use an A/B Testing Tool Compliant with Privacy Regulations

Before choosing an A/B Testing tool, you should select which data privacy criteria you want to adopt in your company. If you want, for example, to fully comply with German laws, you should look for a tool that provides you with a Terms and Conditions and Privacy Policy including the following information about the company:

• How the company prepared for data compliance,
• There is a Data Protection Officer Appointed (required for large companies),
• The company ensures reporting data protection violations to the Supervisory Authority in a timely manner,
• Who owns the data,
• International data transfers allowed,
• Data protection is being design and default respected,

And the following information about the A/B testing tool:

• The legal basis the A/B Testing tool process personal data
• What data the A/B testing tool keeps in its server logs
• Where the A/B testing tool stores data
• The A/B testing tool respects Do Not Track (DNT) settings
• The A/B testing tool does anonymized tracking

For instance, if you choose to use Nelio A/B Testing to create your A/B tests on your website, you have all this information in our Legal Information, you can respect the privacy of your users and comply with EU privacy policies.

Be Transparent

One of the first obligations you have to your users if you use cookies and/or perform any type of A/B testing on your site is to inform them. Make sure that your users are aware that you are collecting information anonymously and conducting behavioral testing on your website. This could be done through a notification banner or pop-up on your website.

Obtain Consent

Remember that before conducting any A/B testing, you need to obtain explicit consent from your customers. This can be done through a simple checkbox or button as shown above.

Note that the user must have the option to reject non-operational cookies and continue browsing your site. This means that your A/B Testing tool fullfils privacy regulation and includes a mechanism to ensure that it will not collect or track any data from users who reject cookies.

Note that explicit cookie consent implies that when you run an A/B test on your website, it limits your sample size to the users who consent cookies.

Protect User Data

In addition to the explicit consent, to protect your user data privacy and ensure you are following all relevant data protection laws and regulations, the A/B testing tool you use should only collect the necessary data and anonymize any sensitive information.

Make sure that you know all the details about the information gathered by your A/B testing tool: what information is collected, what type of cookies or other mechanisms are used to collect such information, how long the data is stored, and for what purpose it is used.

To Sum Up

We’re moving into a digital world where user privacy and personal data protection are paramount. This paradigm shift poses many challenges to marketers where their decisions were based on data that relied on user tracking to personalize the customer experience and accurately calculate conversions. However, you can still conduct A/B testing while maintaining the trust and privacy of your customers if you use the right A/B testing tool and follow the tips above.

Nelio A/B Testing is committed to user data privacy. As an EU-based technology company, Nelio A/B Testing complies with the strictest privacy laws and regulations globally to ensure consumer rights and consent are honored to the highest degree. It can help you run smarter A/B tests while ensuring the strictest of privacy standards are met.

Featured image by Matthew Henry on Unsplash.