The first rule of A/B testing is “You don’t talk about A/B Testing“. You probably saw this one coming but it perfectly applies here. As soon as your visitors are aware they are part of a split or multivariate test, the test loses all its validity since visitors may change their behaviour. This is a basic rule but, surprisingly, many companies violate it by making public one way or the other the fact they are A/B testing their site (even sometimes going to the extreme of asking their visitors to try to see the different alternatives and say which one they prefer).
Luca Sartoni collects this and other basic A/B testing rules in a “Fight Club style” in his post: The Rules of A/B Testing by Tyler Durden. Most of them are common sense but you know that common sense is the less common of all senses so better to take a look!
And once you’re done, go ahead and complement your dosis of A/B Testing common sense by watching this recent presentation by Ton Wesseling:
Hope you found them useful and please share with us your best A/B testing practices! Are they similar to the suggestions above? Why (or why not)?
Leave a Reply