If there is something that is trendy today in the world of software and technology in general, it is everything that has to do with artificial intelligence (AI). Just look at the recent releases of GitHub Copilot, the artificial wizard capable of developing code to make life easier for programmers. Or even Tesla Bot, the humanoid robot that Elon Musk’s company wants to create and whose presentation leaves no one indifferent.
There is a lot of hype around everything that has to do with AI. Many say that in the future –a near one, by the way–, lots of professions will be threatened by AIs doing the job better than humans.
Beyond all the discussions on this subject, and all the opinions expressed on the Internet about the apocalyptic future that awaits us, I believe it is important to ask the following question: Can an artificial intelligence do things better than you today?
The answer, as always, is “it depends.” Depending on the type of tasks we are referring to, we can find AIs that produce incredible results. Even when we have creative tasks, which usually have been more difficult for computers and in which humans have always been superior. It may seem that AIs are beating us, but there are also cases where a human is still better.
So, a while ago I wondered if an AI could create a better web page than a human. And when I say human, I do not mean just any person, but someone with real experience in creating web pages. To be able to answer this question with real data, I found this web page generator based on GPT3 AI . My idea was to get a page created with this AI and test it using an A/B test against a web page made by us at Nelio.
Below I show you all the details of the test. What happened in the end will surprise you 😁
Definition of the A/B test
The first thing I did was selecting which page of our website would be the one to compete against the artificial intelligence. The chosen one was the landing page of Nelio A/B Testing, whose main task is to direct visitors to the pricing page.
Then, it was the time for the AI to make its magic. To create a web page, the web page generator that I found asks you to write the name of the product, the type of audience it is aimed at and a short description. With that information, it automatically fills in a collection of blocks that you can choose to build the page. You can see the user interface of this tool below:
There are different types of blocks, for the different sections that you can find in a landing page. Keep in mind that only the texts are generated and that the images do not have relationship with the rest. With all this, the page generated by the AI is as follows:
In order to make a correct comparison, what I have done is to adapt the result generated by the AI to the theme of our website. That is, I have created the same block structure with the options that WordPress have and the styles of our theme.
Regarding the price table that appears in the proposal generated by the AI, I have ignored it since we use a specific page to display the prices. I have also changed the images for others that make more sense with the topic of our website. You can see the result in the following comparison:
Original page of Nelio A/B Testing. Page adapted from the version generated by artificial intelligence.
On the left side you have the version created by us, and on the right side the version based on what the artificial intelligence generated automatically.
In order to check which version is better, we have created an A/B test of pages with both versions. The version that manages to bring more visitors to the pricing page will be the winner of the test. Our A/B testing tool takes care of automatically splitting the traffic between the two versions and analyzing which one works better without you having to do anything else.
Which version will be victorious? Ours or the one generated by an AI?
Nelio A/B Testing
I was very impressed by the quality of this plugin, how easy it was to set up, and the outstanding support Nelio provides. I highly recommend Nelio A/B Testing.
Josette Millar
Analysis of the A/B test results
After a while, the test tells us that we have a winning version with enough statistical confidence. And the winning version is the one we created. Unfortunately, the AI bites the dust, with its version of the page being 4.4% worse.
Seeing this, we might think AI still has a long way to go if it wants to succeed in replacing us, but the truth is that after the tests carried out, I’ve been pleasantly surprised by the results obtained.
The quality of the texts and blocks generated by the AI is high enough to look as if they were created by a human. In fact, I am sure that if you take the web page generator tool that we have seen and create a page automatically for a customer, that person will not even notice it.
In addition, using artificial intelligence can be useful to create alternative versions of your page and thus apply A/B testing in search of a better version. Now, you cannot say that you have no further ideas to test on your site, since it is the artificial intelligence itself the one that generates those alternative versions with no effort for you to test.
Admittedly, further integration with WordPress would have been necessary to avoid having to convert the AI-generated page to a native WordPress page using Gutenberg blocks and theme styles. But this part shouldn’t be too complex to develop, if we already have the core of the AI-based web page generator.
Even though in our case the AI-created version has performed somewhat worse, I’m sure it is feasible to get AI-generated pages that perform better than what you already have. And this, clearly, is something that can make us think that we are very close to get an AI that generates web pages better than a human.
For this reason, we should not be surprised to learn that this AI has already been sold for close to $3 million. It is clear that whoever has paid that amount foresees a great future to this type of tools.
What do you think about it?
Featured image by Brett Jordan on Unsplash.
Leave a Reply