The People vs AI: A $7m Experiment

Is Creativity a Uniquely Human Skill?

I have been asked this question countless times by marketing leaders: is creativity a uniquely human skill? 

And, if not, how can we use AI to create greater creative efficiencies?  

For example, a client at a leading global hair care brand asked me if AI should be used to create all images on social media channels in order to save money on agency creative.

My point of view is similar to that expressed by MIT: “AI is getting better at passing tests designed to measure human creativity", but I don't think that it can beat the experts when it comes to connecting two facets of people's lives in a new and novel way.

Fortunately, a $7 million experiment has given us greater empirical insight into this debate

The great DTC podcast recently interviewed digital marketer, Neil Patel, who looked at 28 different businesses - and an overall media spend of USD $7 million - to tell us which performs best: AI or human creativity.

I recommend listening to the episode; however, if you are short on time, I have summarised all the key points below and added additional analysis.

What was Analysed?

1,709 pieces of creative were created by his team and 4,041 by the AI machine. Of course, we must remember that while this experiment is fascinating, no single test can be considered conclusive as there are so many different variables, like category, product and media spend.

What was Found?

Human creativity outperformed AI 68.3% of the time. The human creative had a 1.54% conversion rate (across the different categories) versus 1.28% conversion for the AI creative.

But here’s where it gets interesting… when AI was asked to iterate on the best performing human creative, the output delivered the best conversion rate of all: 1.61%.

What can we Conclude?

It’s not a case of real people or AI; it’s both. One day, AI might be better at ideas than any individual or creative team, but right now, it is an incredible tool to make our human creativity even better.

A similar test in the aforementioned MIT article found that the creative ceiling for humans is currently higher, which feels like a good way to summarise the debate as it stands today:

“Although the chatbots’ responses were rated as better than the humans’ on average, the best-scoring human responses were higher.”

Why does this matter for marketers?

This World Economic Forum article discusses how there are three types of creativity: combinational, exploratory and transformational:

“Combinational creativity combines familiar ideas together. Exploratory creativity generates new ideas by exploring ‘structured conceptual spaces’, that is tweaking an accepted style of thinking…

“Transformational creativity, however, means generating ideas beyond existing structures and styles to create something entirely original…”

If we are to get the best out of AI and real people, we should focus the former on combinational and exploratory creative and the latter on ideas that are transformational.

As I explained to the aforementioned client, this is especially important if the brand isn't number one in the category. It is even more important to capture attention by being as distinctive as possible when the brand is looking to grow share, not just the overall size of the pie.