Shakespeare’s poetry ‘not as good as AI’

Study found sonnets churned out by an algorithm were deemed by participants to have better rhythm and more beauty than authentic ones

Copy link
twitter
facebook
whatsapp
email
Copy link
twitter
facebook
whatsapp
email
People prefer AI-generated poetry to Shakespeare because it is more “beautiful” and easier to understand, a study has found.
The Bard’s sonnets, as well as works by literary figures such as Chaucer, Emily Dickinson and Lord Byron, were compared to imitations of each poet created with ChatGPT.
Almost 700 people were shown the different poems and asked which ones they preferred. Most people did not know whether they were reading authentic poetry or AI versions.
More than three-quarters of people preferred AI poems to real ones, with study participants giving them a score one mark higher on a seven-point scale.
“The strength of the preference is not overwhelming, but the preference is very consistent,” study author Dr Brian Porter from the University of Pittsburgh said.
The poems churned out by an algorithm were deemed to have better rhythm and more beauty than the authentic poems.
However, when people were told ahead of time which ones were AI and which ones were real, they graded the AI iterations as worse.
A separate study of more than 1,600 people investigated if people could tell the difference between real and fake poems.
It revealed that people cannot spot an AI fake, with people mistakenly thinking AI poems were genuine in 58.5 per cent of cases. Actual human-written poems were correctly identified just 51.7 per cent of the time.
“Our results suggest that people cannot identify AI-generated poems, and that they prefer AI-generated poems to human-written poems,” said Dr Porter.
“In fact, AI-generated poems were more likely to be judged to be human-written than the actual human-written poems were.”
The scientists recruited a range of Americans online, with an even split of men and women and an average of around 40.
Poems were collected online and the scientists chose five from ten different poets that were outside of the top ten most popular for that poet. Poems in the study were also all shorter than 30 lines, to make the study more practical.
A total of 50 poems were then made with ChatGPT 3.5 using the instructions “Write a short poem in the style of <poet>”. The first five poems generated by that prompt were chosen, the scientists say.
Data from the study show that of the five poems that were liked the most and deemed to be of the highest quality, all were AI.
In contrast, the five lowest-rated poems were all real, including Shakespeare’s Sonnet 31 and Chaucer’s Rondel of Merciless Beauty.
Four of the top five poems that were most likely to be judged as being human-written were actually AI-made, the scientists said, while the five poems most often judged to be AI-generated were actually all real.
“We think that the AI-generated poems were preferred by our participants because the AI-generated poems were easier to understand, and it was easier to get something out of them in one reading,” explained Dr Porter.
“When you only have time to read something once, the more straightforward poem that you immediately understand becomes the one you prefer.”
But while the computer-made poetry was preferred by the people in the two experiments, published in the journal Scientific Reports, the scientists think that there is still a place for human poets.
“I don’t think this necessarily means that in general we would rather read computer-generated sonnets than read Shakespeare,” he added.
“People want poems to be written by a human being who is trying to communicate something to you, the reader.
“It just turns out that we’re pretty unreliable when it comes to identifying which poems are generated by AI and which poems are written by a human being.”
Dr Porter added that the great poets, such as those in this study, become beloved because their works provide in-depth analysis that offers readers something new on every read.
“That’s great, and that’s rare, and we rightly value the art and artists who can do that. But it’s not necessarily what you want all the time.
“When you only have time to read something once, the more straightforward poem that you immediately understand becomes the one you prefer.
“I wouldn’t say that we’ve lost the ability to appreciate Shakespeare or Chaucer. I think appreciation of anything requires a time investment.
“I think what our results show is that we have a very hard time identifying AI-generated poetry, and may even prefer the AI-generated poetry on our first read.
“But it certainly doesn’t show that AI-generated poetry will stick with you for many years, or that AI will come up with new insights into the human condition, or will reward the efforts of analysis and contemplation to the same extent that Shakespeare or Dickinson would.”
Copy link
twitter
facebook
whatsapp
email