AI-generated literature may have an unfair reputation for poor quality. Despite a cultural backlash to AI-generated content, people seem to prefer it 鈥� and if they鈥檙e lied to and told that it was created by a human, their opinion rises even higher.
and at Villanova University in Pennsylvania recruited 1682 adults 鈥� selected to be representative of the US in terms of their gender, age, and race 鈥� and asked them to read a story. Half the participants read a story written by a human, and half read one written by AI. Each of these groups was again split in two, with half told that their story was written by a human and half that their story was written by AI, regardless of its true origin.
The human-created stories were sourced from a literary journal, while the AI-generated ones were created using ChatGPT 4.0.
Advertisement
Those who read an AI-generated story ranked its quality as 1.54, on average, when asked to score it on several qualities on a scale ranging from -3 to 3. Meanwhile, participants who read a human-written story gave it an average score of just 0.97.
Participants who read an AI-generated story also ranked it as more absorbing, based on several qualities on a similar scale, with an average score of 1.42. Those who read a human-written story, meanwhile, gave an average score of just 1. In all cases, the story was rated slightly higher if the participant was told it was written by a human.
The researchers also ran two further studies, where several hundred people were given an AI-generated story and a human-written story and asked to identify which was which.
In the first test, only 39 per cent made a correct identification, while in the second, 52 per cent were correct. The researchers have no explanation for the difference, except that the second test took place several months after the first; society may be adapting to better understand AI output. In any case, they claim that it shows people are no better than chance at distinguishing human work from AI output.
at Lancaster University, UK, says that humans are having to come to terms with the uncomfortable idea that we aren鈥檛 any better than robots at writing short stories. She says that in on testing people鈥檚 ability to identify AI-generated speech, text and music, it is the music that angers them most of all when they are tricked.
鈥淚 think there鈥檚 a huge sensitivity [around AI generating art],鈥� she says. 鈥淲hen you have AI that is continually improving at the speed that it鈥檚 improving at, there is going to come a point where every single model can produce Shakespeare-like output. What does that mean for humans?鈥�
While AI is becoming increasingly capable with every passing month, the concept of AI-generated literature and art is creating a backlash. Earlier this year, a short story given the Commonwealth prize and published in Granta was accused by some of being AI-generated 鈥� something which the competition explicitly forbade. An . Last year, criticised an auction of AI-generated art at Christie鈥檚 because they saw the work as 鈥渕ass theft鈥�, due to models being trained on existing images.
at the University of Reading, UK, says that part of the preference for AI-generated writing in this case may be down to researchers taking stories from a literary journal, rather than choosing a more accessible and popular genre.
鈥淧eople like expected things. People like easy-to-process things. Real human writing, real kind of human literary writing, is, frankly, not very popular, because it鈥檚 hard work,鈥� says Jones.
Although ChatGPT was asked to create new stories for the experiment that had traits similar to those of the human-written stories, Jones thinks it would have produced a simpler, easier-to-read version by default.
Weisberg agrees that AI produces more 鈥渧anilla鈥� content than humans do, and ascribes the perceived difference in quality to this. She plans to run experiments in the future on different types of writing to see if AI retains its lead.
鈥淚 think AI will be able to accomplish some truly amazing feats of creativity, but I think AI creativity is just different from human creativity,鈥� says Weisberg. 鈥淚 think it will be able to do something that we will recognise as having artistic value, but it might be measured on a different scale.鈥�
Judgment and Decision Making DOI: 10.1017/jdm.2026.10042
