鈥淚t has been two weeks, and the last of my kind has gone.鈥� As opening lines to a story about alien abduction go, it isn鈥檛 bad. It is even more impressive when you realise the writer isn鈥檛 human, but a newly developed artificial intelligence.
Getting machines to create stories is hard. At a minimum, a good yarn must have a plausible sequence of events that lead you from the beginning to the end. Yet听maintaining coherence over multiple sentences is something that existing text-generation systems struggle with.
鈥淭hey write in a very simplistic way, deciding word by word what to say next,鈥� says Angela Fan at Facebook AI Research, who helped create the story-writing AI. That means each sentence may not relate to the ones before it.
Advertisement
To get around this issue, the Facebook team broke the story-generation task into two steps. First, the system creates a sketch of the story鈥檚 structure. It听then fleshes out this scaffold with full sentences to produce stories that are 150 words long.
The team trained its neural network, a type of algorithm inspired by the human brain, on 272,600 human-written stories paired with prompts, such as 鈥淭he Mage, the Warrior, and the Priest鈥� or 鈥淎liens start abducting humans鈥�. These were taken from an online forum for writing prompts, where users inspire each other to write by submitting story premises. See examples below.
Room for improvement
The AI sometimes repeated itself in sentences, such as in one line 鈥淪he wasn鈥檛 sure if she was going to cry or if she just wanted to cry.鈥� Fan wants to improve the system so that it can look back at what it has already written and amend it.
But the team is happy that the software could produce stories in which a character remained sad or听the weather stayed stormy. 鈥淪taying on topic is quite difficult for neural network models because they have no explicit memory,鈥� says Fan.
The researchers evaluated their system by asking crowdworkers on Amazon鈥檚 Mechanical Turk internet marketplace whether they preferred its stories over those created by a previous text-generation technique. People preferred the new system鈥檚 stories around 67 per cent of the time.
鈥淭he results are impressive,鈥� says Mark Riedl at the Georgia Institute of Technology. 鈥淚t is always reasonable to be sceptical of cherry-picked examples, but overall I鈥檓 a big fan of this work.鈥�
Riedl and his colleagues have developed their own technique for generating more-consistent stories. Instead of sticking to a prompt, their neural network aims for a particular climax, such as two characters getting married or a villain being punished. It is drip-fed rewards as it generates story events to keep it on track.
Riedl鈥檚 team trained its system on a series of events taken from the summaries of movie plots. People on Mechanical Turk again judged the stories generated by this technique to have more-plausible event ordering and more-coherent plots than stories written by a previous approach.
Facebook AI Research鈥檚 work on听story generation is part of its efforts to improve text generation in general, says Fan. For example, it could improve the suggestions of what to write next that pop up听on your phone when you text听someone.
Software that can string sentences together with more consistency will also help machine translation. Existing algorithms are good at translating individual sentences but cannot translate entire paragraphs well, says Fan. 鈥淪tory generation tests our algorithms in these challenging areas.鈥�
A Facebook AI writes鈥�
听
Prompt: Aliens start abducting humans
It has been two weeks, and the last of my kind has gone. It is only a matter of time until there will be nothing left. I鈥檓 not sure what the hell is going on鈥� I can鈥檛 think. I can hear a distant scream. I think of a strange, alien sound. I try to speak, but am interrupted by something, something that feels like a drum, I cannot tell. I mean I鈥檓 just a little bit older than an average human. But I can, and I can feel the vibrations . I hear the sound of a ship approaching. The ground quakes at the force of the impact, and a loud explosion shatters the silence.
听
Prompt: Write a letter to your ex
I鈥檓 so proud of you! I miss you so much. I just want to hold on. I miss all the times we鈥檝e spent together. And every time we don鈥檛 get along anymore, I think about her sometimes and think she鈥檚 still there and she鈥檚 never really seen me at all. I wish I could tell you more than you would have wanted. I am sorry you didn鈥檛 know that but you left a lot of questions in my mind. We both have a good relationship, both of us are in a relationship. I love you. You鈥檝e always been so happy to share. The only thing that I know for sure is that if you were to ask me, I would be the one to tell you.
Reference: arXiv, ,听
