Bard or Bot?
Duke team wins prize for most human-like sonnet written by A.I.
Shall I compare thee to a robo-poet? Last spring, a team of Duke students embarked on a class project to see if they could teach a computer to write Shakespearean sonnets that could pass for ones written by humans.
One semester later, the algorithm they developed has won the 2018 “PoetiX” competition for computer-generated poetry.
Team member Peter Hase ’19 admits he’s no poet. “I could not write a sonnet. I’m not artistic,” Hase said.
But the exercise forced him and fellow student programmers and data geeks to think about what goes into writing poetry, while picking up machine learning skills along the way.
Poetry presents unique challenges for computers, said the team’s coach Cynthia Rudin, associate professor of computer science, electrical and computer engineering, and statistics at Duke.
As humans, we intuitively understand the often unwritten rules of grammar and syntax, not to mention metaphors, idioms and turns of phrase. But “there are a lot of nuances in the English language that computers don’t know,” Rudin said.
A human would recognize that the word “blue” can refer to a mood or a color, for example -- one of many double meanings that machines fail to grasp.
Not to mention that you can’t tell a computer to tap into its feelings.
The system learned how to compose sentences by studying other writers. To train their model, the team fed their bot the collected works of Walt Whitman, as well as “The Hunger Games” trilogy, and a poem in iambic pentameter by John Keats. It was then up to the computer to write its own sonnets.
Given a one-word prompt -- such as “love,” “sad” or “nature” -- it automatically spits out 14 lines of rhyming verse, with the da-DUM da-DUM da-DUM rhythm of iambic pentameter.
The algorithm composes each line in reverse, starting with the end rhyming words and working backward one word at a time until it has 10 syllables per line.
It can quickly identify all the words in its 14,000-word vocabulary that give it the right meter and crank out its creations in roughly 10 minutes.
“You can produce a lot of poems in a short period of time,” said team member John Benhart ’19.
Joined by Duke students Tianlin Duan ’18 and Liuyi Zhu ’19, the team hasn’t created a computerized Shakespeare successor or Walt Whitman hopeful yet.
Some of the more cringe-worthy results made their friends and classmates laugh. Commas sometimes appear in the wrong place, or grammatical errors give them away.
“If we went through and picked the worst quatrains you’d say ‘this doesn’t make any sense,’” Hase said. “It’s kind of high-variance.”
But their algorithm serves up more than word salad with the correct rhythm and rhyme scheme. “Sometimes these poems are coincidentally really good,” Rudin said. “Every once in a while it generates something that just happened to make beautiful sense.”
For the PoetiX competition, computer-generated sonnets were mixed with sonnets written by flesh-and-blood humans, then a panel of judges had to guess whether they were written by a person or a machine.
While none of the machine-written entries in this year’s competition managed to dupe the judges, a computer-generated sonnet from the Duke team’s algorithm was deemed the most convincing.
Computer-generated poetry is not new. In the 1830s and 1840s, an eccentric English inventor named John Clark spent over a decade building Eureka, a nightstand-sized machine that generated gloomy Latin verse, one line per minute, for the price of a shilling.
And in 2011, a Duke undergraduate named Zachary Scholl developed a poetry bot that got published in Duke's literary magazine The Archive, without revealing to the editors that the poems were written by a machine.
“After working on this it feels like computers are pretty far away from being able to write a play or a novel,” said Benhart, who also enjoys writing the old fashioned way.
“Now we’re trying to add more meaning and coherence in between lines,” Benhart said. “There’s still so much room to improve.”
The Duke Data Science Team got its start as a computer science class first offered in spring 2018, taught by Cynthia Rudin, with teaching assistant Rachel Ballantyne-Draelos. Within its first six months the team has entered and won two competitions.
CITATION: "Shall I Compare Thee to a Machine-Written Sonnet? An Approach to Algorithmic Sonnet Generation," John Benhart, Tianlin Duan, Peter Hase, Liuyi Zhu and Cynthia Rudin. https://arxiv.org/abs/1811.05067. November 13, 2018.