Smoothing Google
Why AI is probably going to get worse at poetry
Why has poetry featured in so many recent articles about conversational Artificial Intelligence (“ChatGPT”)? And why are their human authors so reluctant to seem intelligent about poetry themselves?
One answer is that it pays to play dumb for clicks. On Valentine’s Day, the Mail Online asked readers to compare love poems written by a human and a computer. The result looked touchingly like a primary school project:
The primary school test is all you need, too: one poem sets down images and feelings that are precise but oblique, and one reads like 5-7-5 syllables written by a child counting on their fingers. Even a child, though, would have spotted that “forever” makes the middle line run to 8 syllables; ChatGPT has presumably scanned it “fore/ver”. By contrast, the uncredited translators of the first poem — Lucien Stryk and Takashi Ikemoto — have deliberately not followed this formal convention in their deft English rendering of the swiftly expanding universe of Kobayashi Issa.
An alternative angle on the possibilities of AI poetry came from Trucknews.com, which asked ChatGPT to write in a range of forms about truck drivers.
A poetry critic might object that a villanelle repeats the first and last lines of its first stanza in subsequent stanzas, bringing them together in conclusion (as in Dylan Thomas’s “Do not go gentle into that good night”). This is not, therefore, a villanelle. But Trucknews drives eighteen wheels over such distinctions.
And this is what most ChatGPT poetry news stories have in common: indifference to actual poems. The fact that a computer can write verse on any subject, in seconds, is impressive in itself, as well as cause for nervous laughter; how well it can do it is, for most people, irrelevant.
A New Yorker article from last summer, “The New Poem-Making Machinery”, may have begun the craze. The author, Simon Rich, reported on what happened when one of his friends, a programmer working on ChatGPT, showed his laptop to the guys:
“The A.I. can write in any poet’s style,” Dan explained. “Pick one.”
Someone threw out Philip Larkin.
“How do you spell Philip Larkin?” Dan asked.
None of us were sure how to spell Philip Larkin. Brent looked it up on his phone. I remember being surprised to learn that Philip had only one “L.”
I would soon be significantly more surprised.
Dan pressed a button, and in less than a second the computer produced a poem in the style of Philip Larkin that was so much like a Philip Larkin poem, we thought it was a poem by Philip Larkin. We Googled the first line, expecting it to be an existing Philip Larkin poem, but we couldn’t find it on the Internet. It was an original work, composed by the A.I. in less time than it takes a man to sneeze.
Nobody wants to look ignorant in front of their friends (so, credit to Brent here). But Dan, Simon and ChatGPT are all wrong. This simple-minded rhyme is not a poem by Philip Larkin:
It does vaguely resemble “Days”, an uncharacteristically short and simple Larkin poem (“What are days for?”). But compared to “People”, “Days” is a masterpiece of subtlety, staking its initial restraint on the cinematic zoom of its concluding image (“the priest and the doctor / In their long coats / Running over the fields”). The vagueness of the resemblance is exactly why it’s not a poem by Larkin, who exercised an intense quality control over what he did publish; as he once unkindly said of the (short and simple) Old English poem known as “Caedmon’s Hymn”,
If I’d composed a poem like that one I’d keep it jolly dark, my God I would.
It’s no coincidence that Alan Turing, in his famous paper “Computing Machinery and Intelligence” (1950), included poetic composition among his tests of a computer’s ability to mimic a human being. But in Turing’s imagined scenario, the response was not that of conversational AI as we know it. It was something more plausibly human: embarrassment.
Q: Please write me a sonnet on the subject of the Forth Bridge.
A: Count me out on this one. I never could write poetry.
Is this a human or a computer speaking? Turing’s point is that, in order “to play the imitation game satisfactorily”, a computer must be able to imagine human limitations in its responses. Knocking out a sonnet in five seconds flat is going to fool no-one.
“Count me out” is the common response to poetry that Ben Lerner explores in his book-length essay The Hatred of Poetry (2016). Lerner begins by considering the mutual embarrassment that characterises the meeting of “poet and non-poet” (he gives the example of talking to his dentist). The resulting awkwardness, he argues, is
a little interpersonal breach that reveals how inextricable “poetry” is from our imagination of social life. Whatever we think of particular poems, “poetry” is a word for the meeting place of the private and the public, the internal and the external: my capacity to express myself poetically and to comprehend such expressions is a fundamental qualification for social recognition […] the embarrassment, or suspicion, or anger that is often palpable in such meetings derives from this sense of poetry’s tremendous social stakes (combined with a sense of its tremendous social marginalization).
What Lerner means here when he calls the capacity for poetry — exercised or not — “a fundamental qualification for social recognition” is one way of understanding the fascination with it as a test of AI, and the simultaneous embarrassment about whether the results are any good or not. In our culture, poems are treated both as highly human and highly alien objects.
Instead, the question of quality is deflected — by everyone from Turing to Trucknews — onto the question of knowledge. Al-generated poems are judged in defiance of Wittgenstein’s observation that “a poem, although it is composed in the language of information, is not used in the language-game of giving information”. Hence their appeal to local news outlets in the UK, where ChatGPT has been used to write rhyming verse on subjects of local pride, such as these lines on a Cornish pasty:
It is harder, here, to spot the cold mind of the computer because this is so close to the kind of reader’s poem that local newspapers have been printing — and I’ve been collecting — for years. A recent favourite, from a free magazine on the Norfolk coast, evoked the sight of the windfarms out at sea:
It was a cold and frosty morning on the cliff top
But the sky was blue and we just had to stop
Far out on the horizon, shining in the sun
Huge turbines were having nonstop fun.
The poem ends:
To watch and wonder is quite a thrill
And see this very modern windmill.
The poem is signed “Vic” — and I believe Vic to be human.
What I like about local poetry is its lack of embarrassment. Someone who could have said “Count me out” has instead said: here goes. At first glance, Vic’s couplets might sound like the Cornish pasty poem. ChatGPT’s method is to find and pair rhyming phrases from its databanks: “A taste that lingers in your mouth, / And memories that will never go south” being an odd but apt hit, given Cornwall’s southernmost status.
But when ChatGPT evokes the human “mouth”, it is using poetic licence. It doesn’t have one, so it can’t feel when something is right or wrong in it. “Filled with meat and veg that’s so sound” isn’t a bad set of ingredients for the intended meaning — but it doesn’t have what W.S. Graham, a real poet of Cornwall, called “the shape of speech”.
Vic’s poem does. The first four lines tell a story of discovery that follows a physical memory as it happened: “But the sky was blue and we just had to stop”. Unlike a line by ChatGPT, this is not a self-contained gloss on its subject: it raises a question (why did they stop?) which runs beyond the couplet, to be answered two lines later: “Huge turbines were having nonstop fun”.
Even more persuasive, to my mind, is the final couplet:
To watch and wonder is quite a thrill
And see this very modern windmill
The grammatical jack-knife here into the imperative is startling — and speechlike. It is the sound of someone breaking off to conclude. It continues the mood of enthusiasm (“nonstop fun”) that the poem takes towards its subject, which is the sight of windmills, not the fact of windmills (I’ve changed it here, but the original text was printed in BLOCK CAPS). Although Vic does at one point count “eighty eight of these giant sails spinning round”, he is not primarily playing the language-game of giving information. I’m serious when I say that ChatGPT is far from being able to write a poem as convincingly human as this, and only likely to get further away from it.
This is because AI poetry involves what is, revealingly, known as “smoothing”. Programmers add smoothing to a Natural Language Processing model in order to enable it to predict the probability of word combinations not found in the text it has been trained on — including words not encountered before. Smoothing aims to improve accuracy by improving the prediction of probability.
And this is where human poetry has the advantage of its rough magic. As Thomas Hardy wrote in his notebook in March 1875: “the whole secret of a living style […] lies in not having too much style”. Too smooth, and your words become like “worn half-pence — all the fresh images rounded off by rubbing”. Who could have predicted, for example, the resonant word at the heart of the final stanza of Hardy’s “The Voice”, a poem haunted by a dead woman heard in the wind:
Thus I; faltering forward,
Leaves around me falling,
Wind oozing thin through the thorn from norward,
And the woman calling.
The liquid thickness of “oozing” wrongfoots us, as the force of the wind seems to knock the poet off balance. But it is also the right word in so many other ways: in the double “oo” there is the sound of the moving air, while in the “-zing” there is the suggestion of a living voice (“-sing”) — and then there is the way the wider sound of the wind is strained into thinness by the thorn bush, just as the long “oo” leaves the throat to be pushed through four dental fricatives (“thin through the thorn”).
Poems are made by bodies as well as minds — or, rather, poems are made from bodies, just as music is (and just as embarrassment is). As Anthony Vahni Capildeo writes in a poem about “Catgut”, the traditional material used to string instruments: “You like them silken. / I’m listening for the looseness / … / the note this song needs”.
Which is not to say that I think computer models have no place in writing poetry. Where they can do something valuably poetic is in the detuning of smoothness with looseness, thereby allowing socially marginal language into any “silken” definition of poetry and its function as a “fundamental qualification for social recognition”. Twenty years ago, the American avant-garde movement known as Flarf used early search engines to find uniquely poetic combinations of words in the wild. More recently, Harry Josephine Giles and Martin O’Leary trained a computer program on the ballads of Walter Scott’s Minstrelsy of the Scottish Border. You can explore “the messy border zones between success and failure” that they created here: