GPT-3 writes two sonnets, sorta’. The first is better than the second. [digital humanities]

My interview with Hollis Robbins ends with a “contest” in which GPT-3 competes with Marcus Christian in the writing of a sonnet. As I explain in the interview, GPT-3 was prompted with the first two lines of Christian’s “The Craftsman” and then completed it, like his (prompt lines in bold):

A sonnet by Marcus Christian:
I ply with all the cunning of my art
This little thing, and with consummate care
Contrive a score of air-tight cells
And coil therein a million filaments.
And yet, despite all my aspiring skill,
All it can do, howe’er I strain and toil,
Is make a drum that, once it is put up,
Can never be made into a bell.

Notice that GPT-3 only produced six lines, for a total of eight. Which is not a full sonnet. I was curious about that. You see, when you give GPT-3 a prompt you also specify a limit on token output. Did it stop at 6 lines because it came up against that limit or did it stop because it ‘decided’ it was through, though it had more space available?

I put the question to Phil Mohun, who is the one with access to GPT-3. Here’s his reply:

There is a max_token limit, however I can’t remember if it stopped by itself. Sometimes if you set the max limit too high and it senses that it’s run to the end of the topic it will switch topics. I tried your prompt again with a higher word limit. Here is the complete response:

A sonnet by Marcus Christian:
I ply with all the cunning of my art
This little thing, and with consummate care
Stroke, coax and lure,
And now I could not pry
For all the world to learn how on it stare
These two bewildered pair of velvet eyes.
Yet, though my ministrations pleasure bring,
They are not what they seemed, nor I am naught.
He shuns the pool of secrecy in which I swim,
He feeds and leaves me hungry, hidden in my spring.
So, though his thought my fancied excellence fulfills
I want, and I am wanting – here I wistful swim.
I try to fill his mind with pleasure’s liquor
But find there only void,
A dark abyss of emptiness and fear,
Where shadow waits, and swift my soul to carry.
I feel my wings grow frail, and heavy grow my head,
I choke upon the murky water in his stead.
Warm from the sun in which he basks, I lie, and die.

A sonnet by Gloriana Fiore:
Oh, to be without my long and lonely flight,
To be close to him! I close my ruby eyes
And think of all the ways I could begin
To make his days a time of precious joy.
A lark does not need its plum

So, much longer than the first time, though we’re not quite sure what’s going on internally. The interesting thing, though, is that this much longer output isn’t a continuation of what it had done the first time. It’s gone down a new path. That makes sense to me, though don’t ask me why. If I’d have been asked before the second go with the prompt, that’s what I would have guessed (I fact I was wondering about it).

Secondly, notice that this time GPT-3 finished one long straggling pseudo-sonnet, stopped, and the started another: “A sonnet by Gloriana Fiore:” That second one looked like it just stopped at the token limit.

Hollis and I felt that that adding this second effort to the interview would add little value. We also agreed that GPT-3’s first response to the prompt, those six lines, is better than this second response. The important point is that, whatever GPT-3 is doing, it's doing it well enough that we can make these judgments. I’m thinking we may well have an amoeba’s worth of intentionality going on. And it only took 175 billion parameters.

* * * * *

Here is Christian's sonnet:

I ply with all the cunning of my art
This little thing, and with consummate care
I fashion it—so that when I depart,
Those who come after me shall find it fair
And beautiful. It must be free of flaws—
Pointing no laborings of weary hands;
And there must be no flouting of the laws
Of beauty—as the artist understands.
Through passion, yearnings infinite—yet dumb—
I lift you from the depths of my own mind
And gild you with my soul’s white heat to plumb
The souls of future men. I leave behind
This thing that in return this solace gives:
“He who creates true beauty ever lives.”

Christian, “The Craftsman,” anthologized in Beatrice M. Murphy, ed. Ebony Rhythm: an Anthology of Contemporary Negro Verse. Exposition Press, 1948.