Wednesday, October 9, 2019

Reading the Human Swarm 3: A digression about an image with an applicaton to human cultural evolution [#DH]

What’s this image represent?

I supposed I’ve you’ve been reading New Savanna for awhile you know what it represents. But imagine you didn’t know. What would you think it is?

Since this is a post about recent book by Mark Moffett, and Moffett’s an expert on ants, you might think it’s a nest, one with 3300 chambers, each represented by a node in the graph. From The Human Swarm, pp. 57-58:
The size of leafcutter settlements can be gargantuan: in the French Guiana jungles I came upon a nest the square footage of a tennis court. A drawback of such a metropolis is the same face by a human city: pulling in enough resources means a lot of communiting. From the far corners of that large nest spring a half-dozen speedways along with the workers doubtless hauled hundreds of pounds of fresh foliage over the course of each year. To unearth just a part of another colony I once hired six men with pickaxes and shovelss near São Paulo. The bloody bites I sustained that week didn’t stop me from feeling like an archaeologist eshuming a citadel. Hundreds of gardens grew in chambers arrayed along meters upon meters of superbly arranged tunnels, some at least six meters below the surface. Scaled to human dimensions, their subway systems would be kilometers deep.
Pretty impressive, no?

If this were Tom Sawyer you might think it’s a map of the cave where Becky Thatcher got lost with Injun Joe.

If this were a post about the brain, you might think it’s a map of some neural system.

And so on.

Of course, it’s none of those things. Each of those nodes represents the text of a 19th century novel, British, Irish, or American. It’s from Matt Jockers, Macroanalysis: Digital Methods & Literary History (2013), a book I’ve written quite a bit about. Matt “measured” each text on each of roughly 600 features – just what I mean by measure and how Matt did it is secondary at this point, but if you dig around in those posts, you’ll find out – and then placed a point for each text in 600 dimensional space (one dimension for each feature). Why’d he do that? Because he wanted to identify books that are similar to one another. If the books are similar, then they will be close together in this 600-dimensional space.

So, he’s got these points (representing texts) in 600D space. Now he calculates the distance between each pair; with 3300 points, that’s a lot of pairs. How’d he do that? The principle’s the same as measuring the distance between, say, two cities, or two windows on a wall. We can treat the earth’s surface as a plane (two dimensions) and, of course, the surface of a wall IS a plane. Since these books have 600 features, we’ve got to embed them in a space of 600 dimensions. Tricky, but the principle’s the same.

Now Jockers made the graph by 1) connecting all the points together and then, 2) pruning away all the links that are longer than some appropriately low value. He then projected the resulting graph onto two dimensions and produced that image. All with the help of a computer, of course.

Easy as pie.

Imagine if ants could dig nests in 600 dimensional space. Yikes!

So what? you ask.

I’m getting there, I’m getting there.

Of course each of those texts have been read by many people, a few of them by millions of people. That graph thus implies the existence of millions of readers over the course of time. Now we’re getting to ant colony numbers, millions and millions.

Now we’re talking about a human swarm. And THAT’s how we have to think about human cultural evolution. That’s why I’m interested in that graph. Because it implies an ant colony we HAVE to think about it in evolutionary thinking. Evolutionary thinking is population thinking. That graph implies a population of people reading a population of texts yielding a population of readings. It boggles the mind.

Imagine millions of ants gathered together moving over the ground in a swath I don’t know how many meters wide – Moffett has described such things, but I can’t put my finger on one of those descriptions at the moment. There it is, a band of red-brown-blackish particles speckled by the sun, moving in a single mass. Well that's how, with the aid of Jockers’ graph, I think about cultural evolution. Millions of humans, their minds linked, moving through history like the Mississippi River over the flood plane of human events.

The human swarm indeed. And it’s culture that makes us a swarm. Without it we’d just be small bands of very clever apes, using twigs to fish for termites. Instead, we’ve stood on the moon.


  1. "I supposed I’ve you’ve been reading New Savanna for awhile you know what it represents."

    Yes or sort of. Issue I have is I can't really assess what is being done and evaluate.

    I half read some of the criticism alter the sting of inflection I don't disagree with it but I could apply the same criticism to anything I read or write.

    But clearly a charitable perspective is not the intent.

    I don't have accesses to the tool kit used. So I can't even do a basic assessment of what can be done, where it me be of value.

    I also assume that in an argument at this stage it is that both sides will over inflect like crazy and cherry pick subjects which may have a high degree of social value in the space( disputed territory) but far less value beyond that.

    I don't want to read here. I want an easy to use program that I can impute data and see something I can use or evaluate come out the other end.

    That would be interesting.

    Tools I can actually use for myself.

    1. I'm afraid Jockers wasn't using an easy-to-use anything. That graph must have taken a half-dozen programs of one sort of another. It's something he was able to create after he'd created a whole bunch of other things.

    2. I can but dream. The idea of imputing the data pressing a button and having a comparative model to work with is rather appealing at the moment.