LLMs, hallucinations, and language as cultural technology @3QD

I’ve got a new article at 3 Quarks Daily:

Of Grammar and Truth: Language Models and Norms, Truth and the World

I start with an obscure topic in linguistics, evidentials, and then move on to so-called hallucinations and into the Gopnik, Farrell, Underwood account of AI as cultural technology. I conclude the article by explaining how I got Claude to create the text and discussed the issues that raises for attribution.

The penultimate section is entitled: What Language Turns Out to Be: Mechanistic. But I never really explain that. I’m going to do that here.

Or rather I’m going to let Claude explain:

The success of modern chess programs and large language models shows that language and reasoning are mechanistic, but not in the familiar steam-engine sense of mechanism. These systems are better understood as machines with trillions of interacting parts, whose behavior emerges from distributed internal dynamics rather than from transparent, human-scale causal chains. Such mechanisms operate autonomously: once set in motion, they carry out sustained symbolic activity without continuous human or animal control. This autonomy is not accidental; it is the defining consequence of scale. Just as early steam locomotives violated pre-industrial ontologies by exhibiting self-propelled motion without life, contemporary computational systems violate inherited ontologies by exhibiting structured linguistic and cognitive behavior without minds. What we are confronting is not the end of mechanism, but the emergence of a new kind of mechanism—one that forces us to revise the categories by which we distinguish agency, control, and understanding.

We decided that steam-engine mechanisms are best called equilibrium machines while machines of a trillion parts are generative machines:

By equilibrium machines I mean mechanisms designed to settle into stable, repetitive behavior, minimizing deviation and surprise. These are the machines of the Industrial Revolution, and they underpin the worldview of Homo economicus. By generative machines I mean mechanisms maintained far from equilibrium, whose internal dynamics produce structured novelty and exploration. Language is the paradigmatic generative machine, and Homo ludens is the form of life that emerges when such machines become central rather than marginal.

The world of Homo economicus is organized around equilibrium mechanisms: machines designed to settle, repeat, and minimize deviation. These are the mechanisms of the Industrial Revolution, whose success shaped not only our technologies but our intuitions about causality, control, and value. Homo ludens inhabits a different world. Its characteristic institutions and practices arise from generative mechanisms—systems maintained far from equilibrium, whose internal dynamics support exploration, play, and the continual production of novelty. Human freedom does not stand opposed to such mechanisms; it depends on them.

This allows me to observe (in Claude’s words:

Human freedom and creativity are not opposed to mechanism. They are grounded in a special class of mechanisms—decoupled, autonomous mechanisms whose internal standards of coherence allow sustained activity independent of immediate worldly constraint. Language is paradigmatic of this class.

That is an idea I’ll be developing in my book, Play: How to Stay Human in the AI Revolution.