New account of AI Doom @3QD: A new counter culture arises

I’ve been plugging away on this AI Doom business since forever it seems. I’ve certainly run up a bunch of posts on the subject. But I’ve finally arrived at what I think is a decent way to think about it. I’ve written a long post about it over at 3 Quarks Daily (link below, followed by the introduction).

* * * * *

A New Counter Culture: From the Reification of IQ to the AI Apocalypse

I have been thinking about the culture of AI existential risk (AI Doom) for some time now. I have already written about it as a cult phenomenon, nor am I the only one. But I believe that’s rather thin. While I still believe it to be true, it explains nothing. It’s just a matter of slapping on a label and letting it go at that.

While I am still unable to explain the phenomenon – culture and society are enormously complex: Just what would it take to explain AI Doom culture? – I now believe that “counter culture” is a much more accurate label than “cult.” In using that label I am deliberately evoking the counter culture of the 1960s and 1970s: LSD, Timothy Leary, rock and roll (The Beatles, Jimi Hendrix, The Jefferson Airplane, The Grateful Dead, and on and on), happenings and be-ins, bell bottoms and love beads, the Maharishi, patchouli, communes...all of it, the whole glorious, confused gaggle of humanity. I was an eager observer and fellow traveler. While I did tune in and turn on, I never dropped out. I became a ronin scholar with broad interests in the human mind and culture.

I suppose that “Rationalist” is the closest thing this community has as a name for itself. Where psychedelic experience was at the heart of the old counter culture, Bayesian reasoning seems to be at the heart of this counter culture. Where the old counter culture dreamed of a coming Aquarian Age of peace, love, and happiness, this one fears the destruction of humanity by a super-intelligent AI and seeks to prevent it by figuring out how to align AIs with human values.

I’ll leave Bayesian reasoning to others. I’m interested in AI Doom. But to begin understanding that we must investigate what a post-structuralist culture critic would call the discourse or perhaps the ideology of intelligence. To that end I begin with a look at Adrian Monk, a fictional detective who exemplifies a certain trope through which our culture struggles with extreme brilliance. Then I take up the emergence of intelligence testing in the late 19th century and the reification of intelligence in a number, one’s IQ. In the middle of the 20th century the discourse of intelligence moved to the quest for artificial intelligence. With that we are at last ready to think about artificial x-risk (as it is called, “x” for “existential”).

This is going to take a while. Pull up a comfortable chair, turn on a reading light, perhaps get a plate of nachos and some tea, or scotch – heck, maybe roll a joint, it’s legal these days, at least in some places – and read.