Robert Wright talks with Connor Leahy about the cult of AI

YouTube:

Robert Wright and Connor Leahy, US executive director of ControlAI, discuss the history of AI culture and the influences that shaped it—from the hippies, to the rationalists, to Marx, to sci-fi stories, and more. Plus: Mythos as “digital nuke," Dario's real motivations, and Connor's case against any and all attempts at building superintelligence.

Subscribe to The NonZero Newsletter at: https://www.nonzero.org/subscribe
Episode post on Substack: https://www.nonzero.org/p/the-ideolog...
Join NonZero's Discord server: / discord

0:00 Teaser
0:47 Connor’s place in the AI world
3:16 Mythos: “digital nuke”?
14:12 The milieu that birthed Dario and other AI titans
20:52 Yudkowsky, Marx, and other AI harbingers
35:34 Effective altruism’s effects on big tech
46:22 The "cult" in Silicon Valley culture
59:59 Is the AI race a market failure?
1:05:51 What’s really driving the AI titans?
1:19:04 Connor: ASI is a worse bet than Russian roulette

I've been aware of much of this story for some time, though many details are new. Back in 2023 I published an article in 3 Quarks Daily about the cultish nature of Silicon Valley computer culture, A New Counter Culture: From the Reification of IQ to the AI Apocalypse. Nonetheless, hearing the story once again, in one place, with new details has been interesting. It may shed light on just why the dominant AI culture is an technology monoculture. The cultish nature of the surrounding industrial culture has something to do with it.

Note that both Apple (1976) and Microsoft (1975) date back to the personal computer revolution of the 1970s. Amazon was founded when the web was born (1994), as was Google (Alphabet, 1998). Facebook (Meta) didn't emerge until the social web was well-established, 2004. None of them are pure AI companies. They came later; OpenAI in 2015 and Anthropic in 2021. Those were the products of this ingrown culture. The other companies were founded as business ventures. OpenAI and Anthropic were founded as expressions of a quasi-religious vision.

Note: As far as I can tell, Leahy seems rather adjacent and Wright seems to take their technological visions more seriously than is intellectually warranted.