Tech Utopians and AI Doomers are two sides of the same coin

Matt Yglesias pushes back against Andreesen’s Techno-Optimism, The techno-optimist's fallacy. He sets it up:

Earlier this year, I kept writing draft versions of an article denouncing something I wanted to call “the Techno-Optimist’s Fallacy.”

What is the fallacy? It starts with the accurate observation that technological progress has, on net, been an incredible source of human betterment, almost certainly the major force of human betterment over the history of our species, and then tries to infer that therefore all individual instances of technological progress are good. This is not true. Indeed, it seems so obviously untrue that I couldn’t quite convince myself that anyone could believe it, which is why I kept abandoning drafts of the article. Because while I had a sense that this was an influential cognitive error, I kept thinking that I was maybe torching a straw man. Was anyone really saying this?

Then along came Marc Andreesen, the influential venture capitalist, with an essay that is not only dedicated to advancing this fallacy, it is even literally titled “The Techno-Optimist Manifesto.”

So now I can say for sure that, yes, this is a real fallacy that people are actually engaged with.

I won’t go into the substance of his argument; you can read it for yourself. I want to look around the edge. Early on he observes, “It seems to me that, in practice, almost everyone in the techno-optimist camp lives in the Bay Area...” Toward the end:

I’m focusing on these somewhat outlandish edge cases because the point of Andreesen’s article is to try to salt the conversation around the risks associated with the rapid development of artificial intelligence. He is unalterably opposed to any kind of regulation and massively dismissive of concerns about existential risk, algorithmic bias, and everything else.

And he doesn’t want to argue any of the specifics, he just wants to mobilize general pro-technology, pro-progress, pro-growth sentiment in favor of the specific idea that we should be rushing headlong to try to create an artificial super-intelligence.

My point is simple, the tech-optimists and the AI Doomers are two sides of the same coin, which may be why they’re concentrated in Silicon Valley. One is Kirk, the other is Anti-Kirk, Spock and Anti-Spock, Yin and Yang, more generically, good twin and bad twin, or in a lingo from the belly of this particular beast, Luigi and Waluigi (LessWrong). You can line up the positives and negatives however you please.

All of them are insular, tech-obsessed, intelligence too, and self-regarding, at least on the surface. Who knows what lurks in the hearts of men. Moreover, some of them are very rich, which just magnifies everything else.

I suppose I got this idea from Ted Chiang, for example:

I used to find it odd that these hypothetical AIs were supposed to be smart enough to solve problems that no human could, yet they were incapable of doing something most every adult has done: taking a step back and asking whether their current course of action is really a good idea. Then I realized that we are already surrounded by machines that demonstrate a complete lack of insight, we just call them corporations. Corporations don’t operate autonomously, of course, and the humans in charge of them are presumably capable of insight, but capitalism doesn’t reward them for using it. On the contrary, capitalism actively erodes this capacity in people by demanding that they replace their own judgment of what “good” means with “whatever the market decides.” [...]

The fears of superintelligent AI are probably genuine on the part of the doomsayers. That doesn’t mean they reflect a real threat; what they reflect is the inability of technologists to conceive of moderation as a virtue. Billionaires like Bill Gates and Elon Musk assume that a superintelligent AI will stop at nothing to achieve its goals because that’s the attitude they adopted.