Richard Hanania has posted an interesting conversation with Steve Pinker, Pinker on Alignment and Intelligence as a "Magical Potion", March 17, 2023. Here a long excerpt from Pinker on AI Doom:
There’s a recurring fallacy in AI-existential-threat speculations to treat intelligence as a kind of magical pixie dust, a miracle elixir that, if a system only had enough of it, would grant it omniscience and omnipotence and the ability to instantly accomplish any outcome we can imagine. This is in contrast to what intelligence really is: a gadget that can compute particular outputs that are useful in particular worlds.
That’s an interesting speculation about resistance to IQ denial as a source of support for the concept of superintelligence. I suspect it’s not historically accurate – the superintelligence proponents I’ve seen don’t bring up or refute the Gouldian arguments against IQ, but just seem to operate under the folk theory that IQ is a measure of a magical potion that you can have in various amounts. I may be wrong, but I can’t recall any mentions of the psychometrics or behavioral genetics of intelligence in these discussions.
I think there are many things wrong with the argument that we should worry about AI creating a virus that kills us all.
First, why would an AI have the goal of killing us all (assuming we’re not talking about a James Bond villain who designs an AI with that in mind)? Why not the goal of building a life-size model of the Eiffel Tower out of popsicle sticks? There’s nothing inherent in being smart that turns a system into a genocidal maniac – the goals of a system are independent of the means to achieve a goal, which is what intelligence is. The confusion arises because intelligence and dominance happen to be bundled together in many members of Homo sapiens, but that’s because we’re products of natural selection, an inherently competitive process. (As I note in Enlightenment Now, “There is no law of complex systems that says that intelligent agents must turn into ruthless conquistadors. Indeed, we know of one highly advanced form of intelligence that evolved without this defect. They’re called women.”) An engineered system would pursue whatever goal it’s given.
Sometimes you see the assumption that any engineer would naturally program the generic goal of self-preservation, or self-aggrandizement, at all costs into an AI. No, only an idiot would do that. This is crude anthropomorphization, perhaps Freudian projection.
Second, and relatedly, these scenarios assume that an AI would be given a single goal and programmed to pursue it monomaniacally. But this is not Artificial Intelligence: it’s Artificial Stupidity. No product of engineering (or for that matter natural selection) pursues a single goal. It’s like worrying that since the purpose of a car is to get somewhere quickly, we should worry about autonomous vehicles that rocket in a straight line at 120 MPH, mowing down trees and pedestrians, without brakes or steering. I’ll quote myself again: “The ability to choose an action that best satisfies conflicting goals is not an add-on to intelligence that engineers might slap themselves in the forehead for forgetting to install; it *is* intelligence.” And “Of course, one can always imagine a Doomsday Computer that is malevolent, universally empowered, always on, and tamperproof. The way to deal with this threat is straightforward: don’t build one.”
The third fallacy is one that I mentioned in the excerpt you reposted: that sheer rational cogitation is sufficient to solve any problem. In reality intelligence is limited by knowledge of the world, which is an exponential space of possibilities governed by countless chaotic and random processes. Knowledge of the world is expensive and time-consuming to attain incrementally. Me again: “Unlike Laplace’s demon, the mythical being that knows the location and momentum of every particle in the universe and feeds them into equations for physical laws to calculate the state of everything at any time in the future, a real-life knower has to acquire information about the messy world of objects and people by engaging with it one domain at a time. Understanding does not obey Moore’s Law: knowledge is acquired by formulating explanations and testing them against reality, not by running an algorithm faster and faster. Devouring the information on the Internet will not confer omniscience either: big data is still finite data, and the universe of knowledge is infinite.”
Even the Bond-villain scenario is too facile. As Kevin Kelley noted in “The Myth of the Lone Villain,” in real life we don’t see solitary evil geniuses who wreak mass havoc, because it takes a team to do anything impressive, which multiplies the risk of detection and defection, and it inevitably faces a massively larger coalition of smarter people working to prevent the havoc from happening. And as Kelley and Hanson point out, no technology accomplishes something awesome the first time it’s turned on; there are always bugs and crashes, which would tip off the white hats. This doesn’t guarantee that there won’t be a successful solitary sociopathic AI-virus-designer-designer, but it’s not terribly likely.
Many of the scenarios pile up more layers of Artificial Stupidity, such as assuming that human flesh is a good source of material for paperclips, or even that annihilating humans is plausible means to the end of self-preservation.
The AI-existential-threat discussions are unmoored from evolutionary biology, cognitive psychology, real AI, sociology, the history of technology and other sources of knowledge outside the theater of the imagination. I think this points to a meta-problem. The AI-ET community shares a bad epistemic habit (not to mention membership) with parts of the Rationality and EA communities, at least since they jumped the shark from preventing malaria in the developing world to seeding the galaxy with supercomputers hosting trillions of consciousnesses from uploaded connectomes. They start with a couple of assumptions, and lay out a chain of abstract reasoning, throwing in one dubious assumption after another, till they end up way beyond the land of experience or plausibility. The whole deduction exponentiates our ignorance with each link in the chain of hypotheticals, and depends on blowing off the countless messy and unanticipatable nuisances of the human and physical world. It’s an occupational hazard of belonging to a “community” that distinguishes itself by raw brainpower. OK, enough for today – hope you find some of it interesting.
No comments:
Post a Comment