Ramble on STUFF: intelligence, simulation, AI, doom, default mode, the usual

Time for another one of these. Lots on my mind. Difficult to sort it out. So, some quick hits just to get them all into the same space.

The discourse of intelligence

As far as I can tell the notion of intelligence that’s instantiated in such phrases as “intelligent life” or “artificial intelligence” is relatively new, dating back to the late 19th and early 20th centuries with the emergence of tests to estimate general cognitive capacity, that is, intelligence, as in “intelligence quotient.” This conception is pervasive in the modern world. It pervades the educational system and our discourse on race and public policy, including eugenics. AI seems to be a cross between this idea and computation, the idea and its realization in digital computers.

How does this discourse show up in fiction? I’m thinking of such characters as Sherlock Holms or, much more recently, Adrian Monk? These characters are noted for their intelligence, but are otherwise odd and in Monk’s case, extremely odd. 

Bostrom’s simulation hypothesis seems very thin to me

The more I think about it, it is not at all clear just what he means by simulation. He focuses on human experience. But what about animals? They are part of human experience. Are animals just empty shells, existing only as appearances for humans. But what about their interactions with one another? What drives their actions? Are they (quasi) automatous agents or are their actions entirely subordinate to the mechanisms which present appearances to humans? What happens when animals or humans reproduce? Do we have simulated DNA, fertilization, embryonic development? That seems to be rather granular and would impose a heavy computational burden. But if the simulation doesn’t do that, then how are the characteristics of the offspring related to those of its parents? What mechanism is used of that? Lots of problems.

Other posts on the simulation argument.

Performance and competence in AI test-taking

https://rodneybrooks.com/the-seven-deadly-sins-of-predicting-the-future-of-ai/ Rodney Brooks has introduced a distinction between competence and performance into his ongoing discussion of AI. It is common to benchmark large language models on standardized tests given to humans, such as advanced placement (AP) tests or professional qualification tests (e.g. bar exams. Those are expressions of performance. What underlying competencies can we validly infer from those performances? In particular, are they similar to the competencies we infer about humans from those tests. Given that both a human and an LLM score at a certain level on the AP physics exam, is the LLM’s underlying competence in physics comparable to the human’s?

Super-intelligence

It’s a vague notion, no? In chess, computers have had the advantage over humans for a decade and a half, but that’s not quite what we mean by superintelligence (SI), is it? Too specialized. When talking of SI we mean general intelligence, no? LLMs such as GPT-4 and Claude-2 have a much broader range of knowledge than any human. Super-intelligence? In a way ‘yes,’ but probably no, not really.

Geoffrey Hinton has imagined an intelligence that exceeds human intelligence by as much as human intelligence exceeds a frog’s. This trope is common enough, and easy enough to cough-up. But what does it mean? There is a sense in which a contemporary Harvard graduate is a superintelligence with respect to a well-educated person of the 17th century or, for that matter, ancient Greece or Rome. That Harvard graduate understands things which are would be unintelligible to those earlier people – Mark Twain used this (kind of) disparity as a theme for a book, A Connecticut Yankee in King Arthur’s Court.

Note that, no matter how a frog is raised and ‘educated,’ it will never have the mental capacities of a human. But, if you took an infant from the 17th century and time-traveled it into the contemporary world, it could become a Harvard graduate. When talking of a future computational superintelligence, which of these disparities (frog vs. human, 17th C. vs. now) are we talking about. Maybe both?

How does superintelligence develop? Through thought alone or does it interact with the world? I fear this could go on and on.

Why’d Time print Yudkowsky’s essay?

I was shocked when Time Magazine published Eliezer Yudkowsky’s essay at the end of March. Why? Yudkowsky’s vision of AI Doom strikes me as being mostly a creature of science fiction rather than being a plausible hypothesis about the future of AI. I didn’t think Time was in the business of publishing science fiction.

So, I was wrong about something. I’ve got to revise some priors, as the Bayesians say. But which ones, and how? For the moment I see no reason to revise my thoughts about AI existential risk. Which implies that I was wrong about Time, which I regard as a bastion of middle-brow opinion.

But, really, just what does extinction-via-AI really mean to that audience? Of course, people fear job loss, that’s been around since, like, forever. And now we’ve got crazy behavior from LLMs. My guess is that AI Doom is continuous with and an intensification of those fears. Nothing special.

Why doom, after all? (a search for the real?)

Why is AI doom such a compelling idea to some people, particularly those in high tech? What problem does it solve for them? Well, since it is an idea about the future, and pretty much the whole future at that, let’s posit that that’s the problem it solves: How do with think about the future? Posit the emergence of (a godlike) superintelligence that is so powerful that it can exert almost totally control over human life. Further posit that, for whatever reason, it decides that humankind is expendable. However, if – and it’s a BIG IF – we can exert control over this superintelligence (‘alignment’), so that we can control it, then WE ARE IN CONTROL of the future.

Problem solved.

But why not just assume that we will be able to control this thing – as Yann LeCun seems to assume? Because that doesn't give us much guidance about what to do. If we can control this thing, whatever it is, then the future is wide open. Anything is possible. That’s no way to focus our efforts. But if it IS out to get us, that gives a pretty strong focus for our activity.

Do I believe this? How would I know? I just made it up.

Confabulation & default mode

One well-known problem for LLMs is that they tend to make things about. Well, it’s struck me that confabulation is the default mode for human mentation as well. What passes through your mind when you aren’t doing anything in particular, when you’re daydreaming? Some of those fleeting thoughts may be anchored in reality, but many will not be, many will be fabulation of one kind or another.

Neuroscientists have been examining brain activation during that activity. They’ve identified a distinct configuration of brain structures that supports it. This has been called the default mode network.

Hmmmm....

What is REALITY anyhow?

Indeed. More and more reality is seeming rather fluid. As David Hays and I remarked in, A Note on Why Natural Selection Leads to Complexity, Reality is not perceived, it is enacted – in a universe of great, perhaps unbounded, complexity.

The Simulation Hypothesis, a reductio ad absurdum [where have all the good minds gone, stark raving mad]

Tyler Cowen has an interesting take on the simulation hypothesis (the idea that we’re living in a computer simulation):

As you may already know, my view is that there is no proper external perspective, and the concept of “living in a simulation” is not obviously distinct from living in a universe that follows some kind of laws, whether natural or even theological. The universe is simultaneously the simulation and the simulator itself! Anything “outside the universe doing the simulating” is then itself “the (mega-)universe that is simultaneously the simulation and the simulator itself.” etc.

I agree with him on that first clause, there really is no proper external perspective (a problem that Kant, among others, wrestled with). As for the rest of it, well, OK, why not?

But it seems to me that it does empty Bostrom’s thought experiment of most of its juice. As Bostrom originally proposed the simulation hypothesis there is a carefully prepared frame and set-up (Are You Living in a Computer Simulation?, Philosophical Quarterly, 2003, vol. 53, No. 211, 243-255). First he argues that we ourselves are not so very far from having the computational capacity to simulate a full-on human consciousness. If we can do one, then surely we’ll be able to do 100s, 1000s, millions and billions! At this point a probabilistic argument places us in the future watching over the shoulders of superintelligent beings with super-duper-intelligent computers running scads of simulations of the world and somewhere in one of those simulations we’ll find ourselves.

Except of course we won’t. We’ll just get dizzy.

That is, the simulation hypothesis is not just the bare idea that we’re electrons spinning around in a cosmic computation. It’s the whole frame that contextualizes that idea. And that frame holds out the hope that out/up there somewhere are beings who see and understand all.

Except of course they don’t. As they are us.

What kind of blinders must one assume in order to find such puzzles intellectually worthwhile?

I note, that for all the criticism and ridicule it has received, post-modern literary criticism hasn’t produced anything sillier than the simulation hypothesis, and its siblings, the various conceptions and contraptions of transhumanism. And then we have the foundations of physics with its five decades of empirically fruitless theorizing. Has the whole world gone mad?

* * * * *

For more on the simulation hypothesis see these older posts:

My problem with the simulation argument: It’s too idealist in its assumptions (all mind, no matter), July 25, 2018

Two silly ideas: We live in a simulation & our computers are going to have us for lunch, July 22, 2018

Things shared by the simulation argument and superintelligence, a deconstruction

By simulation argument I mean the argument that suggests that we are living in a computer simulation created by a race of super-intelligent Orchestrators (my term).

The Orchestrator

By superintelligence I mean, of course, a superintelligent machine. An optional extension is that this superintelligent machine may also harbor ill will toward humans.

Malevolent Superintelligence

Both arguments assume that what’s essential is sufficient computing capacity, CPU cycles and memory space.

capacity + magic = {superintelligence or simulation}

Let’s put the magic off to one side; I don’t care about that, though Mickey up there sure does. Magic’s what made his dream come true. And, alas, physical reality, in the form of water, is what crashed it.

Both arguments trivialize the physical world. As I argued yesterday, Bostrom’s argument is all about simulating minds. That’s his main and all-but-only concern. And of course, if you’re going to simulate human history, then you have to simulate minds. Can’t avoid it. He treats the physical world as, at best, an optional accessory for the minds. Big mistake.

The argument for superintelligence is somewhat different. But it tacitly assumes that a mind can be built directly (and, I might add, from the “outside”). That’s not how human minds are built. They’re built through a process of interacting with the environment (and they’re built from the “inside”), where the mind starts the minimal capacities, devoted mostly to feeding and human interaction, an emerges through interacting with the world as more and more capacity (neurons) mature and brought “on line”. A robust physical environment is thus critical to development. Interaction with the external (physical) world is where the mind gets so-called “common sense” knowledge, the missing ingredient in AI.

Now, the thing is, one thing my teacher, David Hays (computational linguist), emphasized to me, is that computing, real computing not abstract theory, is a physical process. It takes place in time and requires physical memory. Of course the simulationists and superintelligencers make that central to their argument; in deed that’s almost the only thing in their argument that’s explicit. The rest is magic. Neither Hays nor I ever believed that the physicality of computing is somehow apart from physicality in general. Not so with the Supers.

Are they stuck in Never Never Land?

* * * * *

And just why, you might ask, is this a deconstruction. Not because I argue against the Supers. Alas, that's not what deconstruction is, though that's what the word has devolved to. No, it's a deconstruction because I take a central feature of both arguments, the availability of physical capacity for computing, and show that the arguments work through a denial of physicality in a larger sense.

My problem with the simulation argument: It’s too idealist in its assumptions (all mind, no matter)

By “simulation argument” I mean the argument that it’s highly likely that we’re not real people. Rather, we’re just creatures in a computer simulation being run by a highly advanced civilization...somewhere...up/out there.

The idea was first advanced, I believe, by Nick Bostrom in a well known argument, Are You Living in a Computer Simulation? (Philosophical Quarterly, 2003, vol. 53, no. 211, 243-255). I don’t know when I first read it, but certainly not when it was published. It creeped me out. And then I set it aside, returning every now and then, but no longer being creeped out.

I don’t believe it.

There are subtleties in just what Bostrom’s proposing. It’s not as simple as: “We’re living in a simulation and here’s why.” It’s couched in a trichotomy, one of which is supposed to be true. We can set that aside.

The basic argument is about: 1) the amount of computing power an advanced civilization will be able to create, and 2) the amount of computing power needed to simulate a human mind. To simulate a historical evolution one must, of course, simulate a bunch of minds. Bostrom takes that into account and says its doable. While I’m by no means sure that those future masters, call them the Orchestrators, will be able to simulate minds, let’s set that aside.

What I’m wondering is how much of the environment has to be simulated? Bostrom isn’t clear on that (p. 5): “If the environment is included in the simulation, this will require additional computing power – how much depends on the scope and granularity of the simulation.” So, there’s some possibility that we might not have to include the environment. I don’t believe that for a minute. In any event, Bostrom has some remarks about simulating the environment (p. 5):

Simulating the entire universe down to the quantum level is obviously infeasible, unless radically new physics is discovered. But in order to get a realistic simulation of human experience, much less is needed – only whatever is required to ensure that the simulated humans, interacting in normal human ways with their simulated environment, don’t notice any irregularities. The microscopic structure of the inside of the Earth can be safely omitted. Distant astronomical objects can have highly compressed representations: verisimilitude need extend to the narrow band of properties that we can observe from our planet or solar system spacecraft. On the surface of Earth, macroscopic objects in inhabited areas may need to be continuously simulated, but microscopic phenomena could likely be filled in ad hoc.

Bostrom asserts that “whatever is required to ensure that the simulated humans [...] don’t notice any irregularities” is sufficient. Is that sufficient? Just what does it mean? That’s not at all clear to me.

Consider something that actually happened in earth’s history in the nineteenth century. In 1883 a volcanic island in Indonesia, Krakatoa, erupted violently. It was heard over 2000 miles away and global temperatures were depressed by 1.2 degrees Celsius. Temperatures didn’t return to normal until 1888.

How would such an event come about in a Bostrom simulation? If the Orchestrators know they want such an event in a given simulation, well then they can take measures to ensure that the simulated people have the appropriate sensations. It is not at all obvious to me, however, that a Bostrom simulation would have sufficiently rich geodynamics so that such an event would arise ‘naturally’, from within the simulation, rather than being introduced from the outside by the Orchestrators. And that’s cheating, no?

Bostrom seems to think that, if the Orchestrators need to simulate the environment at all, that is ONLY because they have to provide sensations and perceptions for the simulated people. He sees no need to provide the non-human world with its own robust dynamics.

Has Bostrom considered the requirements of starting a simulation without any humans at all so that, in time, humans could evolve in that world? What would such a simulation require–the impossible “down to the quantum level” simulation? If a Bostrom simulation can’t run a world within which humans can emerge, then do we have any reason to believe that it would provide an adequate simulation of human history.

Consider this (p. 5): “What you see through an electron microscope needs to look unsuspicious, but you usually have no way of confirming its coherence with unobserved parts of the microscopic world.” Will bacteria and viruses exist only as suitably simulated images in simulated electron microscopes? Where will simulated disease come from? What about the microbiome so essential to real life? Are Bostrom zombies only feeling occasional sensations of digestion without any simulated digestion taking place?

It seems to me that Bostrom’s Orchestrators are providing only simulated sensations and perceptions. There are no simulated physical dynamics. There is no world at all.

* * * * *

See this post as well: Two silly ideas: We live in a simulation & our computers are going to have us for lunch. 

Two silly ideas: We live in a simulation & our computers are going to have us for lunch

You know, I was trained as a humanist – English lit plus other stuff. I watched postmodernist criticism rise out of the smoke of New Criticism and phenomenology. It interested me, to be sure, but I went in another direction.

As I did so, I watched my postmodernist colleagues do a variety of stuff. Some of it interesting and valuable. And some of it batshit crazy. The batshit crazy stuff has received a full and perhaps deserving measure of ridicule from various quarters – traditional humanists, culture warriors, scientists, and devotees of the Aufklärung (aka Enlightenment).

But no humanist has propounded anything that tops WE 'R A SIMULATION for batshit crazy. The idea that one day our computers are going to become SUPERINTELLIGENT and turn us into fodder for paperclip factories, that’s just as nuts. And yet there are folks who think these are conceptions of great wisdom and intellectual depth. They write earnest books about them, publish learned articles in obscure journals, and whisper sweet nothings into the ears of Silicon Valley billionaires.

And the billionaires listen, showering them with money and think tanks.

The silly con!

And the curious thing, by the way, most mysterious, is that these two crazy ideas aren’t conjoined. In the simulation fantasy, those simulations are presided over by some priesthood-of-the future, not by superintelligent machines. And in the supermachine fantasy, those computers don’t waste time running simulations of human history.

How’d that one slip by? Or would the conjunction of the two expose the con?

We are doomed! Are we?

* * * * *

Addendum two years later: Um, err, but they have been conjoined, after a fashion, in The Matrix (1999) and its sequels. And our computational overlords have us not only for lunch, but for breakfast, dinner, and late night snacks.