MORE on the issue of meaning in large language models (LLMs)

I've been having a long discussion with gjm over at LessWrong about a post I'd originally published here at New Savanna back on March 11,  The issue of meaning in large language models (LLMs). I am now considering a position that is somewhat different from the one I had originally argued.  The position I am currently considering is based on a 2014 article in The New York Review of Books in which Searle takes on two recent books:

Luciano Floridi, The 4th Revolution: How the Infosphere Is Reshaping Human Reality, Oxford University Press, 2014.

Nick Bostrom, Superintelligence: Paths, Dangers, Strategies, 
Oxford University Press, 2014.

Searle’s argument depends on understanding that both objectivity and subjectivity can be taken in ontological and epistemological senses. I’m not going to recount that part of the argument. If you’re curious what Searle’s up in this business to you can read his full argument and/or you can read the appendix to a post from 2017.

Searle sets up his argument by pointing out that, at the time Turing wrote his famous article, “Computing Machinery and Intelligence”, the term “computer” originally applied to people (generally women, BTW) who performed computations. The term was then transferred to the appropriate machines via the intermediary, “computing machinery”. Searle observes:

But it is important to see that in the literal, real, observer-independent sense in which humans compute, mechanical computers do not compute. They go through a set of transitions in electronic states that we can interpret computationally. The transitions in those electronic states are absolute or observer independent, but the computation is observer relative. The transitions in physical states are just electrical sequences unless some conscious agent can give them a computational interpretation.

This is an important point for understanding the significance of the computer revolution. When I, a human computer, add 2 + 2 to get 4, that computation is observer independent, intrinsic, original, and real. When my pocket calculator, a mechanical computer, does the same computation, the computation is observer relative, derivative, and dependent on human interpretation. There is no psychological reality at all to what is happening in the pocket calculator.

Searle goes on to say:

Except for the cases of computations carried out by conscious human beings, computation, as defined by Alan Turing and as implemented in actual pieces of machinery, is observer relative. The brute physical state transitions in a piece of electronic machinery are only computations relative to some actual or possible consciousness that can interpret the processes computationally.

What if I take the same attitude toward LLMs? That's the position I'm currently considering. To paraphrase Searle:

The brute physical state transitions in a piece of electronic machinery are only meaningful relative to some actual or possible consciousness that can interpret the processes as being meaningful.

It seems to me, then, that the question of whether or not the language produced by LLMs is meaningful is up to us. Do you trust it? Do WE trust it? Why or why not?

That's the position I'm considering. If you understand "WE" to mean society as a whole, then the answer is that the question is under discussion and is undetermined. But some individuals do seem to trust the text from certain LLMs at least under certain circumstances. For the most part I trust the output of ChatGPT and GPT-4, with which I have considerably less experience than I do with ChatGPT. I know that both systems make mistakes of various kinds, including what is called "hallucination." It's not clear to me that that differentiates them from ordinary humans, who make mistakes and often say things without foundation in reality.

Note that I am considering this position without respect to the nature of the processes operative in those LLMs. This is, of course, somewhat different from the case with e.g. pocket calculators, where we do understand those physical processes. Nonetheless, at this point this trust in these strange new devices, if only provisional, seems warranted. I cannot same, though, about trust in the users of these strange new devices. Bad actors are and will continue to use them for purposes of fraud, deception, and (political) manipulation. That's not the fault of these devices. Those faults are in us.

Let's return to Searle. Toward the end of the article he takes up the question of consciousness:

Suppose we took seriously the project of creating an artificial brain that does what real human brains do. As far as I know, neither author, nor for that matter anyone in Artificial Intelligence, has ever taken this project seriously. How should we go about it? The absolutely first step is to get clear about the distinction between a simulation or model on the one hand, and a duplication of the causal mechanisms on the other. Consider an artificial heart as an example. Computer models were useful in constructing artificial hearts, but such a model is not an actual functioning causal mechanism. The actual artificial heart has to duplicate the causal powers of real hearts to pump blood. Both the real and artificial hearts are physical pumps, unlike the computer model or simulation.

Now exactly the same distinctions apply to the brain. An artificial brain has to literally create consciousness, unlike the computer model of the brain, which only creates a simulation. So an actual artificial brain, like the artificial heart, would have to duplicate and not just simulate the real causal powers of the original. In the case of the heart, we found that you do not need muscle tissue to duplicate the causal powers. We do not now know enough about the operation of the brain to know how much of the specific biochemistry is essential for duplicating the causal powers of the original. Perhaps we can make artificial brains using completely different physical substances as we did with the heart. The point, however, is that whatever the substance is, it has to duplicate and not just simulate, emulate, or model the real causal powers of the original organ. The organ, remember, is a biological mechanism like any other, and it functions on specific causal principles.

That, it seems to me is the question: How much of specifically human biochemistry is essential for duplicating the causal powers of the human brains? Perhaps the answer is zero, in which case so-called strong AI is in business and one day we’ll see computers whose psychological behavior is indistinguishable from that of humans and, who knows, perhaps superior. 

 

Note, however, that I regard the question of consciousness as being different from that of meaning. The question of consciousness is about the nature of the underlying physical process. The question of meaning, as far as I can tell, is NOT about the nature of the underlying physical process. It is about the fidelity of that process. It is not clear to me that the nature of the process must be identical to the human process in order for its fidelity to be adequate for a wide, if unspecified, range of human purposes.

 

In accepting text from LLMs (such as ChatGPT and GPT-4) as meaningful, I do not mean to imply that I think they are conscious. I see no evidence of that. Given my recent string of posts about polyviscosity (my coinage, though apparently a hyphenated version of the term is in use for a somewhat different meaning), I do consider consciousness an essential property of human minds as, in my view (derived from William Powers) it is essential to memory. And then we have the idea of thinking. Do I believe that meaningful language from LLMs implies that they are thinking? How do I know, I just made this stuff up. It's under advisement, all of it, whatever "it" refers to.

Getting back to meaning, there is an escape clause in what I've said, the range of trust-justified human purposes must be specified. Do I think that LLMs are currently capable of creating high-quality works of literary art? No, I do not. Could they be of use to humans interested in creating such works? Sure, why not? (Remember Burroughs and his cut-ups.) Will LLMs ever be capable of creating high-quality works of literary art? If by LLMs we are to understand devices built on current architectures, then, no, I do not think they will ever be capable of creating high-quality works of literary art. Why not? For the standard science fiction reason, emotion. These LLMs don't have access to anything like the motivational and emotional circuitry of a human brain. For the purposes of creating art, that is problematic. What about future devices? Well, what about them?

More later.

Addendum (later in the day): At the moment it seems to me that the question of substrate independence must be raised independently for meaning and consciousness. It may hold for meaning, but not consciousness.