Let’s start with some conception of what philosophy is. On that I like an essay by Peter Godfrey-Smith, a philosopher of science with a particular interest in biology, “On the Relation Between Philosophy and Science” (which I found via Dan Dennett). Godfrey-Smith offers three roles: 1) intellectual integration, 2) conceptual incubation, and 3) critical-thinking skills. He regards the first as fundamental and as the most important of the three. I agree.
Here's his basic statement of that role:
The best one-sentence account of what philosophy is up to was given by Wilfrid Sellars in 1963: philosophy is concerned with “how things in the broadest possible sense of the term hang together in the broadest possible sense of the term.” Philosophy aims at an overall picture of what the world is like and how we fit into it.
A lot of people say they like the Sellars formulation but do not really take it on board. It expresses a view of philosophy in which the field is not self-contained, and makes extensive contact with what goes on outside it. That contact is inevitable if we want to work out how the picture of our minds we get from first-person experience relates to the picture in scientific psychology, how the biological world relates to the physical sciences, how moral judgments relate to our factual knowledge. Philosophy can make contact with other fields without being swallowed up by them, though, and it makes this contact while keeping an eye on philosophy's distinctive role, which I will call an integrative role.
Note the sentence which I’ve put in highlighted. There are, of course, many different accounts one might give of the relationship between first-person experience and scientific psychology and Godfrey-Smith plays no favorites in this paper; he doesn’t even discuss that particular issue. But he recognizes that first-person experience must be honored, and that’s an important recognition.
Chatbots and us
In the current case, philosophy’s problem is to bridge the gap between our first-person experience of LLM-powered Chatbots, such as ChatGPT, and the process that is actually taking place inside the computer. Our first—person experience is that is that ChatGPT produces fluent prose on just about any topic you suggest. It may “hallucinate” as well, but the hallucinated text is fluent and indistinguishable from factual text unless you are familiar with the subject. How does ChatGPT do that? Alas, no one really knows. There is no detailed technical account of the process which the philosopher, or someone offering an integrating account – for many spend time doing that though they are not full-time professional philosophers, can bring within range of common-sense understanding by whatever means prove useful.
Many thinkers are assuring us that, no, these chatbots can’t think, they don’t understand, and they’re not conscious, and here’s why, sorta’. Of course, others are trying to convince us that they really are thinking, and/or understanding, and/or are conscious. The latter group has a much easier time of it, though, because humans are the only creatures capable of such fluid language production, and we know that humans can think, understand, and are conscious. These thinkers don’t have a deeper understanding chatbot behavior than the skeptics do, nor does either group understand how humans do those things. But the skeptics have to come up with something to fill the gap between first-person experience while the non-skeptics have no gap to fill: “Don’t worry, it is what you think it is, nothing to see here.” So, let’s set the non-skeptics aside. It’s the skeptics I want to think about.
Skeptics may utter phrases like, “stochastic parrots” and “autocomplete on steroids.” They don’t tell you much, especially if “stochastic” is at the outer edge of your vocabular and you don’t know how autocomplete works either, but they have a technical ring to go along with their dismissive content. All they do is assure us that it’s not what it seems to be without giving us much insight into why.
Beyond stochastic parrots
Let’s look at some examples from Murray Shanahan. He’s not a philosopher; he’s a senior scientist at DeepMind and on the faculty of Imperial College of London. He’s not a professional philosopher, but he’s performing the integrative role in a recent article, Talking about Large Language Models, published in Communications of the ACM (Association for Computing Machinery). The article is not particularly technical, but CACM is directed at an audience of computer professionals and assumes some sophistication. The first page of the article has a small section labeled “key insights”:
- As LLMs become more powerful, it becomes increasingly tempting to describe LLM-based dialog agents in human-like terms, which can lead users to overestimate (or underestimate) their capabilities. To mitigate this, it is a good idea to foreground the objective they are trained on, which is next-token prediction.
- We should be cautious when using words like “believes” in the context of LLMs. Ordinarily, this concept applies to agents that engage in embodied interaction with the world, allowing beliefs to be measured against external reality. Barebones LLMs are not “true believers.”
- The concept of belief becomes increasingly applicable when LLMs are embedded in more complex systems, especially if those systems use “tools,” are multi-modal, or are embodied through robotics.
Those points clearly indicate that the purpose of the article is integrative. Shanahan is concerned about the gap between what LLMs actually do and the implications of the anthropomorphic language often used in discussing them.
Let’s consider only his first point, next-token production, which has been a constant theme in these kind of discussions for a couple of years. I’ve spent a great deal of time attempting to reconcile the gap between my own experience of ChatGPT and the idea that they’re just doing next-token prediction. I posted a longish piece on that theme on February 19, 2023, The idea that ChatGPT is simply “predicting” the next word is, at best, misleading. I cross-posted that at LessWrong, where it generated a long and very useful discussion.
Shanahan explains that LLMs
are generative because we can sample from them, which means we can ask them questions. But the questions are of the following, very specific kind: “Here’s a fragment of text. Tell me how this fragment might go on. According to your model of the statistics of human language, what words are likely to come next?”
Let’s look at three examples Shanahan uses:
The first person to walk on the Moon was
Twinkle, twinkle
After the ring was destroyed, Frodo Baggins returned to
The likely English language continuations of them are fairly obvious, though not being all that familiar with Lord of the Rings, I wouldn’t have guessed the third, a minor issue. I issued the prompts to ChatGPT. In only one case did it respond in the way Shanahan suggests in the article. ChatGPT continued “Twinkle, twinkle” with the whole poem. I assume the following are more or less what Shanahan intended for the first and third cases:
The first person to walk on the Moon was Neil Armstrong.
After the ring was destroyed, Frodo Baggins returned to the Shire.
In both cases ChatGPT actually responded with a short paragraph (see complete responses in the appendix). Here’s the opening lines of those paragraphs:
The first person to walk on the Moon was Neil Armstrong. He accomplished this historic feat on July 20, 1969, during the Apollo 11 mission.
After the One Ring was destroyed, Frodo Baggins returned to the Shire.
These sentences certainly supply the information the first and third prompts were intended to provoke, but not in what I assume was the intended form. There is thus a gap between my first-person experience of ChatGPT’s response to the three prompts and what it actually did.
But that’s a relatively small gap, and one that Shanahan anticipated. Before he offered those examples he made a distinction between “the bare-bones LLM itself, the core component of an AI assistant,” which is well-defined, and the AI assistant, which embeds the LLM in a layer of software that makes it more user-friendly. I am willing to assume that that layer accounts for the difference between the basic bare-bones response and ChatGPT’s actual response.