I recently posted a long report to the web: ChatGPT: Exploring the Digital Wilderness, Findings and Prospects. That report summarizes and reflects upon the work I did mostly in 2023. But I’ve not stopped working on those issues. So I want to say a bit about that, a bit about what I’m doing now and where I’m going and conclude with some remarks about what I’d originally intended to include in that report, but didn’t.
Phase 1: Exploring ChatGPT
I began writing this report in December 2023 and expected it would take me, say, two or three weeks. I figured I’d be done, possibly by the end of the year, but certainly early in January 2024. Things didn’t work out that way. Why, I don’t quite know – though going on a down turn in 2024 was part of the problem. But not everything.
Anyhow I ended up saying more about ontology than I had originally intended (pp. 34-38) and associative memory (pp. 39-41). I concluded by arguing (p. 42):
I have already argued that the conceptual ontologies underlying human thought are implicit in the structure of LLMs (pp. 14 ff., pp. 35 ff.) – otherwise they would be unable to generate coherent texts. Beyond whatever practical use we can get from LLMs, perhaps the most important intellectual prospect is that of making those implicit ontologies explicit. For it we are to extend the capacities of neural networks with more classical symbolic capabilities, as Gary Marcus and others have been arguing, it would be useful if we could develop programmatic techniques for discovering the ontologies latent in the models.
That’s the important idea, developing programmatic techniques for identifying ontological structure. Just how we’re going to do that, I haven’t the foggiest idea.
My final two paragraphs (pp.43-44):
Think of it like this: Minds, all minds including chimpanzees, gerbils, crows, carp, even octipi, contain large associative memories prompted by external objects and events as they move through the world. With the development of speech, humans gained the capacities both to prompt one another, and to prompt oneself, in ways not directly related to, and thus arbitrary with respect to, immediate external circumstances. The development of writing allowed linguistic prompts to exist in the world separate from the prompter. As a highly specialized and rigorously organized system of prompts, arithmetic (with the decimal point and zero) paved the way for the recognition of a system that has finite parts but can use those to generate infinite sequences. From that we have the abstract Turing Machine leading to the modern digital computer. The transformer architecture in turn allows us to create digital engines that treat the universe of human text as a series of prompts through which it is able to bootstrap a model of that universe. Latent within those models, those LLMs, is an approximation to the metaphysical structure of our universe.
We now have before us the prospect of figuring out the principles on which LLMs are built. With those in hand we can write software to compile that structure into a series of symbolic models. As those models emerge from the neural matrix of the LLMs we can remake the world. As far as I can tell, this is not a five, ten, or thirty-year job. It is the opportunity of a lifetime for new generations, women and men privileged to take up the Star Trek mantra: to boldly go where no man has gone before.
That’s a stronger statement than I had originally intended.
Phase 2: “Opening the hood”
I continue to work with Ramesh Viswanathan, of Goethe University Frankfurt. He’s got mathematical skills that I don’t have, skills that will be necessary to figure out what’s going on under the hood. He’s also got students. He reports progress.
And I’m beginning to get a better idea of what I need to be doing. You can find the barest beginning of those ideas in my report, From LLM mechanisms to ring-composition: A conversation with Claude 3.5. Note that that report takes the form of a conversation with a chatbot, Claude 3.5. Doing that is new, and exciting. What’s important about this report is that I was able to introduce some work I’d previously done on Heart of Darkness as “evidence” for our discussion of how positional encoding in LLMs might be used to encode narrative structure.
What I skipped
When I’d originally planned that quasi-final report, I’d intended to talk about prompt engineering and local cultures of LLM use. I saw those activities as steps on the way to developing programmatic control over the entire model. That is, rather than having one big push that would result in an explicit model covering the whole LLM, I imagined a proliferation of local efforts, each working on the region of the model relevant to their work. I was imagining a process like that involved in settling a new geographic area: You establish settlements here and there. Then you work outward from each settlement until the entire land has become settled.
2024 saw the proliferation of work on prompt engineering and on techniques for wringing more “juice” from the LLMs. In particular, the end of the year saw the emergence of so-called “reasoning” models, and now agents are beginning to show up.
This is all very interesting. I’m not sure what to make of it. For the most part the machine learning mainstream seems to have decided that this is what we’ve got, this is what we’re going to build on from now until, well, either the machines take over or the end of time. The idea that we might develop symbolic means to cover this area, that’s not being given a second thought.
I’m inclined to think that most of this work will turn out to have been work-arounds, stuff we build for now with the tools we’ve got because we don’t have any better tools. Of course, the people doing the work don’t imagine that we’ll see a class of better tools, one that include symbolic capabilities. I think they’re wrong.
Anyhow, as a consequence of this faith in the current intellectual regime, we’re seeing grandiose plans for spending 100s of billions of dollars constructing infrastructure to support all this computing, powerplants and server farms. No doubt some of this will be useful but, on the whole, I think it’s daft.
Perhaps I should say a bit more about all of this. But not now. Later.
No comments:
Post a Comment