Language is a lower-dimensional projection of high-dimensional neural dynamics.

But it also allows for content addressed memory. That’s very important, for it gives fine-grain control over the memory and planning systems. That’s the job of sentence-level syntax together with discourse structure.

“Classical” semantic or cognitive networks had a problem with coming up with just the right set of node types and arc types. David Hays dissolved the problem in his 1981 book, Cognitive Structures (scan down the page), by grounding cognition in an analog system modeled on William Powers perceptual control stack (in Behavior: The Control of Perception, 1973). The identity of a cognitive node is a function of its parameter values, where the parameters are derived from the control stack. The identity of the arcs is a function of the difference in parameter values between the nodes it connects.

Concerning Chomsky’s approach to syntax: It depends on a sharp distinction between grammatical and ungrammatical sentences. A generative grammar, in Chomsky’s theory, must account for all and only the grammatical sentences.

However, there are no explicit criteria for separating sentences into the two categories, grammatical and ungrammatical. Rather, the separation depends on the intuitions of the linguist. Naturally enough, different syntacticians have different intuitions. The problem is insoluble.

Moreover, anyone who pays close attention to real speech soon realizes that people do not (always) speak in complete grammatically correct sentences. Real language is sloppy, but nonetheless effective. A neural net of very high dimensionality can deal with this readily enough. A purely symbolic system cannot. Augmenting the system through fuzzy logic and the like doesn’t fix the problem.

LLMs provide a very useful simulacrum of the natural language system. Since LLMs are trained on written texts, the resulting model necessarily conflates the functions of semantics and syntax/discourse. Thus they cannot achieve the flexibility and precision of the full system, where semantics and syntax/discourse are separated.

Hays, Forth, and creativity in man-machine-interaction

I've just been talking with ChatGPT about David Hays back in the 1970s. From my prompt:

In 1977 or 1978 he got a personal computer, one of those based on the Z-80 chip, and started learning to program it in Forth. He also sent out a weekly snail-mail newsletter entitled “The Diary of a Man and His Machine.” In one issue he asked for replies from people whose work feel between the cracks. He got about 100 replies. We read them all. We concluded that most of them were just oddballs; they didn’t even seem to know the standard literature in the field(s) relevant to their work. But a handful seemed legit. I don’t think we ever did anything with them.

...working between the cracks is not the same as working without foundations

From ChatGPT's response:

First, David Hays’s early adoption of a Z-80–based personal computer and Forth in 1977–78 places him well ahead of the curve, not just technically but cognitively. Forth, in particular, is telling: it is intimate with the machine, minimalist, stack-based, and reflective. It invites thinking about process, control, and representation in ways that high-level languages often conceal. That choice alone aligns with his lifelong interest in cognitive architecture rather than surface performance.

working between the cracks is not the same as working without foundations

Later:

Third—and this is perhaps the most interesting part—the call for people whose work “fell between the cracks,” and the response it generated, reads like an early, analog precursor to today’s online intellectual ecosystems. The fact that you and Hays received about a hundred replies, and that most of them struck you as oddballs, is not a disappointment; it’s a sociological constant. When disciplinary boundaries loosen, noise floods in first. What matters is that a handful were clearly legitimate, even if nothing came of it at the time. [...] Finally, your retrospective judgment—that many respondents lacked grounding in the relevant literature—underscores a distinction you’ve been circling throughout this chat: working between the cracks is not the same as working without foundations. Hays was interested in the former, not the latter.

Another prompt:

Your comments about Forth are apt. Hays valued the intimacy with the machine. He eventually programmed a whole environment using Forth and used it for keeping notes and generating text. Some of the text for our articles came out of that machine. That detail completes the picture—and it makes the point even stronger.

Chat replies:

What David Hays was doing with Forth was not “early word processing” or “personal productivity.” He was building a cognitive prosthesis—a writable environment in which thinking, note-keeping, and text generation were continuous with computation rather than merely supported by it.

A few things are worth making explicit.

First, the intimacy you mention is decisive. Forth collapses distance between intention and execution: no elaborate compilation pipeline, no abstraction layers to hide what the machine is doing. That intimacy encourages a style of thought in which process is visible. For someone like Hays—already committed to understanding cognition as structured activity—that mattered more than convenience or speed.

Second, the fact that he built an entire environment—not just scripts, but a living workspace for notes and text—means that the machine was not ancillary. It was part of the thinking loop. That some of the prose in your joint articles emerged from that environment is not incidental; it means the machine participated, materially, in the production of theory. This is not metaphorical augmentation. It is literal.

How do we think together? [Tyler Cowen]

Tyler Cowen has a very interesting post today (April 9, 2025): Why not inquire together more? He sets things up by quoting Robin Hanson. Here's Cowen's complete response to that passage:

I find that “inquiring together” works best when you are traveling together, and confronted with new questions. They can be as mundane as “do you think the two people at that restaurant table are on a first date or not?” From the point of view of the observers, the inquiry is de novo. And the joint inquiry will be fun, and may make some progress. You both have more or less the same starting point. There isn’t really a better way to proceed, short of asking them.

For most established social science and philosophy questions, however, there is so much preexisting analysis and literature that the “chains of thought” are very long. The frontier point is not well maintained by a dyadic conversation, because doing so is computationally complex and further the two individuals likely have at least marginally separate agendas. So the pair end up talking around in circles, rather than progressively. It would be better if one person wrote a short memo or brief and the other offered comments. In fact we use that method frequently, and fairly often it succeeds in keeping the dialogue at the epistemic frontier.

I find that when two people converse, they often make more progress by joking, and one person (or both) taking some inspiration or insight from the joke. As the joke evolves through time, and is repeated in different guises, each person — somewhat separately — refines their intuitions on the question related to the joke. The process is joint, and each person may be presenting new ideas to the other, but the crucial progress-making work still occurs individually.

When people do wish to “talk through a question with me,” I find I am personally most useful offering reading references (I do have a lot of those), rather than ideas or analysis per se. The reading reference is a short computational strand, and it does not require joint, coordinated maneuvering at the end of very long computational strands.

Sometimes Alex and I make progress working through problems together, most of all if it concerns one of our concrete projects. But keep in mind a) we have been working together pretty closely for 35 years, b) often we are working together on the same concrete problem and with common incentives, c) we are pretty close to immune when it comes to offending each other, and d) our conversations themselves do not necessarily go all that well. So I view this data as both exceptional (in a very good way), and also broadly supportive of my thesis here.

For related reasons, I am most optimistic about “inquiring together more” in the context of concrete business decisions. Perhaps John and Patrick Collison are pretty good at this?

Or so it seems to me. Maybe I should go ask someone else.

After quoting the paragraph where Cowen talks of joking, which I liked, I say this:

David Hays and I worked together quite closely for two decades, from the mid-1970s (when I became his student) through to the mid-1990s (when he died). He had been a first-generation researcher in machine translation and thus one of the founders of computational linguistics (a term he coined). By the time I began working with him he was interested in semantics and how cognition was grounded in sensorimotor perception and action (perceiving and moving).

Our interests and skills were complementary. He'd been trained in the social sciences at Harvard and had that methodology down. He was mathematically sophisticated and had technical skills that I lacked. But I had sophisticated mathematical intuition and was a killed analyst of literary texts. We both liked to draw diagrams and worked together on models of mind best expressed in diagrams. My 1978 dissertation, "Cognitive Science and Literary Theory" was Both a quasi-technical exercise in cognitive science and examination of two literary phenomena, 1) the long-term cultural evolution of narrative form, 2) a detailed semantic analysis of Shakespeare's sonnet 129, "The Expense of Spirit." Once I'd completed my degree we continued to work closely together on our common intellectual project. My point is simple: we were deeply familiar with one another's thoughts and had complimentry skills.

Periodically I would visit him in Manhattan for two or three days and we'd work. Sometimes we'd stay in his apartment and think. And sometimes we'd walk in nearby Fort Tryon Park. Inevitably we'd get to a point where we were stuck. Here's a passage from my eulogy:

This ritual began when both of us were exhausted from the intellectual work, and frustrated because we weren’t making progress. Each of us would lie back and drop into fitful reverie. Every so often one of us would make a comment or ask a question. The other would reply, to no mutual satisfaction, and the fitful reverie would continue. Eventually we would work through it, begin talking and talking, and Dave would sit down to the computer and write up some notes on what we had accomplished.

Looking back I surmise that the point of all the talk was to get us to the point where we could no longer talk. The deep work happened during those (mutual) reveries.

To which I appended this:

Perhaps our single deepest paper is "Principles and Development of Natural Intelligence." I forget just how we decided to pool our knowledge and write something about the brain, a mutual interest we'd been pursuing independently for awhile. The actual work began when we met at my parents house in Allentown, PA. We sat at the kitchen table, pen and paper in front of us, and began listing the various ideas, observations, models, etc. we thought should be included. When the list had reached, say, fifty or so items, I suggested that we start grouping them into piles that seemed to go together. That gave us five piles. Hays then suggested that we come up with a principle for each pile. I forget whether or not we named any of the principles in that session. But however that actually happened, we did end up with five: 1) mode, 2) diagonalization, 3) decision, 4) finitization, and 5) indexing. In the paper we identified brain structures having primary responsibility for implementing each principle and associated each principle with characteristic behaviors. FWIW the final paper had 15 diagrams or illustrations. I take that as an indication of how much our thinking depended on visualization.

As a final comment, I note that my book about music, Beethoven's Anvil, was pretty much about how individual minds collaborate, albeit in making music rather than discursive thinking. "How do we THINK together may be as deep a question as one can ask about homo sapiens sapiens."

David Hays, language and love + Leslie Farber [why techbros are dangerous]

I first learned of David Hays, who would become my teacher, mentor, and colleague, from an article he published in Dædalus (Vol. 102, No. 2, 1973): “Language and Interpersonal Relationships.” I note, however, that that’s not the title he gave it. That was the editor’s title. His title: “How Now, I and Thou?” He begins with a question: “How does language engender love?” He then sketches a possible answer in the rest of the article, drawing on his knowledge of psychology, sociology, linguistics, and computing. He was, after all, one of the founders of computational linguistics.

On the second page of the article Hays remarks:

According to Martin Buber, the conversation of friends and lovers serves, at its best, to confirm them as particular human beings. The psychoanalyst Leslie H. Farber puts it this way: “Real talk between a man and a woman offers the supreme privilege of keeping the other sane and being kept sane by the other.” To which I would add that good talk also makes each aware of his own sanity.

Farber’s remark has stuck with me ever since I read Hays’s article back in the summer of 1973, though I can never remember the exact wording.

Last night I went looking for Farber on the internet and found a book review by Anatole Broyard, which was published in The New York Times (June 5, 1976), which quotes that sentence near the end. Here’s how the review opens:

“The attempt of the will to do the work of the imagination”: W. B. Yeats applied this phrase to an incorrect approach to poetry. In “Lying, Despair, Jealousy, Envy, Sex, Suicide, Drugs, and the Good Life,” Leslie Farber applies it to an incorrect approach to life. Ours, he says, is the age of the disordered will. It is our conceit that no human possibility is beyond our conscious will. T. S. Eliot had something similar in mind when he said that the bad poet is conscious when he should be unconscious, and unconscious when he should be conscious.

LYING, DESPAIR, JEALOUSY, ENVY, SEX, SUICIDE, DRUGS, AND THE GOOD LIFE. By Leslie H. Farber, 232 pages. Basic Books. $10.

Trying to will what cannot be willed, according to Mr. Farber, brings on anxiety, and this anxiety, in turn, cripples our other faculties so that we are left with nothing but anxiety about anxiety, a double unease. Among the things we try to will are happiness, creativity, love, sex and immortality. Sex has been emancipated from a repressive morality only to fall a victim to our coercive will. Instead of experiencing or knowing sex, we increasingly tend to know about it. The inappropriate intrusion of will has the effect of distancing us from emotion, substituting the theoretical for the phenomenal.

That sentence hit me, hard: “It is our conceit that no human possibility is beyond our conscious will.”

That encapsulates what is so very wrong with homo economicus, and with AI. And that is what drivers the Doomers crazy, that through an act of conscious will they should create something that is beyond both their will and their understanding. THAT, more than anything else, is what makes AI so dangerous. The technology is currently dominated by people who can see nothing beyond their will. They are blind.

Claude and I discuss the idea of progress in the arts, from Hazlitt to Bloom on Shakespeare, and concluding with Arthur Danto and Ernst Gombrich

This is from an ongoing discussion I have been having with Claude. For this discussion I had uploaded the paper that David Hays and I wrote on cognitive evolution, which is our basic paper of cultural ranks, and Max Tabarrok’s post, Romae Industriae, which poses the question of why Rome didn’t have an industrial revolution. I have posted earlier segments of this conversation: 1) Why didn’t Rome have an industrial evolution? and 2) where I argued that the printing press did not play the role in cultural history that is usually assigned to it.

The general idea of cultural ranks came out of my undergraduate years at Johns Hopkins. I took a course on the Scientific Revolution, where we read Thomas Kuhn’s famous book, The Structure of Scientific Revolutions. In another course I learned that the novel emerged as a literary form in roughly the same time frame. In yet another course I read Centuries of Childhood, by Philippe Aries. He argued that the conception of childhood that is familiar to us, a distinct phase in human development, was not a “natural” concept, but rather emerged, yes, again within roughly the same time period. In a course on the theatre, taught by Dick Macksey, I read Nietzsche’s The Birth of Tragedy, which concocted a story about the historical emergence of tragedy in ancient Greece which made it seem as though the Athenians had “swallowed” an older society and put it on the stage. Ritual celebrants in the older social formation became the principle characters in tragedies while the general popular of that formation became the chorus.

During my sophomore year I’d taken a course in developmental psychology where I was introduced to the work of Jean Piaget. Piaget theorized that children’s mental development progressed through a series of more or less distinct stages, where each stage is characterized by specific cognitive capabilities, with later stages building on earlier ones. He also applied this concept to the history of ideas, e.g. in Genetic Epistmology, which read while working on an MA thesis. That’s also when I read Walter Wiora’s The Four Ages of Music. It was a slim volume in which Wiora argued that the first age emphasized rhythm, the second added an intense development of melody, then came harmony (emerging in Europe) and the contemporary phase, which involved exploration of new forms and methods. I read that and posed a question: Isn’t everything like that, four stages?

I took that question with me to SUNY Buffalo, where I enrolled for a PhD in the English Department, but also spent a great deal of time discussing computational linguistics, cognitive science, and cultural evolution with David Hays in Linguistics. That’s where we developed the idea of cultural ranks. I included an account of narrative development in my 1978 dissertation, “Cognitive Science and Literary Theory.”

I note, then, that while the scientific revolution certainly played an important role in my thinking about cultural evolution, the arts played perhaps an even more important role in developing the idea. However, the ranks idea only accounts for the overall shape of development. What about development within a given rank? I have some preliminary notes about that in a recent post, Confabulation, Dylan’s epistemic stance, and progress in the arts: “I’ll let you be in my dreams of I can be in yours.”

The rest of this post consists of a conversation I had with Claude 3.7 on March 3, 2025. As always, my comments are set in bold type while Claude’s are plain.

* * * * *

Hazlitt on progress in the arts

While it is common to say that there is progress in the sciences and technology, it’s less common to think of progress in the arts. William Hazlitt published a well-known essay in 1814, Why The Arts Are Not Progressive, and that’s remained the standard view. He observed:

“What is mechanical, reducible to rule, or capable of demonstration, is progressive, and admits of gradual improvement: what is not mechanical or definite, but depends on genius, taste, and feeling, very soon becomes stationary or retrograde, and loses more than it gains by transfusion. [...] The greatest poets, the ablest orators, the best painters, and the finest sculptors that the world ever saw, appeared soon after the birth barbarous. Those arts, which depend on individual genius and incommunicable power, of these arts, and lived in a state of society which was, in other respects, comparatively have always leaped at once from infancy to manhood, from the first rude dawn of invention to their meridian height and dazzling lustre, and have in general declined ever after.”

I see you're interested in exploring the contrast between progress in sciences/technology versus the arts. This is a fascinating philosophical question that bridges cultural evolution, aesthetics, and the nature of human creativity.

Hazlitt's essay presents the classic view that the arts, unlike sciences, don't experience linear progress. His central argument distinguishes between domains that are "mechanical, reducible to rule" (sciences) versus those dependent on "genius, taste, and feeling" (arts). He suggests artistic achievements often peak early in civilizations and then stagnate or decline.

The two documents you've shared offer interesting frameworks to consider this question: From "Cognitive Evolution," Benzon and Hays propose a four-rank model of cognitive development across cultures:

1. Rank 1: Abstraction through metaphor (preliterate societies)
2. Rank 2: Rationalization through metalingual definition (literate societies)
3. Rank 3: Theory through algorithm (Renaissance onward)
4. Rank 4: Model building through control (20th century)

They argue these cognitive advances shape all cultural domains, not just science. While they don't specifically refute Hazlitt's view on art, their model suggests that artistic expression would also evolve with new cognitive tools.

The "Romae Industriae" document examines why Rome didn't achieve an industrial revolution despite its technological sophistication. The author concludes cultural factors, particularly the lack of a printing press and the disdain for mechanical arts in slave-based societies, were key constraints rather than just technological limitations.

What's interesting is how these perspectives might inform a response to Hazlitt. Could we argue that while individual artistic genius may appear at any time, the evolution of artistic forms and movements does show a pattern of development tied to cognitive and cultural evolution? For example, the Renaissance's perspective techniques in painting required Rank 3 thinking about algorithmic representation of space.

Would you like to explore this tension between Hazlitt's view and the cognitive evolution framework more deeply, or focus on another aspect of these materials? 

Bloom, Shakespeare, the novel

Ah, yes, this ranks business. Benzon has published on both the evolution of narrative and the evolution of music using ranks theory and Hays has published on expressive culture in general. Perhaps one day I’ll upload those papers for you to examine. I note, though, that they don’t address the issue of progressive change within individual ranks. For example, I suspect that there is expressive change in the novel, from the 18th century up through the end of the 19th century, but that’s entirely within Rank 3 culture, and I’d think the progress continues in the 20th century. [...] And then we have Harold Bloom’s assertion that Shakespeare, in some sense, “invented” the human, but which I think he means that Shakespeare was the first writer to give voice to certain aspects of human behavior and mental life. But I don’t think Bloom ever systematically developed that idea, not even in is book on the Western canon or in his Shakespeare book.

[...] You raise an excellent point about progressive change within individual ranks that wasn't fully addressed in the paper. The progression of the novel within Rank 3 culture is a perfect example of this phenomenon. From Defoe and Richardson through Austen, the Brontës, Dickens, Eliot, and on to James and Conrad, we see remarkable development in narrative technique, psychological depth, and formal complexity—all while remaining within the algorithmic thinking characteristic of Rank 3.

This intra-rank progression continues into the 20th century with modernist innovations by Joyce, Woolf, and Faulkner, who pushed the boundaries of what the novel could express while still operating within Rank 3 cognitive structures. Then perhaps writers like Pynchon and DeLillo begin to engage with Rank 4 model-building approaches to fiction.

Regarding Bloom's assertion about Shakespeare "inventing the human," this does seem to suggest a kind of progressive leap in artistic representation. Bloom never fully systematized this idea, but it appears to indicate that Shakespeare's achievement wasn't just artistic excellence but a cognitive breakthrough in representing human interiority. In ranks terminology, one might say Shakespeare was pushing against the upper limits of Rank 2 representation toward something like early Rank 3 thinking about human psychology, even before the scientific revolution had fully taken hold.

LLM as Collaborator, Part 1: Claude the Graduate Student

I started using ChatGPT on December 1, 2022 and have used it quite extensively ever since. I’ve spent some of my time just poking around, somewhat more time looking things up, and most of my time systematically investigating its performance. That resulted in a number of working papers, the most interesting of which is about stories: ChatGPT tells stories, and a note about reverse engineering: A Working Paper, Version 3.

I started working with Claude 3.5 Sonnet on November 18, 2024. I’ve used it in those three capacities, though obviously not as much as I’ve used ChatGPT. In particular, I’ve used it for background information on melancholy in various aspects. I’ve also done something I’d never done with ChatGPT, asked it to describe photographs. I’m doing this to see how well it does.

Then on November 24, 2024, I began using it in a somewhat more interesting new capacity, though I’m not sure what to call it. The phrase “thought partner” comes to mind, though it seems too much like “thought leader,” which I don’t like. I’m using it as a sounding board. Better yet, it’s a collaborator playing the role of sounding board. It’s not an equal collaborator in the intellectual dialog; academic norms would not require me to offer it co-authorship of papers. But those norms might well require an explicit acknowledgement, not to alert the reader that one of those new-fangled LLM things has been involved in the thinking, but simply acknowledging the help it has given me.

As for just what kind of help that is, the best way is to look at some examples. I’ve already published two of these dialogues on New Savanna: Computer chess, super-intelligence, A Connecticut Yankee, cargo cults, and Claude 3.5 Sonnet speculates about future developments in AI and beyond. The rest of this post contains the first interaction where Claude began acting in intellectual partnership.

When I started the interaction I was simply interested in seeing what it knew about my teacher and mentor, the late David Hays. I wasn’t looking for new information. I just wanted to see what it had picked up. That began to change when it asked me about the work of Quillian and of Schank and Abelson. Then I told it about Hays’s interest in the work of William Powers. Now we’re getting serious. This guy knows something. And then WHAM! Claude responded intelligently when I mentioned the work of Peter Gärdenfors. The interaction continued in that mode to the end.

Utterly remarkable. AGI? What the hell is that? But no. And I fear that thinking in those terms is likely to blind me to whatever it is that Claude is doing. I realize that the concept of AGI is written into the contractual relationship between OpenAI and Microsoft, but I’m not playing that game. Will they end up litigating it? Who knows?

I’m planning two more posts in this series. The next will be about the implications of paragraph length in Heart of Darkness. Then I’ll post a long interaction that takes Miriam Yevick’s 1975 paper, “Holographic or fourier Logic,” as its starting point.

In the following dialog my comments are bold while Claude’s are plain. I’ve interpolated a few comments highlighted in blue.

* * * * *

Are you familiar with the work of a computational linguist named David Hays? He worked on machine translation at RAND in the 1950s and 1960s?

Yes, I am somewhat familiar with David Hays' work, though I should note that since this involves fairly obscure historical details from computational linguistics, I want to be careful about potential inaccuracies in my knowledge. From what I understand, David Hays was one of the pioneers of machine translation and computational linguistics during the early years of the field. He worked at RAND Corporation in the 1950s and 1960s where he was involved in some of the earliest machine translation projects.

He was particularly interested in developing dependency grammar approaches for machine translation, which was quite innovative for that time period. Dependency grammar focuses on the relationships between words rather than constituent structure, which was the dominant paradigm in American linguistics at the time.

I believe he also wrote an influential book called "Computational Linguistics: Programming Linguistics in Automatic Language Translation" in the early 1960s, though I'd encourage you to verify this citation since I may be misremembering details.

Claude got the name wrong. It’s Introduction to Computational Linguistics, American Elsevier, New York, 1967.

Would you be interested in hearing more about any particular aspect of his work or that early period of machine translation research? I find the history of early computational linguistics fascinating, though again I want to be upfront that with someone as historically specific as Hays, some of my knowledge may be incomplete or imperfect.

Yes, that’s him. I studied with him at SUNY Buffalo in the 1970s. We were working on computational semantics and using a cognitive networks formalism.

Oh, that's fascinating that you studied directly with David Hays! I would be very interested to hear more about your experiences working with him and the cognitive networks approach you were developing at SUNY Buffalo. That was an important period in the development of semantic representation systems.

Could you tell me more about the cognitive networks formalism you were using? I'm particularly curious about how it compared to other semantic network approaches being developed around that time, like Quillian's work or Schank and Abelson's conceptual dependency theory.

It was more developed than Quillian’s work. As I recall, he never really developed is initial ideas. As for Schank and Abelson, they were imagining a self-sufficient cognitive system and were very much interested in identifying the primitive elements necessary for a cognitive network. By the time I began working with him, Hays had realized that cognition had to be grounded in something else – later on Steven Harnad would talk about the “symbol grounding” problem. When I worked with him, Hays was interested in grounding the cognitive system in the control-theory (cybernetic) account of perception and action developed by William T. Powers (perceptual control theory, or PCT).

Something special seems to be happening.

This is really interesting historical context - it suggests Hays was grappling with fundamental issues in cognitive science that are still relevant today. The shift from viewing semantic networks as self-contained symbolic systems to recognizing the need for grounding in perception and action seems quite prescient.

I'm curious about how Hays was trying to integrate Powers' PCT with semantic networks. As I understand it, PCT proposes that behavior emerges from organisms controlling their perceptions to match internal reference signals, rather than through direct control of outputs. Was Hays trying to use this as a way to ground symbolic meaning in perceptual-motor control loops?

This also seems to anticipate some later developments in embodied cognition and grounded cognition theories. The timing is interesting too - Powers published "Behavior: The Control of Perception" in 1973, so it sounds like Hays was engaging with these ideas pretty quickly after they were published.

Yes, that's what Hays was doing. His idea was that particular servo units in a Powers-like control system would be linked to cognitive units through what Hays called "parameters of perception." Hays identified a half dozen or so such parameters. Each cognitive node would be characterized by the values it took on those parameters. Relationships between nodes were thus derived from differences in parameter values for the nodes connected by an arc (or edge).

I’m Four Degrees of Separation from Bertrand Russell

Bertrand Russell taught Ludwig Wittgenstein, who in turn taught Margaret Masterman. Masterman is one of the founders of computational linguistics, as is my teacher, David Hays. I don’t know much about their intellectual relationship, but I know it was substantial. Hays had a white sweater which she had knitted for him and she was a close intellectual confidant. Hays hired one of her students, Martin Kay, to work with him at RAND.

These four relationships, 1) Russell & Wittgenstein, 2) Wittgenstein and Masterman, 3) Masterman & Hays, and 4) Hays & Benzon, are all substantial ones and not of mere acquaintance. Did I get any “juice” from Russell through that chain that I hadn't already picked up by reading Russell and Wittgenstein before I ever even knew about Hays?

Personality, understanding, and anxiety as the driver of cultural evolution [Tech Evol]

I'm bumping this to the top of the queue, 1) on general principle, 2) in recognition of the strife that's broken out recently, cued by tragic events in the Middle East, and 3) out of curiosity about what Robert Wright calls "cognitive empathy," the ability to take another's point of view for the purpose of understanding their behavior. What does the theory of cultural ranks have to contribute to our understanding of cognitive empathy?

 

This is another excerpt from David Hays, The Evolution of Technology Through Four Cognitive Ranks (1995). This is from Chapter 5, “Politics, Cognition, and Personality”, section 5.4, “Personality and Understanding”. You might well wonder, as I did, what these topics have to do with technology. Hays’s reply, more or less: Why not? That is, he was taking a comprehensive view of the history of technology, but why not, in this chapter, use this comprehensive view of technology as an opportunity to look around the corner at other matters of interest, like governance and cognition more generally.

So that’s what he did.

In this section he argues that anxiety is what drives cultural evolution. It’s a powerful suggestion. I also believe it, but it’s not clear to me what we need to do to make a strong argument. I devoted a fair amount of energy to characterizing anxiety at the neural level in Beethoven’s Anvil (pp. 86 ff.) Note that I have a fair number of posts on the topic of anxiety.

* * * * *

Life is hard. Life is hard because it is lived in brains that strive always to understand, and often fail to understand, both the world and themselves. Our brains are the most complex devices known. The workings of cognition, emotion, and volition are scarcely understood at all even now. We do not know enough to guide parents and teachers, eliminate crime and the use of drugs, or achieve universal and lasting peace.

The psychobiological consequence of failure to understand is often fear, anxiety, anger, hostility, and hatred. These states are unpleasant and disruptive. They disrupt thought, and they disrupt society.

The fundamental significance of cultural evolution, in my opinion, is that the human capacity for understanding has grown. The striving was there from the beginning; gradually, over some 50,000 years, the striving has come to be satisfied more fully:

"Kroeber, who is by no means an evolutionist, suggests three criteria for measuring progress: "the atrophy of magic based on psychopathology; the decline of infantile obsession with the outstanding physiological events of human life; and the persistent tendency of technology and science to grow accumulatively (Kroeber, 1948: 304)." (Quoted from Julian Stewart, Theory of Culture Change, p. 14)

In rank 1, psychotic visions are treated with great respect. Too much attention is given to "blood and death and decay". These are indications that the understanding is unable to cope with the emotions of ordinary life.

Alfred L. Kroeber, who may be the greatest American anthropologist of the century, published a book called _Anthropology_ from which his three points are quoted.

The same Peace Corps volunteer I quoted above describes an event that illustrates the manifestation of anxiety in a culture not far up the growth curve from rank 1 to rank 2:

I awoke at one o'clock in the morning to horrible wailing and moaning. All the Tuaregs I live with were up, too. Umuna, one of the Bouzoo women, was making all this noise, writhing around on the ground beside one of the Tuareg tents. She claimed that she was 'invaded' by a bad spirit sent to her by her great-grandmother.

None of the others were concerned. They told me it was an odd time for spirits to be possessing people, in the middle of the Ramadan, a holy month. I was assured that she would be all right. I was glad to have so many experts on possession; had it been left to me, I would have thought she was seriously ill, perhaps dying. After fifteen minutes of carrying on, Umuna was fine. Interesting how the old animist beliefs coexist with Islam. (_Washington Spectator_, Jan 1, 1990, p. 1)

The Bouzoos, he explains, were "slaves of the Tuareg until very recently," having been taken in raids southward. The Bouzoos still live about as before. This Peace Corps volunteer also saw sex magic for sale, and healing of a headache by laying on of hands and chanting verses from the Koran.

In reading to prepare to write this book, I have learned that the wheel was used for ritual over many years before it was put to use in war and, still later, work. The motivation for improvement of astronomical instruments in the late Middle Ages was to obtain measurements accurate enough for _astrology_. Critics wrote that even if the dubious doctrines of astrology were valid, the measurements were not close enough for their predictions to be meaningful. So they set out to make their instruments better, and all kinds of instrumentation followed from this beginning. (That from White, MRTe*). Metals were used for ornaments very early – before any practical use?

In its original manifestation the compass was a divi- nation, or future-predicting, instrument made of lode- stone, which is naturally magnetic." (George Basalla, p. 172; in BIBLNOTE*)

I suspect that we could get many further examples, up into the growth curve from rank 2 to rank 3.

In fact, someone in the future may look back on psychoanalysis and remark that its origin was in parapsychology – dreams were interpreted first for divination, second for diagnosis of pathology.

Here is my first point: The driving force behind progress in social organization, government, technology, science, and art is the need to control anxiety, to satisfy the brain's striving for understanding.

To take a political example: In the origin of government, is the key problem why men choose to follow leaders, or how men succeed in making themselves into leaders? [Not a sexist formulation; just the way things happened.] For most of my life, I took for granted the first answer. Recently I recognized the second problem and adopted it. Ethnographies (culture reports) from hunting-and-gathering societies show absolute egalitarianism. But more: They show an absolute unwillingness to rise above one's fellows in any respect whatsoever.

Here and there we find a clue as to the kind of child-rearing practices (sometimes brutal) that produce such adults.

Since the earliest community leaders were war leaders, who gradually came to exercise some authority between wars, perhaps the answer to the key question is this: The first leaders draw their ability to accept the responsibility of leadership from success in war, from religious experience, and from innate genius or special accidents of handling in early childhood.

And my second point: At each rank, and excepting the few persons of extraordinary talent, a person's capacity for understanding is limited to analytic systems of quite specific logical structure. When rank shifts upward, a new system appears that transcends the old. The new system can rationalize situations, both internal and external, that were beyond the logical capacity of the old system. Understanding fails less often, and the negative states of fear, anxiety, hostility, etc. arise less often. As Kroeber put it, there is less concern "with the gratification of the ego" (KrAn* p. 301). The person of higher rank has a stronger character.

This means, among other things, that a higher rank can solve more kinds of social problems without resorting to (a) withdrawal, (b) fighting, or (c) submission.

But technology, art, science, and social-political-economic organization feed on each other. A favorable change in one makes possible changes – sometimes favorable – in all the rest.

Education to a sophisticated level of intellectual capability

In a discussion over at LessWrong I was asked:

I'd love to hear your thoughts on how to compress the training that one gets beginning and throughout a phd about how to learn effectively from ongoing research. Many folks on here either don't have time or don't think we have time to go to school, so it would be nice to get resources together about how to learn it quickly.

Here's my reply:

That's a tough one, in part because the fields vary so much. I was in an English department, so that's what my degree is in. But my real training came as part of a research group in computational linguistics that was in the linguistics department. I didn't actually do any programming. I worked on knowledge representation, a big part of old-school computational linguistics (before NLP).

But there are two aspects to this. One is getting the level of intellectual maturity and sophistication you need to function as a disciplined independent thinker. The other is knowing the literature. In some ways they interact and support one another but in some ways they are orthogonal.

I learned the most when I found a mentor, the late David G. Hays. I wanted to learn his approach to semantics. He tutored me for an hour or two once a week for a semester. That's the best. It's also relatively rare to get that kind of individual attention. Still, finding a mentor is the best possible thing you could do.

At the same time I had a job preparing abstracts for The American Journal of Computational Linguistics. That meant I had to read a wide variety of material and prepare abstracts four times a year. You need to learn to extract the gist of an article without reading the whole thing. Look at the introduction and conclusion. Does that tell you what you need? Scan the rest. You should be able to do that – scan the article and write the abstract – in no more than an hour or two.

Note, that many/most abstracts that come with an article are not very good. The idea is, if you trust the journal and the author and don't need the details, the abstract should tell you all you need. Working up that skill is good discipline.

If you're working on a project with others here, each of you agree to produce 3, 4, 5 abstracts a week to contribute to the project. Post them to a place where you can all get at them. It becomes your project library.

As for the level of intellectual maturity, the only way to acquire that is to pick a problem, work on it, and come up with a coherent written account of what you've done. The account should be intelligible to others. I don't know whether a formal dissertation is required, but you need to tackle a problem that is both interesting to you and has “weight.”

On Revising Prospero Yet Again [and again]

I thought I'd bump this to the top more or less on general principle. I've done another version of Prospero since the version posted below. This one dates from 2018 and incorporates ideas based computational techniques that didn't exist in 2014, of that just barely existed. It's called Virtual Reading: The Prospero Project Redux, where the idea of virtual reading replaces the type of computational reading I'd imagined in the old Prospero.

* * * * *

 

By Prospero I mean a thought experiment that David Hays and I proposed back in 1976 in a review article, Computational Linguistics and the Humanist, we published in Computers and the Humanities. Sometime in the last decade or so, when I began thinking about doing a book on naturalist criticism, I took that old idea, of a computer program capable of a non-trivial simulation of a reader of Shakespeare, and elaborated it more or less as a stand-alone piece. In particular, I contextualized it with some remarks Stanley Fish had made about stylistic analysis in Is There as Text in This Class? I posted that piece about half a year ago, then revised and reposted it about a month ago.

I’ve now done yet another revision, this time incorporating the notion of a tabula rasa interpretation from Alan Liu’s article, “The Meaning of the Digital Humanities” (PMLA 128, 2013, 409-423). I’ve posted that version at my Academia.edu page. I’ve appended the abstract to this post.

Why yet another revision?

It would seem that this notion, this Prospero, has become a touchstone, something through which I gauge the state of my thinking on certain possibilities of literary analysis. But that’s not quite what it was when Hays and I advanced it almost 40 decades ago. Then it was a way of conveying something to an audience we presumed to be unfamiliar with current ideas in computational linguistics.

At that time, the mid-1970s, semantics was the Big New Thing. Computational linguistics was born in the early 1950s under the rubric of machine translation. The US Federal Government needed to translate a lot of Russian documents into English. Perhaps that could be done with computers?

And so a variety of investigators went to work recasting phonology, morphology, and syntax into computational form appropriate to machine translation. By the late 1960s it became apparent, both in computational linguistics and artificial intelligence, that it would be necessary to tackle meaning. Thus was born the computational semantics of natural language.

 

THAT’s what Hays was interested in when I began working with him in the spring of 1974. That’s what everyone was interested in. By the time we wrote that essay I’d completed and published preliminary work on Shakespeare’s sonnet “The Expense of Spirit”, which we discussed in the article. But we couldn’t go into any of the details in that article. So, to give our readers some sense of what that work portended we concocted Prospero.

The idea was straightforward: Code the Elizabethan worldview into a computer using formalisms then being developed, have a computer “read” a Shakespeare play, and then examine what it did in the process. For bonus points you could also program the computer with the knowledge needed to crank out Freudian, Marxist, feminist, and other interpretations. Who wouldn’t be excited at the prospect of working on such a project, even if it were a long-term (decades) project?

I don’t know what Hays thought about the real possibility of such a thing – he’d already lived through the institutional collapse of machine translation when it failed to deliver on some rather extravagant promises – but I figured that I’d be working on Prospero in my lifetime. Certainly not in the near-term future, but 20, 30 years out...?

It didn’t happen. Nor is there any immediate prospect of such a thing. IBM’s Watson is the state of the art, and it’s nowhere near the capability needed to implement Prospero.

What would it take to implement Prospero? I don’t know.

Oh, sure, it’s easy enough to say that it would require a good model of how the human brain operates, including conversation with others. But what would THAT require? We don’t know.

By way of comparison, the folks who want to send a manned mission to Mars know a great deal about what that would require. After all, we’ve already sent humans to the moon and brought them back. And we’ve sent probes that have landed on Mars and beamed back information about what’s there. All that’s directly relevant to the task of a manned mission to Mars. It may not be sufficient – it doesn’t tell us how the human body will adapt to months of weightlessness in transit, nor the mind to those months being bound to a very small group – but it IS a lot.

It is much more than we know about simulating the human brain or coding up the Elizabethan worldview.

So, if Prospero is not possible, then why think about it at all?

Let me put that more personally. What can you learn from me by reading about Prospero? To some extent that depends on what you already know about things such as cognitive science, AI, and computational linguistics, and how much you are willing to trust my knowledge of such things. And what could I learn from you through conversing about Prospero? And that depends on what you already know and on my willingness to trust in your knowledge.

That is to say, Prospero is a set of ideas for organizing a conversation, a conversation about what computers bring to the study of literature.

For example, in one of the essays in Is There a Text in This Class? (1980) Stanley Fish asserts that Michael Halliday, a linguist, is one of many lured on by “the promise of an automatic interpretive procedure” (p. 78). Though I can’t be sure of this, I rather doubt that Halliday himself had any such idea, certainly not as Fish attributes it to him. The idea’s a straw man. Thinking about Prospero is a way of thinking about why the idea IS a straw man. It may even get you to the edge of thinking about why Stanley Fish posits such a thing.

I suppose what Fish had in mind was that you “feed” your text into “the automatic interpretive procedure” and it “spits out an interpretation.” Given such a device, why should you trust the interpretation? Wouldn’t you have to know what it does? And if you know that, do you need the device?

Prospero is useful for thinking about that. So, we’ve got Prospero and we feed it, say, Much Ado About Nothing. How do we know that Prospero understood the play in some meaningful way? Of course we could ask: Did you understand it? But what would we know once Prospero answers Yes? Not much. And if Prospero were to tell us that, no, he didn’t understanding the play, would THAT tell us anything useful? We could ask Prospero questions about the play: Who is Hero? What’s her relationship to Beatrice? And when Prospero answers correctly (or not) then what do we know?

As I say in my various versions of this little thought experiment, what’s important is that knowledge that goes into building Prospero. But Prospero could give reasonable answers to those questions without understanding much about the play, no? After all, aren’t there students like that?

I’ve also said that, given that Prospero is a reasonable simulation (as determined by some as yet unspecified set of procedures), what we really want to do is examine what Prospero does internally in the process of reading a play. That, surely, would tell us a lot.

And now we’ve got a problem, one I’ve been aware of but chose not to bring up. Assume that Prospero IS a fairly robust simulation of a human mind. Isn’t there an ethical problem in opening it/her/him up and examining what happens while reading? Don’t we have to ask permission and, if permission is not granted, go no farther?

If I was aware of this issue, why didn’t I bring it up? I don’t quite know. The matter seems both obvious and beside the point. It exists in some other thought-world, one that impinges on the Prospero world, but that’s outside the actual business of creating a Prospero machine.

Perhaps that’s what Prospero is, a boundary marker, or a guardian spirit.

❖ ❖ ❖

The Prospero Project: A Thought Experiment

Revision 3, May 2014

Abstract: Prospero is a thought experiment, a computer program powerful enough to simulate, in an interesting way, the reading of a literary text. To do that it must simulate a reader. Which reader? Prospero would also simulate literary criticism, and controversies among critics. The point of Prospero, if we could build it, is the knowledge required to build it. If we had it, we could examine its activities as it reads and comments on texts. But our knowledge of Prospero is of a different kind and order from our knowledge of the world and of life, though those things are central to literary texts. The point of this thought experiment is to clarify that difference, for that is what we will have to do to build a naturalist literary criticism grounded in the neuro-, cognitive, and evolutionary psychologies. Though contemplation of this experiment we can see that, whatever computing promises literary study, it will not yield automatic interpretive procedures.

What I've been up to in the last two weeks, across the Continental Divide and on to the Pacific [how the mind works]

Here's the most recent two entries from my intellectual diary, which is mostly just short notations. The second entry is unusually long.

* * * * *

Emergent Ventures Grant
4.28.22

Applied in April to work over my attractor-net stuff, run it through the brain paper, on to “Kubla Khan” and out into a differentiation between natural and artificial minds. It was turned down on April 25, 2022. By that time I’d started thinking my way back into the old work on Attractor Nets.

3rd Version of “The Theory”
4.30.22

By which I mean the theory I’d been working on with Dave Hays. The first version was based on Mechanisms of Language. That was in place when first met Hays in Spring of 1974. Hays wrote Cognitive Structures in the Spring of 1975. That marks the second version of the theory, where Hays grounded cognition in the servomechanical model developed by William Powers (Behavior: The Control of Perception). My 1978 dissertation, “Cognitive Science and Literary Theory,” advanced that a notch, mainly with the addition of what I call The TV Tube model. Then Hays and I wrote and published “Principles and Development of Natural Intelligence” (1988).

That marks the beginning of the third phase of the theory – though we’d published on metaphor the year before. We’d completed the “brain paper” in ’85, I believe, the review process took three years. The brain paper retained by four degrees from the stage 2 theory – sensorimotor, systemic, episodic, gnomonic – and added a fifth at the bottom, modal. That was based on McCulloch’s model of the reticular activating system (RAS). We stuck Pribram’s holographic ‘model’ in at the second degree, sensorimotor, and the Powers stack at three, systemic. But we didn’t actually know how to construct cognitive models (comparable to those of stage 1 & 2) in those terms.

I began that work in 2003 when I had the idea of taking Sydney Lamb’s relational notation and using it to make logical relations between the basins of attraction in patches of cortical tissue, such as Walter Freeman found in his work. That went well for two or three months until I decided that things were beginning to seem arbitrary and unmotivated. So I stopped working. But by that time I had a pile of very interesting diagrams and some provocative prose. I created two documents, a text document (MSWord), and a diagrams document (PowerPoint), and sent those around to various people. I also came up with the idea of an open-ended natural language front-end for end-user software. I sent that around as well. Sydney Lamb thought it was a good idea. A decade later I put those three documents on the web at on my Academia page.

And that was that, until a week or so ago. That’s when I decided it was time to get back into the fray. By then I’d been thinking seriously about work in machine learning, mostly in cognitive criticism. But I’d also been thinking about NLP, especially the machine translation work. Along came GPT-3 and I got serious. I wrote a working paper, “GPT-3: Waterloo or Rubicon? Here be Dragons”, in 2020. I made real progress on that, began to get a sense that what’s going on in those engines is intelligible. The work of Peter Gärdenfors was important.

The upshot: By the time I’d submitted the proposal to Emergent Ventures I’d begun to think my way back into it. When I got turned down, I couldn’t stop. Yesterday I figured it out:

	physical substrate	data reduction	concepts
Hays	Powers stack (analog servos)	parameters of perception	cognition
Gärdenfors	subsymbolic neural net	conceptual spaces, dimensions	symbolic

What does that mean, figured it out? It means I finally made it across the continent and am viewing the Pacific Ocean. I’ve put a boundary around the territory. Most of the territory has yet to be explored, much less become settled and domesticated.

These diagrams are nice as well. This diagram relates to the work of Gärdenfors. Each rectangle is a domain in his terminology. Conceptual spaces (again, his terminology) exist in different domains.

This diagram relates to both Hays-Benzon and Gärdenfors. The rectangles are Gärdenfors. The network structure in Benzon-Hays.

Geoffrey Hinton says deep learning will do everything. I’m not sure what he means, but I offer some pointers. Version 2.

This is updated from a previous version to include a passage by Sydney Lamb.

* * * * *

Late last year Geoffrey Hinton had an interview with Karen Hao [1] in which he said “I do believe deep learning is going to be able to do everything,” with the qualification that “there’s going to have to be quite a few conceptual breakthroughs.” I’m trying to figure out whether or not, to what extent, in what way I (might) agree with him.

Neural Vectors, Symbols, Reasoning, and Understanding

Hinton believes that “What’s inside the brain is these big vectors of neural activity” and that one of the breakthroughs we need is “how you get big vectors of neural activity to implement things like reason.” That will certainly require a massive increase in scale. Thus while GPT-3 has 175 billion parameters, the brain has trillion, where Hinton treats each synapse as a parameter. 

Correspondingly Hinton rejects the idea that symbolic reasoning is primitive to the nervous system (my formulation), rather “we do internal operations on big vectors.” What about language? He doesn’t address the issue directly but he does say that “symbols just exist out there in the external world.” I do think that covers language, speech sounds, written words, gestural signs, those things are out there in the external world. But the brain uses “big vectors of neural activity” to process those. 

Hinton’s remark about symbols bears comparison with a remark by Sydney Lamb: “the linguistic system is a relational network and as such does not contain lexemes or any objects at all. Rather it is a system that can produce and receive such objects. Those objects are external to the system, not within it”[2]. Lamb has come to think of his approach as neurocognitive linguistics and, while his sense of the nervous system is somewhat different from Hinton’s, they agree on this issue and, in the current intellectual climate, that agreement is of some significance. For Lamb is a first generation researcher in machine translation  and so was working when most AI research was committed to symbolic systems. We’ll return to Lamb later as I think the notation he developed is a way to bring symbolic reasoning within range of Hinton’s “big vectors of neural activity”.

But now let’s return to Hinton with a passage from an article he co-authored with Yann LeCun and Yoshua Bengio [3]:

In the logic-inspired paradigm, an instance of a symbol is something for which the only property is that it is either identical or non-identical to other symbol instances. It has no internal structure that is relevant to its use; and to reason with symbols, they must be bound to the variables in judiciously chosen rules of inference. By contrast, neural networks just use big activity vectors, big weight matrices and scalar non-linearities to perform the type of fast ‘intuitive’ inference that underpins effortless commonsense reasoning.

I note, moreover, commonsense reasoning seems to be problematic for everyone.[4]

Let’s look at one more passage from the interview:

For things like GPT-3, which generates this wonderful text, it’s clear it must understand a lot to generate that text, but it’s not quite clear how much it understands.

I’m not sure that it is at all useful to say that GPT-3 understands anything. I think that, in using that term, Hinton is displaying what I’ve come to think of as the word illusion.[5] Briefly, GPT-3’s language model is constructed over a corpus consisting entirely of word forms, of signifiers without signifieds, to use an old terminology. Hinton knows that, of course, but, after all, he understands texts from seeing or hearing word forms alone, as do we all, and so, in effect, credits GPT-3 with somehow having induced meaning from a statistical distribution. The text it generates looks pretty good, no? Yes. And that is something we do need to understand, just what is GPT-3 doing and how does it do it? But this is not the place to enter into that.[6]

I think that GPT-3’s remarkable performance based on such ‘shallow’ material should prompt us into reconsidering just what humans are doing when we produce everyday ‘boilerplate’ text. Consider this passage from LeCun, Bengio, and Hinton, where they are referring to the use of an RNN:

This rather naive way of performing machine translation has quickly become competitive with the state-of-the-art, and this raises serious doubts about whether understanding a sentence requires anything like the internal symbolic expressions that are manipulated by using inference rules. It is more compatible with the view that everyday reasoning involves many simultaneous analogies that each contribute plausibility to a conclusion
.

In dealing with these utterly remarkable devices, we would be rein in our narcissistic investment in the routine use of our ‘higher’ cognitive and linguistic capacities as opposed to our mere sensory-motor competence. It’s all neural vectors. 

Note, however, that it is one thing to say that “we do internal operations on big vectors.” I agree with that. That’s not quite the same as saying we can do everything with deep learning. Deep learning is a collection of architectures, but I’m not sure such architectures are adequate for internalizing the vectors needed to effectively mimic human perceptual and cognitive behavior. The necessary conceptual breakthroughs will likely take us considerably beyond deep learning engines. With that qualification, let’s continue.

How the brain might be doing it

I find that, with the caveats I’ve mentioned, this is rather congenial. Which is to say that I can make sense of it in terms of issues I’ve thought through in my own work.

Some years ago David Hays and I wanted to come to terms with neuroscience and ended up reviewing a wide range of work and writing a paper entitled, “Principles and Development of Natural Intelligence.”[7] The principles are ordered such that principle N assumed N-1. We called the fifth and last principle indexing:

The indexing principle is about computational geometry, by which we mean the geometry, that is, the architecture (Pylyshyn, 1980) of computation rather than computing geometrical structures. While the other four principles can be construed as being principles of computation, only the indexing principle deals with computing in the sense it has had since the advent of the stored program digital computer. Indexed computation requires (1) an alphabet of symbols and (2) relations over places, where tokens of the alphabet exist at the various places in the system. The alphabet of symbols encodes the contents of the calculation while the relations over places, i.e. addresses, provide the means of manipulating alphabet tokens in carrying out the computation. [...] Within the context of natural intelligence, indexing is embodied in language. Linguists talk of duality of patterning (Hockett, 1960), the fact that language patterns both sounds and sense. The system which patterns sound is used to index the system which patterns sense.

In short, “indexing gives computational geometry, and language enables the system to operate on its own geometry.” This is where we get symbols and complex reasoning.

I should note that, while we talked of “an alphabet of symbols” and “relations over places” we were not asserting that that’s what was going on in the brain. That’s what’s actually going on in computers, but it applies only figuratively to the brain. The system that is using sound patterns to index patterns of sense is using one set of neural vectors (though we didn’t use that term) to index a different set of neural vectors.

How do we get deep learning to figure that out? I note that automatic image annotation is a step in that direction [8], but have nothing to say about that here.

Instead I want to mention some informal work I did some years ago on something I call attractor nets.[9] The general idea was to use Sydney Lamb’s relational networks, in which nodes are logical operators, as a tertium quid between the symbol-based semantic networks Hays and I had worked on in the 1970s and the attractor landscapes of Walter Freeman’s neurodynamics. I showed – informally, using diagrams – how using logical operators (AND, OR) over attractor basins in different neurofunctional areas could reconstruct symbolic systems represented as directed graphs. Each node in a symbolic graph corresponds to a basin of attraction, that is, an attractor. In the present context we can think of each neurofunctional area as corresponding to a collection of neural vectors and the attractors as objects represented by those vectors. An attractor net would then become a way of thinking about how complex reasoning could be accomplished with neural vectors.

In the attractor net notation word forms, or signifiers, are distinct from word meanings, of signifieds. Is that distinction important for complex reasoning? I believe it is, though I’m not interested in constructing an argument at this point. That, I believe, puts a limit on what one can expect of engines like GPT-3. That too requires an argument.

So, what about natural vs. artificial intelligence?

The notion of intelligence is somewhat problematic. As a practical matter I believe that a formulation by Robin Hanson is adequate: “’Intelligence’ just means an ability to do mental/calculation tasks, averaged over many tasks.”[10] As for the difference between artificial and natural, that comes down to four things:

1) a living system vs. an inanimate system,
2) a carbon-based organic electro-chemical substrate vs. a silicon-based electronic substrate,
3) real neurons (having on average 10K connections with others) vs. considerably simpler artificial neurons realized in program code, and
4) the neurofunctional architecture and innate capacities of a real brain vs. the system architecture of a digital computing system.

Make no mistake, those differences are considerable. But I think we now have in hand a body of concepts and models that is rich enough to support ever more sophisticated interaction between students of neuroscience and students of artificial intelligence. To the extent that our research and teaching institutions can support that interaction I expect to see progress accelerate in the future. I offer no predictions about what will come of this interaction.

Some related posts

William Benzon, Showdown at the AI Corral, or: What kinds of mental structures are constructible by current ML/neural-net methods? [& Miriam Yevick 1975], New Savanna, June 3, 2020, https://new-savanna.blogspot.com/2020/06/showdown-at-ai-corral-or-what-kinds-of.html.

William Benzon, What’s AI? – Part 2, on the contrasting natures of symbolic and statistical semantics [can GPT-3 do this?], New Savanna, July 17, 2020, https://new-savanna.blogspot.com/2019/11/whats-ai-part-2-on-contrasting-natures.html.

William Benzon, A quick note on the ‘neural code’ [AI meets neuroscience], New Savanna, April 20, 2021, https://new-savanna.blogspot.com/2021/04/a-quick-note-on-neural-code-ai-meets.html.

References

[1] Interview with Karen Hao, AI pioneer Geoff Hinton: “Deep learning is going to be able to do everything”, MIT Technology Review, Nov. 3, 2020. https://www.technologyreview.com/2020/11/03/1011616/ai-godfather-geoffrey-hinton-deep-learning-will-do-everything/. 

[2] Sydney Lamb, Linguistic Structure: A Plausible Theory, Language Under Discussion, 4(1) 2016, 1–37, https://doi.org/10.31885/lud.4.1.229.

[3] From Yann LeCun, Yoshua Bengio, and Geoffrey Hinton, Deep Learning, Nature, 521 28 May 2015, 436-444, https://doi.org/10.1038/nature14539.

[4] As an example I offer a recent post in which I quiz GPT-3 about a Jerry Seinfeld bit: Analyze This! Screaming on the flat part of the roller coaster ride [Does GPT-3 get the joke?], May 7, 2021, https://new-savanna.blogspot.com/2021/05/analyze-this-screaming-on-flat-part-of.html.

[5] See my post, The Word Illusion, May 12, 2021, https://new-savanna.blogspot.com/2021/05/the-word-illusion.html.

[6] For some extended remarks, see my working paper, GPT-3: Waterloo or Rubicon? Here be Dragons, Working Paper, Version 3, August 20, 2020, 34 pp., https://www.academia.edu/43787279/GPT_3_Waterloo_or_Rubicon_Here_be_Dragons_Version_3.

[7] William Benzon and David Hays, Principles and Development of Natural Intelligence, Journal of Social and Biological Structures, Vol. 11, No. 8, July 1988, 293-322, https://www.academia.edu/235116/Principles_and_Development_of_Natural_Intelligence.

[8] Wikipedia, Automatic image annotation, https://en.wikipedia.org/wiki/Automatic_image_annotation.

[9] William Benzon, Attractor Nets, Series I: Notes Toward a New Theory of Mind, Logic, and Dynamics in Relational Networks, Working Paper, 52 pp., https://www.academia.edu/9012847/Attractor_Nets_Series_I_Notes_Toward_a_New_Theory_of_Mind_Logic_and_Dynamics_in_Relational_Networks.

William Benzon, Attractor Nets 2011: Diagrams for a New Theory of Mind, Working Paper, 55 pp., https://www.academia.edu/9012810/Attractor_Nets_2011_Diagrams_for_a_New_Theory_of_Mind.

William Benzon, From Associative Nets to the Fluid Mind, Working Paper. October 2013, 16 pp. https://www.academia.edu/9508938/From_Associative_Nets_to_the_Fluid_Mind.

[10] Robin Hanson, I Still Don’t Get Foom, Overcoming Bias, July 24, 2014, https://www.overcomingbias.com/2014/07/30855.html.