Geoffrey Hinton says deep learning will do everything. I’m not sure what he means, but I offer some pointers. Version 2.

This is updated from a previous version to include a passage by Sydney Lamb.

* * * * *

Late last year Geoffrey Hinton had an interview with Karen Hao [1] in which he said “I do believe deep learning is going to be able to do everything,” with the qualification that “there’s going to have to be quite a few conceptual breakthroughs.” I’m trying to figure out whether or not, to what extent, in what way I (might) agree with him.

Neural Vectors, Symbols, Reasoning, and Understanding

Hinton believes that “What’s inside the brain is these big vectors of neural activity” and that one of the breakthroughs we need is “how you get big vectors of neural activity to implement things like reason.” That will certainly require a massive increase in scale. Thus while GPT-3 has 175 billion parameters, the brain has trillion, where Hinton treats each synapse as a parameter. 

Correspondingly Hinton rejects the idea that symbolic reasoning is primitive to the nervous system (my formulation), rather “we do internal operations on big vectors.” What about language? He doesn’t address the issue directly but he does say that “symbols just exist out there in the external world.” I do think that covers language, speech sounds, written words, gestural signs, those things are out there in the external world. But the brain uses “big vectors of neural activity” to process those. 

Hinton’s remark about symbols bears comparison with a remark by Sydney Lamb: “the linguistic system is a relational network and as such does not contain lexemes or any objects at all. Rather it is a system that can produce and receive such objects. Those objects are external to the system, not within it”[2]. Lamb has come to think of his approach as neurocognitive linguistics and, while his sense of the nervous system is somewhat different from Hinton’s, they agree on this issue and, in the current intellectual climate, that agreement is of some significance. For Lamb is a first generation researcher in machine translation  and so was working when most AI research was committed to symbolic systems. We’ll return to Lamb later as I think the notation he developed is a way to bring symbolic reasoning within range of Hinton’s “big vectors of neural activity”.

But now let’s return to Hinton with a passage from an article he co-authored with Yann LeCun and Yoshua Bengio [3]:

In the logic-inspired paradigm, an instance of a symbol is something for which the only property is that it is either identical or non-identical to other symbol instances. It has no internal structure that is relevant to its use; and to reason with symbols, they must be bound to the variables in judiciously chosen rules of inference. By contrast, neural networks just use big activity vectors, big weight matrices and scalar non-linearities to perform the type of fast ‘intuitive’ inference that underpins effortless commonsense reasoning.

I note, moreover, commonsense reasoning seems to be problematic for everyone.[4]

Let’s look at one more passage from the interview:

For things like GPT-3, which generates this wonderful text, it’s clear it must understand a lot to generate that text, but it’s not quite clear how much it understands.

I’m not sure that it is at all useful to say that GPT-3 understands anything. I think that, in using that term, Hinton is displaying what I’ve come to think of as the word illusion.[5] Briefly, GPT-3’s language model is constructed over a corpus consisting entirely of word forms, of signifiers without signifieds, to use an old terminology. Hinton knows that, of course, but, after all, he understands texts from seeing or hearing word forms alone, as do we all, and so, in effect, credits GPT-3 with somehow having induced meaning from a statistical distribution. The text it generates looks pretty good, no? Yes. And that is something we do need to understand, just what is GPT-3 doing and how does it do it? But this is not the place to enter into that.[6]

I think that GPT-3’s remarkable performance based on such ‘shallow’ material should prompt us into reconsidering just what humans are doing when we produce everyday ‘boilerplate’ text. Consider this passage from LeCun, Bengio, and Hinton, where they are referring to the use of an RNN:

This rather naive way of performing machine translation has quickly become competitive with the state-of-the-art, and this raises serious doubts about whether understanding a sentence requires anything like the internal symbolic expressions that are manipulated by using inference rules. It is more compatible with the view that everyday reasoning involves many simultaneous analogies that each contribute plausibility to a conclusion
.

In dealing with these utterly remarkable devices, we would be rein in our narcissistic investment in the routine use of our ‘higher’ cognitive and linguistic capacities as opposed to our mere sensory-motor competence. It’s all neural vectors. 

Note, however, that it is one thing to say that “we do internal operations on big vectors.” I agree with that. That’s not quite the same as saying we can do everything with deep learning. Deep learning is a collection of architectures, but I’m not sure such architectures are adequate for internalizing the vectors needed to effectively mimic human perceptual and cognitive behavior. The necessary conceptual breakthroughs will likely take us considerably beyond deep learning engines. With that qualification, let’s continue.

How the brain might be doing it

I find that, with the caveats I’ve mentioned, this is rather congenial. Which is to say that I can make sense of it in terms of issues I’ve thought through in my own work.

Some years ago David Hays and I wanted to come to terms with neuroscience and ended up reviewing a wide range of work and writing a paper entitled, “Principles and Development of Natural Intelligence.”[7] The principles are ordered such that principle N assumed N-1. We called the fifth and last principle indexing:

The indexing principle is about computational geometry, by which we mean the geometry, that is, the architecture (Pylyshyn, 1980) of computation rather than computing geometrical structures. While the other four principles can be construed as being principles of computation, only the indexing principle deals with computing in the sense it has had since the advent of the stored program digital computer. Indexed computation requires (1) an alphabet of symbols and (2) relations over places, where tokens of the alphabet exist at the various places in the system. The alphabet of symbols encodes the contents of the calculation while the relations over places, i.e. addresses, provide the means of manipulating alphabet tokens in carrying out the computation. [...] Within the context of natural intelligence, indexing is embodied in language. Linguists talk of duality of patterning (Hockett, 1960), the fact that language patterns both sounds and sense. The system which patterns sound is used to index the system which patterns sense.

In short, “indexing gives computational geometry, and language enables the system to operate on its own geometry.” This is where we get symbols and complex reasoning.

I should note that, while we talked of “an alphabet of symbols” and “relations over places” we were not asserting that that’s what was going on in the brain. That’s what’s actually going on in computers, but it applies only figuratively to the brain. The system that is using sound patterns to index patterns of sense is using one set of neural vectors (though we didn’t use that term) to index a different set of neural vectors.

How do we get deep learning to figure that out? I note that automatic image annotation is a step in that direction [8], but have nothing to say about that here.

Instead I want to mention some informal work I did some years ago on something I call attractor nets.[9] The general idea was to use Sydney Lamb’s relational networks, in which nodes are logical operators, as a tertium quid between the symbol-based semantic networks Hays and I had worked on in the 1970s and the attractor landscapes of Walter Freeman’s neurodynamics. I showed – informally, using diagrams – how using logical operators (AND, OR) over attractor basins in different neurofunctional areas could reconstruct symbolic systems represented as directed graphs. Each node in a symbolic graph corresponds to a basin of attraction, that is, an attractor. In the present context we can think of each neurofunctional area as corresponding to a collection of neural vectors and the attractors as objects represented by those vectors. An attractor net would then become a way of thinking about how complex reasoning could be accomplished with neural vectors.

In the attractor net notation word forms, or signifiers, are distinct from word meanings, of signifieds. Is that distinction important for complex reasoning? I believe it is, though I’m not interested in constructing an argument at this point. That, I believe, puts a limit on what one can expect of engines like GPT-3. That too requires an argument.

So, what about natural vs. artificial intelligence?

The notion of intelligence is somewhat problematic. As a practical matter I believe that a formulation by Robin Hanson is adequate: “’Intelligence’ just means an ability to do mental/calculation tasks, averaged over many tasks.”[10] As for the difference between artificial and natural, that comes down to four things:

1) a living system vs. an inanimate system,
2) a carbon-based organic electro-chemical substrate vs. a silicon-based electronic substrate,
3) real neurons (having on average 10K connections with others) vs. considerably simpler artificial neurons realized in program code, and
4) the neurofunctional architecture and innate capacities of a real brain vs. the system architecture of a digital computing system.

Make no mistake, those differences are considerable. But I think we now have in hand a body of concepts and models that is rich enough to support ever more sophisticated interaction between students of neuroscience and students of artificial intelligence. To the extent that our research and teaching institutions can support that interaction I expect to see progress accelerate in the future. I offer no predictions about what will come of this interaction.

Some related posts

William Benzon, Showdown at the AI Corral, or: What kinds of mental structures are constructible by current ML/neural-net methods? [& Miriam Yevick 1975], New Savanna, June 3, 2020, https://new-savanna.blogspot.com/2020/06/showdown-at-ai-corral-or-what-kinds-of.html.

William Benzon, What’s AI? – Part 2, on the contrasting natures of symbolic and statistical semantics [can GPT-3 do this?], New Savanna, July 17, 2020, https://new-savanna.blogspot.com/2019/11/whats-ai-part-2-on-contrasting-natures.html.

William Benzon, A quick note on the ‘neural code’ [AI meets neuroscience], New Savanna, April 20, 2021, https://new-savanna.blogspot.com/2021/04/a-quick-note-on-neural-code-ai-meets.html.

References

[1] Interview with Karen Hao, AI pioneer Geoff Hinton: “Deep learning is going to be able to do everything”, MIT Technology Review, Nov. 3, 2020. https://www.technologyreview.com/2020/11/03/1011616/ai-godfather-geoffrey-hinton-deep-learning-will-do-everything/. 

[2] Sydney Lamb, Linguistic Structure: A Plausible Theory, Language Under Discussion, 4(1) 2016, 1–37, https://doi.org/10.31885/lud.4.1.229.

[3] From Yann LeCun, Yoshua Bengio, and Geoffrey Hinton, Deep Learning, Nature, 521 28 May 2015, 436-444, https://doi.org/10.1038/nature14539.

[4] As an example I offer a recent post in which I quiz GPT-3 about a Jerry Seinfeld bit: Analyze This! Screaming on the flat part of the roller coaster ride [Does GPT-3 get the joke?], May 7, 2021, https://new-savanna.blogspot.com/2021/05/analyze-this-screaming-on-flat-part-of.html.

[5] See my post, The Word Illusion, May 12, 2021, https://new-savanna.blogspot.com/2021/05/the-word-illusion.html.

[6] For some extended remarks, see my working paper, GPT-3: Waterloo or Rubicon? Here be Dragons, Working Paper, Version 3, August 20, 2020, 34 pp., https://www.academia.edu/43787279/GPT_3_Waterloo_or_Rubicon_Here_be_Dragons_Version_3.

[7] William Benzon and David Hays, Principles and Development of Natural Intelligence, Journal of Social and Biological Structures, Vol. 11, No. 8, July 1988, 293-322, https://www.academia.edu/235116/Principles_and_Development_of_Natural_Intelligence.

[8] Wikipedia, Automatic image annotation, https://en.wikipedia.org/wiki/Automatic_image_annotation.

[9] William Benzon, Attractor Nets, Series I: Notes Toward a New Theory of Mind, Logic, and Dynamics in Relational Networks, Working Paper, 52 pp., https://www.academia.edu/9012847/Attractor_Nets_Series_I_Notes_Toward_a_New_Theory_of_Mind_Logic_and_Dynamics_in_Relational_Networks.

William Benzon, Attractor Nets 2011: Diagrams for a New Theory of Mind, Working Paper, 55 pp., https://www.academia.edu/9012810/Attractor_Nets_2011_Diagrams_for_a_New_Theory_of_Mind.

William Benzon, From Associative Nets to the Fluid Mind, Working Paper. October 2013, 16 pp. https://www.academia.edu/9508938/From_Associative_Nets_to_the_Fluid_Mind.

[10] Robin Hanson, I Still Don’t Get Foom, Overcoming Bias, July 24, 2014, https://www.overcomingbias.com/2014/07/30855.html.

Silly Talk About the Brain & Pleasure Centers

I'm bumping this to the top of the queue because Lamb's story about his daughter assumes new relevance in the context of fantasies about direct brain-to-brain thought transfer.

Sydney Lamb begins Pathways of the Brain with a story about his daughter (p. 1):

Some years ago I asked one of my daughters, as she sat at the piano, "When you hit that piano key with your finger, how does your mind tell your finger what to do?" She thought for a moment, her face brightening with the intellectual challenge, and said, "Well, my brain writes a little note and sends it down my arm to my hand, then my hand reads the note and knows what to do." Not too bad for a five-year old.

Lamb goes on to suggest that an awful lot of professional thinking about the brain takes place in such terms (p. 2):

This mode of theorizing is seen in ... statements about such things as lexical semantic retrieval, and in descriptions of mental processes like that of naming what is in a picture, to the effect that the visual information is transmitted from the visual area to a language area where it gets transformed into a phonological representation so that a spoken description of the picture may be produced....It is the theory of the five-year-old expressed in only slightly more sophisticated terms. This mode of talking about operations in the brain is obscuring just those operations we are most intent in understanding, the fundamental processes of the mind.

I agree with Lamb whole-heartedly. My impression is that most of the neuroscientific effort goes into getting observations – using some really cool technology, too – but when it comes to thinking about what's going on, the inner child just takes over – and a rather dim one at that.

Some talk of pleasure centers, for example, strikes me as being equally dim. This kind of talk asks you to imagine that there is some center in the brain which, when stimulated, gives you pleasure. Call it the P-Center. Such talk is grounded in a misinterpretation of some interesting experiments James Olds did back in the 1950s – which I discuss in Beethoven’s Anvil (pp. 82-85).

Let’s exaggerate and be explicitly and extravagently silly. Imagine that the P-Center is in the middle of your left palm. In order to experience pleasure it must be stimulated by pleasure juice (P-Juice), which is made in the salivary glands. So, to experience pleasure you spit into your left palm and – ZAP! – you get jolt of pleasure.

Silly, no? – I mean, there’s no chance we’re being serious, right? So let’s go on.

You might wonder why Mother Nature would create such a scheme, which could threaten survival. How? Well, if you can get pleasre so easily, you’d probably spend all your time spitting P-Juice into your palm and neglect eating and drinking and sleeping, and so forth. After all, no matter what happens, you just spit in your palm and Violà! pleasure is yours – a problem Bill Powers pointed out in Behavior: The Control of Perception.

So, let’s imagine that Mother Nature, in her wisdom, made the production of pleasure juice contingent upon some concrete achievement. That is, you have to do something that has survival value in order to produce P-Juice. Imagine, for example, that you chase down a rabbit. The brain’s Evaluator says “Ah! a warm meal” and tells the P-Juice gland to produce some P-Juice. The P-Juicer does so and tells the Spitter “Yo yo yo, spit in the palm, NOW!” The Spitter does it and the P-Center tells the Evalulator, “We got pleasure here.”

But this whole thing started with the Evaluator. So why have all this other mess about P-Juice and a P-Center? Why not have the Evaluator produce pleasure directly? Why indeed. But the underlying problems of such talk doesn’t stop people from believing in pleasure centers.

But what about pleasure itself, you ask, what IS it, for real?

Good question. Later. Or just read Chapter 4 of Beethoven’s Anvil.

Once more into the Chinese Room

I’ve been crashing on my GPT-3 working paper and I had a new thought about Searle’s infamous Chinese room [1].

Yet if you would believe John Searle, no matter how rich and detailed the world model included in an AI, understanding would necessarily elude them. When I first encountered the Chinese room argument years ago my reaction was something like: interesting, but irrelevant. Why irrelevant? Because it said absolutely nothing about the techniques AI or cognitive science investigators used and so would provide no guidance toward improving that work. He did, however, have a point: If the machine has no contact with the world, how can it possibly be said to understand anything at all? All it does is grind away on syntax.

What Searle misses, though, is the way in which meaning is a function of relations among concepts, as I pointed out earlier (see [2]). It seems to me, however – and here I’m just making this up off the top of my head – we can think of meaning as having both a intentional aspect, the connection of signs to the world, and a relational aspect, the relations of signs among themselves. Searle’s argument concentrated on the former and said nothing about the latter.

What of the intentional aspect when a person is writing or talking about things not immediately present, which is, after all quite common? In this case the intentional aspect of meaning is not supported by the immediate world. Language use thus must necessarily be driven entirely by the relations of signifiers among themselves, Sydney Lamb’s point which we have already investigated [again, see it in [2]).

Have I at long last wrestled that pesky argument to the ground?

We'll see.

* * * * *

[1] I have written a number of blog posts about this argument. Here’s one of them: Another romp around Searle’s Chinese room, New Savanna, blog post, July 18, 2018, http://new-savanna.blogspot.com/2018/07/another-romp-around-searles-chinese-room.html. You can find others at the Searle link, which, however, contains other Searle posts as well, http://new-savanna.blogspot.com/search/label/Searle.

[2] New Savanna blog post, 2. The brain, the mind, and GPT-3: Dimensions and conceptual spaces, July 29, 2020, http://new-savanna.blogspot.com/2020/07/2-brain-and-gpt-3-part-1-dimensions-and.html.

2. The brain, the mind, and GPT-3: Dimensions and conceptual spaces

[Edited, with a substantial addition, August 2, 2020]

The purpose of this post is to sketch a conceptual framework in which we can understand the success of language models such as GPT-3 despite the fact that they are based on nothing more than massive collections of bare naked signifiers. There’s not a signified in sight, much less any referents. I have no intention of even attempting to explain how GPT-3 works. That it does work, in an astonishing variety of cases if (certainly) not universally, is sufficient for my purposes.

First of all I present the insight that sent me down this path, a comment by Graham Neubig in an online conversation that I was not a part of. Then I set that insight in the context of and insight by Sydney Lamb (meaning resides in relations), a first-generation researcher in machine translation and computational linguistics. I think take a grounding case by Julian Michael, that of color, and suggest that it can be extended by the work of Peter Gärdenfors on conceptual spaces.

A clue: an isomorphic transform into meaning space

At the 58th Annual Meeting of the Association for Computational Linguistics Emily M. Bender and Alexander Koller delivered a paper, Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data [1], where NLU means natural language understanding. The issue is pretty much the one I laid out in my previous posts in the sections “No words, only signifiers” and “Martin Kay, ‘an ignorance model’” [2]. A lively discussion ensured online which Julian Michael has summarized and commented on in a recent blog post [3].

In that post Michael quotes a remark by Graham Neubig:

One thing from the twitter thread that it doesn’t seem made it into the paper... is the idea of how pre-training on form might learn something like an “isomorphic transform” onto meaning space. In other words, it will make it much easier to ground form to meaning with a minimal amount of grounding. There are also concrete ways to measure this, e.g. through work by Lena Voita or Dani Yogatama... This actually seems like an important point to me, and saying “training only on form cannot surface meaning,” while true, might be a little bit too harsh— something like “training on form makes it easier to surface meaning, but at least a little bit of grounding is necessary to do so” may be a bit more fair.

That’s my point of departure in this post, that notion of “an ‘isomorphic transform’ onto meaning space.” I am going to sketch a framework in which we can begin unpacking that idea. But it may take awhile to get there.

Meaning is in relations

I want to develop an idea I have from Sydney Lamb, that meaning resides in relations. The idea is grounded in the “old school” world of symbolic computation, where language is conceived as a relational network of items. The meaning of any item in the network is a function of its position in the network.

Let’s start with this simple diagram:

It represents the fact that the central nervous system (CNS) is coupled to two worlds, each external to it. To the left we have the external world. The CNS is aware of that world through various senses (vision, hearing, smell, touch, taste, and perhaps others) and we act in that world through the motor system. But the CNS is also coupled to the internal milieu, with which it shares a physical body. The net is aware of that milieu by chemical sensors indicating contents of the blood stream and of the lungs, and by sensors in the joints and muscles. And it acts in the world through control of the endocrine system and the smooth muscles. Roughly speaking the CNS guides the organism’s actions in the external world so as to preserve the integrity of the internal milieu. When that integrity is gone, the organism is dead.

Now consider this more differentiated presentation of the same facts:

I have divided the CNS into four sections: A) senses the external world, B) senses the internal milieu, D) guides action in the internal milieu, and D) guides action in the external world. I rather doubt that even a very simple animal, such as C. elegans, with 302 neurons, is so simple. But I trust my point will survive that oversimplification.

Lamb’s point is that the “meaning” or “significance” of any of those nodes – let’s not worry at the moment whether they’re physical neurons or more abstract entities – is a function of its position in the entire network, with its inputs from and outputs to the external world and the inner milieu [4]. To appreciate the full force of Lamb’s point we need to recall the diagrams typical of old school symbolic computing, such as this diagram from Brian Phillips we used in the previous post:

All of the nodes and edges have labels. Lamb’s point is that those labels exist for our convenience, they aren’t actually a part of the system itself. If we think of that network as a fragment from a human cognitive system – and I’m pretty sure that’s how Phillips thought about it, even if he could not justify it in detail (no one could, not then, not now) – then it is ultimately connected to both the external world and the inner milieu. All those labels fall away; they serve no purpose. Alas, Phillips was not building a sophisticated robot, and so those labels are necessary fictions.

But we’re interested in the full real case, a human being making their way in the world. In that case let us assume that, for one thing, the necessary diagram is WAY more complex, and that the nodes and edges do not represent individual neurons. Rather, they represent various entities that are implemented in neurons, sensations, thoughts, perceptions, and so forth. Just how such things are realized in neural structures is, of course, a matter of some importance and is being pursued by hundreds of thousands of investigators around the world. But we need not worry about that now. We’re about to fry some rather more abstract fish (if you will).

Some of those nodes will represent signifiers, to use the Saussurian terminology I used in my previous post, and some will represent signifieds. What’s the difference between a signifier and a signified? Their position in the network as a whole. That’s all. No more, no less. Now, it seems to me, we can begin thinking about Neubig’s “isomorphic transform” onto meaning space.

Let us notice, first of all, that language exists as strings of signifiers in the external world. In the case that interests us, those are strings of written characters that have been encoded into computer-readable form. Let us assume that the signifieds – which bear a major portion of meaning, no? – exist in some high dimensional network in mental space. This is, of course, an abstract space rather than the physical space of neurons, which is necessarily three dimensional. However many dimensions this mental space has, each signified exists at some point in that space and, as such, we can specify that point by a vector containing its value along each dimension.

What happens when one writes? Well, one produces a string of signifiers. The distance between signifiers on this string, and their ordering relative to one another, are a function of the relative distances and orientations of their associated signifieds in mental space. That’s where to look for Neubig’s isometric transform into meaning space. What GPT-3, and other NLP engines, does is to examine the distances and ordering of signifiers in the string and compute over them so as to reverse engineer the distances and orientations of the associated signifieds in high-dimensional mental space.

[A little reflection on that formulation makes it clear that it fails to take into account a distinction central to ‘old school’ symbolic computation, that between semantic and episodic memory. Rather than interrupt this argument with a refined formulation I have placed that in an appendix to this post: A more refined approach to meaning space. I also offer some remarks on need for a connection to the physical world in order to handle common-sense reasoning.]

Is the result perfect? Of course not – but then how do we really know? It’s not as though we’ve got a well-accepted model of human conceptual space just lying around on a shelf somewhere. GPT-3’s language model is perhaps as good as we’ve got at the moment, and we can’t even open the hood and examine it. We know its effectiveness by examining how it performs. And it performs very well.

What’s AI? – Part 2, on the contrasting natures of symbolic and statistical semantics [can GPT-3 do this?]

In view of current excitement over GPT-3 I'm bumping this to the top of the queue. This is about the implementation of Old School semantic or cognitive nets in neural nets where the nodes and edges of the semantic net each represent activity in regions of the neural net. 

* * * * *

Comparison of Semantic Nets (left) with Attractor Nets (right). See below.

Color me pleased with the AI essay I just published in 3 Quarks Daily, Some Notes On Computers, AI, And The Mind, and my follow-up here, What’s AI? – @3QD [& the contrasting natures of symbolic and statistical semantics]. These notes are, in turn, a follow-up to my follow-up. In particular, I want to elaborate on these concluding paragraphs:

Now, let’s compare these two systems. In the symbolic computation semantics we have a relational graph where both the nodes and the arcs are labeled. All that matters is the topology of the graph, that is, the connectivity of the nodes and arcs, and the labels on the nodes and arcs. The nature of those labels is very important.

In the statistical system words are in fixed geometric positions in a high-dimensional space; exact positions, that is, distances, are critical. This system, however, doesn’t need node and arc labels. The vectors do all the work.

These statements are, I feel, at the right level of generalization and abstraction. The purpose of these notes is to lay out some of the things that would have to be taken into consideration in order to further develop those ideas.

HOWEVER, it’s sketchy & full of holes. First the sketchy stuff. Then I’ve got links to material that’s somewhat more worked out. It’ll help full in the holes and flesh out the details.

Stream of consciousness ramble-through

Caveat: These are informal notes, mostly for myself. I list them here as place-holders for work that needs to be done.

How do you make inference in the system? Symbolic model allows for a logic based on node and arc types. Such a model treats the knowledge structure as a database and makes inferences over it. Humans are not like that. In statistical systems there is no capacity for ‘externally’ guided inference. All inference is ‘black-boxed’ and internal to the system.

How do you add “meaning” to a statistical model? What is “meaning” anyhow? Does meaning ultimately require/imply interactive coupling with [living in] a world? And doesn’t it imply/require an intending subject?

Is this the ultimate import of critiques such as those of Dreyfus and Searle? But aren’t those ‘cheap’ critiques, arrived at without knowledge of how these systems work? And what of it?

How do we get rid of arc and node labels? Doesn’t that amount to coupling with the external world? See David Hays in Cognitive Structures (HRAF Press 1981) and Sydney Lamb’s notation for his stratificational grammar. And then we have my attractor nets, where the nodes are logical operators over attractor landscapes and the arcs are attractors within those landscapes.

If symbolic systems are more flexible and ‘deeper’ how come the statistical systems are more powerful in actual applications? The symbolic systems get their knowledge directly from the humans that design them. The makers decide on node and arc types and the way to make inferences over them; and the makers encode knowledge directly into the system. Statistical systems learn. NLP systems in effect back in to a simulacrum of the knowledge humans have ‘baked-in’ to the texts they generate. But what about visual systems based on unsupervised learning? Those systems are directly ‘in touch with’ an ‘external world’, no?

What about the phenomenal power of chess and Go programs? These programs are now more powerful than the best human players. I note as well that neither chess nor Go require contact with a rich physical world. Both take place in a 2D world containing a very limited universe of objects and in which there are very limited opportunities for action. For all practical purposes, there is no world external to the computer, unlike what we have for translation systems, text generation systems, or car-driving systems. Does this imply that perhaps the best and most powerful use of these systems is in the internal configuration of and maintenance of computational systems themselves? But won’t that make these systems even more ‘black-boxy’ than they already are?

And speech-to-text, text-to-speech?

Note that learning systems of various sorts do require exposure to mountains of data, whether externally generated and presented to the system or created by the system itself (as in game systems that compete against themselves). Humans do not require exposure to so many cases for effective learning. What’s this about? Well, for one thing it’s about having a brain and body that evolved to fit the world: no blank slate. Does is all cascade back on to that, or is there more?

Natural intelligence and metaphor

While much of my work with David Hays was anchored in a symbolic systems approach to the mind, we did venture elsewhere. Thus when the symbolic systems approach collapsed in the mid-1980s we were, if not exactly prepared, able to keep moving on. We had other conceptual irons in the forge.

In Cognitive Structures, which I linked above, Hays came up with a scheme to ground a digital cognitive system in an analog sensorimotor system. A few years after that he and I worked on a paper where we grounded our whole system in neural systems:

William L. Benzon and David G. Hays, Principles and development of natural intelligence, Journal of Social and Biological Systems 11, 293-322, 1988, https://www.academia.edu/235116/Principles_and_Development_of_Natural_Intelligence.

Abstract: The phenomena of natural intelligence can be grouped into five classes, and a specific principle of information processing, implemented in neural tissue, produces each class of phenomena. (1) The modal principle subserves feeling and is implemented in the reticular formation. (2) The diagonalization principle subserves coherence and is the basic principle, implemented in neocortex. (3) Action is subserved by the decision principle, which involves interlinked positive and negative feedback loops, and resides in modally differentiated cortex. (4) The problem of finitization resolves into a figural principle, implemented in secondary cortical areas; figurality resolves the conflict between pro-positional and Gestalt accounts of mental representations. (5) Finally, the phenomena of analysis reflect the action of the indexing principle, which is implemented through the neural mechanisms of language.

These principles have an intrinsic ordering (as given above) such that implementation of each principle presupposes the prior implementation of its predecessor. This ordering is preserved in phylogeny: (1) mode, vertebrates; (2) diagonalization, reptiles; (3) decision, mammals; (4) figural, primates; (5) indexing. Homo sapiens sapiens. The same ordering appears in human ontogeny and corresponds to Piaget's stages of intellectual development, and to stages of language acquisition.

I note that we conceived of the second principle, diagonalization, as involving holograph-like processing, which involves convolution. Convolution is important in some forms of contemporary neural net systems. Subsequently we published a paper in which we used convolution to account for metaphor:

William L. Benzon and David G. Hays, Metaphor, Recognition, and Neural Process, The American Journal of Semiotics, Vol. 5. No. 1 (1987), 59-80, https://www.academia.edu/238608/Metaphor_Recognition_and_Neural_Process.

Abstract: Karl Pribram's concept of neural holography suggests a neurological basis for metaphor: the brain creates a new concept by the metaphoric process of using one concept as a filter — better, as an extractor — for another. For example, the concept "Achilles" is "filtered" through the concept "lion" to foreground the pattern of fighting fury the two hold in common. In this model the linguistic capacity of the left cortical hemisphere is augmented by the capacity of the right hemisphere for analysis of images. Left-hemisphere syntax holds the tenor and vehicle in place while right-hemisphere imaging process extracts the metaphor ground. Metaphors can be concatenated one after the other so that the ground of one metaphor can enter into another one as tenor or vehicle. Thus conceived metaphor is a mechanism through which thought can be extended into new conceptual territory.

Showdown at the AI Corral, or: What kinds of mental structures are constructible by current ML/neural-net methods? [& Miriam Yevick 1975]

There’s a big conversation going on in the AI world these days about appropriate architectures: Can new-style machine-learning/neural-net methods take us all the way to the Holy Land or do we need to incorporate old-style structured symbolic models? My sense the that most current practitioners learn toward the former position; I favor the latter. (As for the Holy Land, it’s a mirage.)

In the first section I present an important, if somewhat neglected, paper from 1975 and what David Hays and I made from it. Then some tweets culled from the current stream. I conclude with some crazy stuff, my notes on attractor networks and the fluid mind. That crazy stuff is about blending the two kinds of logic, intermixing structured symbolic systems with data-driven machine/deep learning systems. Something like that.

Yevick’s Law: Two Kinds of Logic

As Louis Armstrong used to say, it’s one of those old time good ones:

Yevick, Miriam Lipschutz (1975) Holographic or Fourier logic. Pattern Recognition 7: 197-213.
https://doi.org/10.1016/0031-3203(75)90005-9

Abstract: A tentative model of a system whose objects are patterns on transparencies and whose primitive operations are those of holography is presented. A formalism is developed in which a variety of operations is expressed in terms of two primitives: recording the hologram and filtering. Some elements of a holographic algebra of sets are given. Some distinctive concepts of a holographic logic are examined, such as holographic identity, equality, containment and “association”. It is argued that a logic in which objects are defined by their “associations” is more akin to visual apprehension than description in terms of sequential strings of symbols.

Yes, it was published in 1975, which is ancient times in the world of artificial intelligence. It was inspired by a body of theorizing and evidence – promulgated by Karl Pribram, among others – that the neocortical processing was based on holographic principles rather than those of propositional/symbolic logic. It seems to me that what Yevick called holographic logic is similar in spirit, and even in mathematics in some respects, to current work on neural networks, while, in contrast, ordinary logic is as the abstract has it, "description in terms of sequential strings of symbols."

A decade later David Hays and I called on that paper in a highly speculative synthesis of a variety of work in cognitive, neural, perceptual, and comparative psychology with computational orientation:

William Benzon and David Hays, Principles and Development of Natural Intelligence, Journal of Social and Biological Structures, Vol. 11, No. 8, July 1988, 293-322.
https://www.academia.edu/235116/Principles_and_Development_of_Natural_Intelligence

We sketched out five principles. The fourth principle, which we called the figural principle, was based on Yevick’s work. Here’s how we opened the discussion:

The figural principle concerns the relationship between Gestalt or analogue process in neural schemas and propositional or digital processes. In our view, both are necessary; the figural principle concerns the relationship between the two types of process. The best way to begin is to consider Miriam Yevick's work (1975, 1978) on the relationship between 'descriptive and holistic' (analogue) and 'recursive and ostensive' (digital) processes in representation.

Fig. 10. Yevick's law. The curves indicate the level of representational complexity required for a good identification

The critical relationship is that between the complexity of the object and the complexity of the representation needed to ensure specific identification. If the object is simple, e.g. a square, a circle, a cross, a simple propositional schema will yield a sharp identification, while a relatively complex Gestalt schema will be required for an equivalently good identification (see Fig. 10). Conversely, if the object is complex, e.g. a Chinese ideogram, a face, a relatively simple Gestalt (Yevick used Fourier transforms) will yield a sharp identification, while an equivalently precise propositional schema will be more complex than the object it represents. Finally, we have those objects which fall in the middle region of Figure 10, objects that have no particularly simple description by either Gestalt or propositional methods and instead require an interweaving of both. That interweaving is the figural principle.

Definition. The figural mechanism brings environments of moderate complexity within the limits of computability by putting a propositional assemblage of local narrow band-width Gestalts into a framework provided by global wide band-width analysis to achieve cross-validation of the two analyses.

At various points later in the essay of the propositional reconstruction Gestalt processes. In our view the mind evolves through an interweaving of these two kinds of logic. Both are necessary.

Subsequently we used this line of thought in a paper about metaphor:

William Benzon and David Hays, Metaphor, Recognition, and Neural Process, The American Journal of Semiotics, Vol. 5, No. 1 (1987), 59-80.
https://www.academia.edu/238608/Metaphor_Recognition_and_Neural_Process

Karl Pribram's concept of neural holography suggests a neurological basis for metaphor: the brain creates a new concept by the metaphoric process of using one concept as a filter — better, as an extractor — for another. For example, the concept “Achilles” is “filtered” through the concept “lion” to foreground the pattern of fighting fury the two hold in common. In this model the linguistic capacity of the left cortical hemisphere is augmented by the capacity of the right hemisphere for analysis of images. Left-hemisphere syntax holds the tenor and vehicle in place while right-hemisphere imaging process extracts the metaphor ground. Metaphors can be concatenated one after the other so that the ground of one metaphor can enter into another one as tenor or vehicle. Thus conceived metaphor is a mechanism through which thought can be extended into new conceptual territory.

Caveat: Students of cognitive linguistics should think of this as an account of blending rather than (cognitive) metaphor.

Some current work, from the Twitterverse

"Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective": updated, #IJCAI2020 https://t.co/FiRnLSbfXl w/@AvilaGarcez @marceloprates_ @vardi @Melleo54Sis Avelar, drawing inspiration from many: @GaryMarcus @kahneman_daniel E. Davis et al #neuralsymbolic #AI pic.twitter.com/gXiKBR55dV

— Luis Lamb (@luislamb) June 1, 2020

GPT-3: Language Models are Few-Shot Learners, by @notTomBrown et al.

“We train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting.”https://t.co/qhcHPSH22I pic.twitter.com/ng1Dc6aFg3

— hardmaru (@hardmaru) May 29, 2020

Exactly what you get when you look for your keys where the streetlight is. https://t.co/PkGOtWsPyu

— Gary Marcus (@GaryMarcus) May 30, 2020

Behavior: The Control of Perception – Bill Powers rediscovered, again!

I decided to cruise by Slate Star Codex and saw a post with the title, What Intellectual Progress did I Make in the 2010S? Sounds ambitious, thought I do myself. [Hmmm...should I write such a post? Umm, err, I think not.*] This, the second paragraph, stopped me dead in my tracks:

I think the single most important thing I discovered this decade (due to a random comment in the SSC subreddit!) was the predictive coding theory of the brain. I started groping towards it (without knowing what I was looking for) in Mysticism And Pattern-Matching, reported the exact moment when I found it in It’s Bayes All The Way Up, and finally got a decent understanding of it after reading Surfing Uncertainty. At the same time, thanks to some other helpful tips from other rationalists, I discovered Behavior: The Control Of Perception, and with some help from Vaniver and a few other people was able to realize how these two overarching theories were basically the same. Discovering this area of research may be the best thing that happened to me the second half of this decade (sorry, everyone I dated, you were pretty good too).

It’s that reference to Behavior: The Control Of Perception, that caught my eye. It was published in 1973 by William Powers, positively reviewed a couple years later in Science, and had been central to the work that I’d done with David Hays in his computational linguistics research group at SUNY Buffalo in the mid-1970s. Back when I listed the ten books that had most influenced my thinking, that was one of them. [Note: I’ve got a number of posts about or at least mentioning Powers.]

But, for some reason, Powers’s thought never really caught on – though I note, in passing, that Ted Cloak, another forgotten thinker, also found his work valuable. By the mid-1980s or so a small group of thinkers had coalesced around him and began holding annual meetings. I never attended any of those, though I joined a mailing list for the group, and I presented with them at some meetings of the American Society for Cybernetics. Powers died in 2013, but I assume that group still meets.

Given that Powers has had relatively little influence, are those of us who HAVE been influenced by him wrong? I suppose that I’m not exactly in a position to offer up a defense, but I do find it interesting that Scott Alexander, proprietor of Slate Star Codex, should put his book front and center in his review of his intellectual decade. He prefaces his review Powers' book with a disclaimer (his italics): “Epistemic status: I only partly understood this book and am trying to review it anyway as best I can.” In the course of his review he expresses major doubts about aspects of Powers’s model. And the review ends in a string of questions without answers:

How useful is this book? I guess that depends on how metaphorical you want to be. Is the brain a control system? I don’t know. Are police a control system trying to control crime? Are police a “response” to the “stimulus” of crime? Is a stimulus-response pairing a control system controlling for the quantity of always making sure the stimulus has the response? I think it’s interesting and helpful to think of some psychological functions with these metaphors. But I’m not sure where to go from there.

That’s a mountain of doubt. And yet somehow that abstract, elegant, and elusive book moved the mountain.

I can understand Alexander’s reservations. When Hays, his other students, and I worked it over, we discarded and/or reworked major portions of the model. But despite that we kept the overall outline, including, believe it or not, his comments about consciousness and reorganization. In a way, especially those.

It’s that overall outline – though outline is an inadequate word, gestalt is perhaps better – that made it so attractive for us. It was and is a biologically grounded model of the mind based on classical control theory engineering – feedback loops, etc. It also assigned a coherent function to consciousness, Powers called it reorganization, but to appreciate the weight and valence of that them, you have to think about his whole model.

That it was based in cybernetics is perhaps why it never found favor. But the time Powers had published the book the so-called cognitive revolution was going into over drive. All the cool kids were thinking about digital computers, while Powers was thinking analog. Of course, we knew all that. Hays had been a first generation researcher in machine translation and, as such, one of the founders of computational linguistics. Our research group was ABOUT computational linguistics. But we, like so many others, were reaching for the mind. And we had decided/realized that computation alone wouldn’t get us there. So we took the gestalt that Powers had created and opened it up to include language, symbolic computation, in a more realistic way [see David Hays, Cognitive Structures, HRAF Press, 1981]. Powers kept us grounded in biology, we opened him up to language. That’s the line we took.

* * * * *

What’s a Language? Evidence from the Brain

Bumping this to the top of the queue. I'm thinking about classification of cultural items and it's useful to be reminded that "language" is an informal notion that doesn't hold up under careful analysis. That's what this post is about. 

Yesterday I put up a post (A Note on Memes and Historical Linguistics) in which I argued that, when historical linguists chart relationships between things they call “languages”, what they’re actually charting is mostly relationships among phonological systems. Though they talk about languages, as we ordinarily use the term, that’s not what they actually look at. In particular, they ignore horizontal transfer of words and concepts between languages.

Consider the English language, which is classified as a Germanic language. As such, it is different from French, which is a Romance language, though of course both Romance and Germanic languages are Indo-European. However, in the 11th Century CE the Norman French invaded Britain and they stuck around, profoundly influencing language and culture in Britain, especially the part that’s come to be known as England. Because of their focus on phonology, historical linguists don’t register this event and its consequences. The considerable French influence on English simply doesn’t count because it affected the vocabulary, but not the phonology.

Well, the historical linguists aren’t the only ones who have a peculiar view of their subject matter. That kind of peculiar vision is widespread.

Let’s take a look at a passage from Sydney Lamb’s Pathways of the Brain (John Benjamins 1999). He begins by talking about Roman Jakobson, one of the great linguists of the previous century:

Born in Russia, he lived in Czechoslovakia and Sweden before coming to the United States, where he became a professor of Slavic Linguistics at Harvard. Using the term language in a way it is commonly used (but which gets in the way of a proper understanding of the situation), we could say that he spoke six languages quite fluently: Russian, Czech, German, English, Swedish, and French, and he had varying amounts of skill in a number of others. But each of them except Russian was spoken with a thick accent. It was said of him that, “He speaks six languages, all of them in Russian”. This phenomenon, quite common except in that most multilinguals don’t control as many ‘languages’, actually provides excellent evidence in support of the conclusion that from a cognitive point of view, the ‘language’ is not a unit at all.

Think about that. “Language” is a noun, nouns are said to represent persons, places, or things – as I recall from some classroom long ago and far away. Language isn’t a person or a place, so it must be a thing. And the generic thing, if it makes any sense at all to talk of such, is a self-contained ‘substance’ (to borrow a word from philosophy), demarcated from the rest of the world. It is, well, it’s a thing, like a ball, you can grab it in your metaphorical hand and turn it around as you inspect it.

A Quick Note on Description: It’s not just words

I’ve been thinking explicitly about description for at least a decade – my manifesto, as it were, Literary Morphology, talks of description and it was published in 2006 – and it’s been a steady topic a New Savanna since 2011. All this time it’s been obvious to me that description is not merely verbal, that one uses visual devices as well. I suppose that’s obvious because that’s been my practice for four decades, but I’m not unique there.

But, as far as I can tell, the emerging discussion of description in literary criticism hasn’t gotten that far yet – I’m thinking particularly of the Representations special issue (Summer 2016) and Sharon Marcus’s recent article, Erich Auerbach’s Mimesis and the Value of Scale, MLQ, 2016, 77(3): 297-319). This is natural, I suppose, not simply because literary scholars would be partial to words, but because that’s what you find in dictionaries, e.g.: “a spoken or written representation or account of a person, object, or event”.

But it’s limiting and won’t get literary criticism very far. It will keep the discipline imprisoned in its discursive box.

Consider this observation by Sydney Lamb, a linguist of Chomsky’s generation but of a very different intellectual temperament. Lamb cut his intellectual teeth on computer models of language processes and was concerned about the neural plausibility of such models. He is one of the first thinkers to use networks as representations of language structures and processes. In his major systematic statement, Pathways of the Brain: The Neurocognitive Basis of Language (John Benjamins 1999) remarked on importance of visual notation (p. 274): “... it is precisely because we are talking about ordinary language that we need to adopt a notation as different from ordinary language as possible, to keep us from getting lost in confusion between the object of description and the means of description.” That is, we need the visual notation in order to objectify language mechanisms.

Franco “distant reading” Moretti has made a similar observation. This is from the Literary Lab’s Pamphlet 2 (May 2011), Network Theory, Plot Analysis (PDF) (p. 4):

Third consequence of this approach: once you make a network of a play, you stop working on the play proper, and work on a model instead: you reduce the text to characters and interactions, abstract them from everything else, and this process of reduction and abstraction makes the model obviously much less than the original object – just think of this: I am discussing Hamlet, and saying nothing about Shakespeare’s words – but also, in another sense, much more than it, because a model allows you to see the underlying structures of a complex object.

By drawing a network of character relationships one has created a model that is clearly distinguishable from the (physical) text itself. One has objectified an (aspect of an) underlying mechanism.

Alas, there it is, that nasty word, “objectified”. But really, much current criticism is so “far” from any text that quibbles about objectification are just that, quibbles.  The fact is, the clean separation between the investigator’s writing and the language under investigation that is valued by Lamb and Moretti gets in the way of the illusion of reading so valued by literary critics. Thus Tony Jackson will assert (“Literary Interpretation” and Cognitive Literary Studies, Poetics Today 24 (2) 2003: 191-205):

That is, a literary interpretation, if we are allowed to distinguish it as a distinct kind of interpretation, joins in with the literariness of the text. Literary interpretation is a peculiar and, I would say, unique conjunction of argument and literature, analytic approach and art form being analyzed. [p. 202]

Just what that means is not at all clear, though Jackson goes on to quote J. Hillis Miller as asserting (here he is quoting Miller) “the one who interprets fictions becomes a fiction and a maker of fictions” (203).

The desire that the critic somehow merges seamlessly with the text is obvious. It is a trope, a way of speaking and thinking, of contemporary literary criticism. It is a pretense and everyone knows it.

Why not drop it?

Moretti Update: Visualization and Objectification

Early in 2006 The Valve sponsored a book event on Franco Moretti’s Graphs, Maps, Trees. A dozen thinkers prepared comments on the book, Moretti responded, The Valve’s commentariat joined in the fun. This event has subsequently be edited into a book, Jonathan Goodwin and John Holbo, eds., Reading Graphs, Maps, Trees (2011). A few years later The Chronicle of Higher Education ran a piece on Moretti’s Literature Lab at Stanford; and this kicked off a lively Valve discussion.

Moretti’s latest work has kicked off a discussion at Crooked Timber. In this work Moretti has graphed networks of relationships between characters in texts. The basic idea is simple: write the names of characters on a sheet of paper. If two characters talk to one another, connect them with a line, thus:

Moretti has reported his results in The New Left Review and in a somewhat longer pamphlet available for download from his lab.

In this post I’m not concerned with the Moretti’s results, I’m interested in a comment he made along the way. This is from page four of the pamphlet:

Third consequence of this approach: once you make a network of a play, you stop working on the play proper, and work on a model instead: you reduce the text to characters and interactions, abstract them from everything else, and this process of reduction and abstraction makes the model obviously much less than the original object – just think of this: I am discussing Hamlet, and saying nothing about Shakespeare’s words – but also, in another sense, much more than it, because a model allows you to see the underlying structures of a complex object.

This is an important methodological point. By drawing a network of character relationships one has created a model that is clearly distinguishable from the text itself. One has objectified an underlying mechanism.