Dwarkesh's problems with LLMs: They can't learn! [the problem's architectural]

Dwarkesh Patel, Why I have slightly longer timelines than some of my guests, Dwarkesh Podcast, June 2, 2025.

Sometimes people say that even if all AI progress totally stopped, the systems of today would still be far more economically transformative than the internet. I disagree. I think the LLMs of today are magical. But the reason that the Fortune 500 aren’t using them to transform their workflows isn’t because the management is too stodgy. Rather, I think it’s genuinely hard to get normal humanlike labor out of LLMs. And this has to do with some fundamental capabilities these models lack.

I like to think I’m “AI forward” here at the Dwarkesh Podcast. I’ve probably spent over a hundred hours trying to build little LLM tools for my post production setup. And the experience of trying to get them to be useful has extended my timelines. I’ll try to get the LLMs to rewrite autogenerated transcripts for readability the way a human would. Or I’ll try to get them to identify clips from the transcript to tweet out. Sometimes I’ll try to get it to co-write an essay with me, passage by passage. These are simple, self contained, short horizon, language in-language out tasks - the kinds of assignments that should be dead center in the LLMs’ repertoire. And they’re 5/10 at them. Don’t get me wrong, that’s impressive.

But the fundamental problem is that LLMs don’t get better over time the way a human would. The lack of continual learning is a huge huge problem. The LLM baseline at many tasks might be higher than an average human’s. But there’s no way to give a model high level feedback. You’re stuck with the abilities you get out of the box. You can keep messing around with the system prompt. In practice this just doesn’t produce anything even close to the kind of learning and improvement that human employees experience.

The reason humans are so useful is not mainly their raw intelligence. It’s their ability to build up context, interrogate their own failures, and pick up small improvements and efficiencies as they practice a task.

There’s a reason why LLMs can’t learn, their architecture. In a standard digital computer we have a clean separation between processing and memory. Consequently it’s easy to add more memories to the system. Just find a chunk of “blank tape,” or attach or splice some “blank tape” in, and then pur your new memories there. While LLMs are themselves implemented on a standard (so-called von Neuman) architecture, LLM itself is quite different. Within the LLM memory and processing are not separate, making it very difficult to add new memories. Where do they go? How are they connected to existing memories? Those are profound issues. How does the brain deal with them? Like so much else about the brain, we don’t know. [Note: I discuss these issues in various posts on the Structured Physical Systems Hypothesis, which are tagged: SPSH.]

Dwarkesh continues:

LLMs actually do get kinda smart and useful in the middle of a session. For example, sometimes I’ll co-write an essay with an LLM. I’ll give it an outline, and I’ll ask it to draft the essay passage by passage. All its suggestions up till 4 paragraphs in will be bad. So I’ll just rewrite the whole paragraph from scratch and tell it, “Hey, your shit sucked. This is what I wrote instead.” At that point, it can actually start giving good suggestions for the next paragraph. But this whole subtle understanding of my preferences and style is lost by the end of the session.

Maybe the easy solution to this looks like a long rolling context window, like Claude Code has, which compacts the session memory into a summary every 30 minutes. I just think that titrating all this rich tacit experience into a text summary will be brittle in domains outside of software engineering (which is very text-based).

Note that in such cases we are not dealing with memory that has “settled” or “been distilled into the weights.” This is a live memory; it’s fluid. It’s for this reason that I’ve come to think of the brain as a polyviscous fluid. It’s all fluid, all the time. As Walter Freeman once pointed out to me, any living neuron is spiking; that’s what it means for a neuron to be alive. But inactive neurons simply don’t spike as often as fully active ones do. Hence, polyviscosity. Some regions of the fluid are very viscous, others not at all, with all gradations in between.

But the brain is an organic system, consisting mostly of water, which is held by somewhat permeable membranes (neuron walls), with varying concentrations of trace substances (neuotransmitters). How can we create such polyviscosity in a silicon-based system?

Dwarkesh continues:

If AI progress totally stalls today, I think [less than] 50% of white collar employment goes away. Sure, many tasks will get automated. Claude 4 Opus can technically rewrite autogenerated transcripts for me. But since it’s not possible for me to have it improve over time and learn my preferences, I still hire a human for this. Without progress in continual learning, I think we will be in a substantially similar position with white collar work - yes, technically AIs might be able to do a lot of subtasks somewhat satisfactorily, but their inability to build up context will make it impossible to have them operate as actual employees at your firm.

While this makes me bearish on transformative AI in the next few years, it makes me especially bullish on AI over the next decades. When we do solve continuous learning, we’ll see a huge discontinuity in the value of the models.

That, I submit, will require new architectures based on a different physical substrate. Who’s currently working on producing that substrate? I’m sure than someone is, though I can’t cough up a name at the moment, but how much funding do they have? I’ll tell you: Not nearly enough. 

 There’s much more at the link.

Consciousness, Intelligence, and AI – Some Quick Notes [call it a mini-ramble]

The subject of consciousness keeps turning up in current discussions of AI and LLMs. Can AIs be conscious? Are current AIs conscious? Maybe a little?

What do consciousness and intelligence have to do with one another? I see no reason to think that dogs, rats, and cats are not conscious, though I have no idea how far down the phylogenetic chain consciousness exists. No one would argue that dogs, rats, and cats are as intelligence as we are. Intelligence is something different from consciousness, no?

And yet the issue gets raised. One line of (implicit) reasoning seems to go like this: It converses with me in an intelligent way, things that converse with me in an intelligent way (or even at all!) are conscious, therefore it must be conscious. And then there’s the fact that you can ask a chatbot about itself and it will say something, though just what it says depends on what it has been RLHFed to say. But, still, these LLMs have been “trained” on tons of text using the word “consciousness” and all its cognates, so sure, it can use the word in human-seeming ways. That doesn’t make it conscious.

An extra stage of information processing?

Fact is, philosophers often argue about consciousness as though it were a further or extra stage in...in what? Human information processing? Thinking? Whatever. It adds something extra, something beyond what was before. Let’s say it adds an extra bit of intelligence. Yeah, let’s say that.

So, a conscious being is more intelligent that its non-conscious simulacrum, to which it is otherwise identical. But then we have those philosophical zombies and they, presumably, are as intelligent as non-zombies.

Reorganization

This strikes me as being wrong-headed. I take my conception of consciousness from Wm Powers (Behavior: The Control of Perception). Consciousness enables reorganization. I explain this in a post from 2022: Consciousness, reorganization and polyviscosity, Part 1: The link to Powers. (It’s complicated, so I’m not going to try to summarize it here.) In that conception, consciousness really isn’t a further step in reasoning, though it may facilitate “moving around” in one’s mind (in particular, think default mode). Reorganization isn’t a further step in reasoning, though it may seem like it. (Should say more about this, but later.)

Turing and non-Turing computing

And then we have the difference between Turing computation and, shall we say, neural computation. Turing computation requires a strict separation of processing from memory (e.g. see article linked in this recent post). That’s not how the brain works. As I recall, von Neumann wondered about that in his little book on the brain. If neurons are memory, as they surely are, then where’s the processing? he wondered. In a Turing device, learning means adding new blocks of memory. In a neural device, where memory and computation are not separate, learning means, well, it means reorganization, to use Powers’s term. Things have to change all over the place, more in some places than others, perhaps a lot in some few places and very damn little in most places.

Well, consciousness is what allows that to happen. Consciousness mediates reorganization. Consciousness modulates and “distributes” change throughout the system. A Turing system doesn’t need consciousness in order to learn, to change. A network (or neural) system does. Consciousness is the mechanism that solves learning for neural systems. I further hypothesize that the glial cells are crucial here.

I’m not sure of how to formulate that, but that seems to be my key thought for the morning. It’s why I got out of bed at 5:41 AM.

Artificial neural nets

So, what of artificial neural nets and consciousness? Well, the machines themselves are Turing-type machines. No need for consciousness. Does that also imply no possibility for consciousness? Skip that for now. The neural net, however, is not a Turing-type machine. It’s, well, it is a network and, as such, does not distinguish between memory and processing. But it’s a network that’s stored in the memory of a Turing-type machine. And we don’t yet know how to reorganize such a network, that is, to add new information to it. But we may well solve that problem one of these days. I don’t see why not?

Would that make the artificial neural net conscious? Or will it only be simulating consciousness? Remember, it’s running on a non-consciousness Turing-type machine. I think it’s only a simulation, not the real thing.

Another item – What Chatbots can’t seem to do

For examples of what I have in mind, see these posts:

• Claude 3.5 Sonnet visits Times Square, waxes philosophical (after being prodded)
• Conversing with ChatGPT about Jaws, Mimetic Desire, and Sacrifice

If that’s the case – and here I’m taking a leap – then I do believe it’s consciousness that connects us to the world. Not a surprising thought on the face of it, but in THIS context, some explaining is required. LATER.

And Superintelligence?

Well, if superintelligence is a species of intelligence, then it doesn’t imply consciousness. And if consciousness is our/then connection to the world, then superintelligence is a zombie (yeah, I know, another leap). I’m mean, these SOA LLMs already “know,” in some meaningful sense, more than any individual human. That’s some kind of superintelligence. Whatever kind of superintelligence that is, it doesn’t bother us.

Nor does the superintelligence of AlphaZero bother us. Superintelligence? But that’s narrow intelligence. Does the fact that it’s narrow mean it can’t be super? Does the emergence of the concept of AGI, it’s differentiation from AI, mark the break-down of the classical concept of intelligence and its pursuit by artificial means?

What about the AIs that do protein folding and now 15-day weather predicting? We can’t do either of those things. Provisionally, why not?

At the moment I’m thinking: Superintelligence? Bring it on. 

Let them be super.

Consciousness is ours, but also dogs, cats, rats, and other animals.

During the Day [addendum]

We are awake, mostly. When we’re awake, we’re conscious, mostly. Once we’re out of infancy, and perhaps toddlerhood, we are conscious most of the time. As I said, it is consciousness that connects us to the world.

Note, however, that consciousness is very mobile. Are attention can flit from one thing to another, quite freely. [Hence the literary technique, stream of consciousness.] I suppose we could analogize it to time-sharing in computers, but...

Consciousness HAS to be flexible and mobile. We live in an unpredictable world. We may get around by predicting the next thing or three, but sometimes the world intervenes, drastically. We’ve got to be able to disconnect from the prediction and attend to the real.

[I recall an email exchange with Walter Freeman about this. I asked him whether or not a state of global coherence was necessary for us to be able to make a quick change from one thing to another. He said yes.]

More later.

Conscious artificial intelligence and biological naturalism

2/ As AI continues to develop, it is natural to ask whether AI systems can be not only intelligent, but also conscious. But is this likely? And what would the consequences be?

— Anil Seth (@anilkseth) December 10, 2024

4/ I first consider why people might think that AI is, or could be, conscious. I show how we can be led astray by built-in biases of anthropocentrism, human exceptionalism, and anthropomorphism. Intelligence and consciousness are not the same thing. pic.twitter.com/EafyFzhQrJ

— Anil Seth (@anilkseth) December 10, 2024

The tweet stream continues through #20. The article:

Anil K. Seth, Conscious artificial intelligence and biological naturalism, PsyArXiv Preprints, 2024-12-10.

Abstract: As artificial intelligence (AI) continues to advance, it is natural to ask whether AI systems can ibe not only intelligent, but also conscious. I consider why people might think AI could develop consciousness, identifying some biases that lead us astray. I ask what it would take for conscious AI to be a realistic prospect, challenging the assumption that computation provides a sufficient basis for consciousness. I’ll instead make the case that consciousness depends on our nature as living organisms – a form of biological naturalism. I lay out a range of scenarios for conscious AI, concluding that real artificial consciousness is unlikely along current trajectories, but becomes more plausible as AI becomes more brain-like and/or life-like. I finish by exploring ethical considerations arising from AI that either is, or convincingly appears to be, conscious. If we sell our minds too cheaply to our machine creations, we not only overestimate them – we underestimate our selves.

From the article, on non-Turing computation:

Turing computation is powerful, but not every function is Turing-computable. Turing himself identified a class of non-computable functions in his response to the ‘halting problem’ posed by Hilbert (Turing, 1936). Other examples include functions involving continuous variables, stochastic/random elements, and unbounded sensitivity to initial conditions (e.g., deterministic chaos). Digital computers based on Turing machines can simulate and approximate non-computable functions – this happens all the time in computational modelling – but these approximations will generally not be exact.

The limited remit of Turing computation means that systems – including brains – might implement functions that are non-Turing-computational. The idea that mental states (including consciousness) depend on non-computational functions is called non-computational functionalism (Piccinini, 2018, 2020). Non-computational neural functions could include processes relating to (continuous) electromagnetic fields, fine-grained timing relations (only order, not dynamics as such, matters for Turing computation), freely diffusing neurotransmitters, and so on. Non-computational biological functions also include those that necessarily involve a particular material property: examples include digestion, circulation of blood, and metabolism. Note that computational and non-computational functions could co-exist. For example, it could be that some aspects of mind are computational, but not consciousness (Piccinini, 2023).

Many other notions of ‘computation’ have been proposed. Some are narrower than Turing computation (e.g., computation requiring an artefact being used by a person in a particular way), but most are broader (N. G. Anderson & G. Piccinini, 2024; Chalmers, 1996b). Broader forms of computation include analogue, neuromorphic, and mortal computation. I will return to these later. For now, a focus on Turing computation is justified since this kind of computation underlies conventional AI, whether based on artificial neural networks or otherwise.

Mortal computation:

The recent concept of mortal computation is particularly interesting (Hinton, 2022; Ororbia & Friston, 2023). Standard Turing computation is ‘immortal’. Its existence and utility outlast the existence of any specific instance of hardware. This reflects the core computer science principle that software should be separable from hardware both in principle and in practice, so that the same algorithm executed on different hardware gives the same result. But immortal computation is expensive. It requires continual error correction to ensure that 1s remain 1s (and 0s remain 0s). As algorithms and models grow in complexity, the computational and energetic costs of error correction, and therefore of computational immortality, grows quickly.

One implication of this argument is that biological brains, which are highly energy efficient, cannot be implementing immortal computations. If they are implementing computations at all, then these computations are likely to be mortal, which means they cannot be separated from the ‘hardware’ (or ‘wetware’) which implements them. This in turn places constraints on the multiple realisability and substrate flexibility of these (mortal) computations. In particular, the substrate flexibility required for conscious AI is unlikely to hold because (conventional) AI is based on an implementation paradigm which assumes computational immortality. I find this a provocative argument against the plausibility of conscious AI because it is based on limitations arising from within a computational view of mind.

There's much more in the article.

The geometry of intelligence: From protein folding to language and the epistemic affordances of the transformer architecture

Fabian Offert, Paul Kim, Qiaoyu Cai, Synthesizing Proteins on the Graphics Card. Protein Folding and the Limits of Critical AI Studies, arXiv:2405.09788v1

Abstract: This paper investigates the application of the transformer architecture in protein folding, as exemplified by DeepMind’s AlphaFold project, and its implications for the understanding of large language models as models of language. The prevailing discourse often assumes a ready-made analogy between proteins – encoded as sequences of amino acids – and natural language – encoded as sequences of discrete symbols. Instead of assuming as given the linguistic structure of proteins, we critically evaluate this analogy to assess the kind of knowledge-making afforded by the transformer architecture. We first trace the analogy’s emergence and historical development, carving out the influence of structural linguistics on structural biology beginning in the mid-20th century. We then examine three often overlooked pre-processing steps essential to the transformer architecture, including subword tokenization, word embedding, and positional encoding, to demonstrate its regime of representation based on continuous, high-dimensional vector spaces, which departs from the discrete, semantically demarcated symbols of language. The successful deployment of transformers in protein folding, we argue, discloses what we consider a non-linguistic approach to token processing intrinsic to the architecture. We contend that through this non-linguistic processing, the transformer architecture carves out unique epistemological territory and produces a new class of knowledge, distinct from established domains. We contend that our search for intelligent machines has to begin with the shape, rather than the place, of intelligence. Consequently, the emerging field of critical AI studies should take methodological inspiration from the history of science in its quest to conceptualize the contributions of artificial intelligence to knowledge-making, within and beyond the domain-specific sciences.

Think of this in relation to the Structured Physical System Hypothesis (SPSH) of Saty Chary:

‘A structured physical system has the necessary and sufficient means for specific intelligent response’. By structured physical system, I mean, an analog design, e.g. a Rube Goldberg apparatus, or a Braitenberg (!) vehicle, etc. This is in contrast to this: PSSH - Physical Symbol System Hypothesis - 'A physical symbol system has the necessary and sufficient means for general intelligent action'.

Once again, what’s computation? I’m thinking in relation to operations of the human brain?

For me, those questions began when I was faced with the structure of “Kubla Khan”:

That’s just the first 36 lines, but the last 18 look the same:

That looked like the trace of some nested loops. If so, what then?

Let’s put that aside. After all, I’ve been working on that for a LONG time and still haven’t figure it out.

The problem I’m trying to figure out is this: How do we draw a principled line between those brain processes that are computational and those that are not?

Some (most, all?) computer scientists ‘seem to define a computer as “a physical mechanism that can theoretically calculate any computable function”.’ If I go with that notion, then it’s all computation and I have no reason to single out those tree structures in “Kubla Khan” for special attention, or any other linguistic tree structures for that matter. That’s not very helpful.

What if I think of the brain as a structured physical system? When Saty Chary advanced his Structured Physical System Hypothesis (SPSH) he was explicitly playing against the Physical System Hypothesis Hypothesis (PSSH) of Newell and Simon (1976). They advanced the PSSH as the central tenet of cognitive science and AI, and it echoes back to the 1943 paper in which McCulloch and Pitts argued that basic neuronal circuits are logic gates. It turns out that, no, they’re not. They’re quite a bit more complicated.

I like the SPSH. The question for me, then, is: Are their processes taking place in the brain that are NOT dominated by the internal physical activity of the nervous system? I should think so. The process of perception involves interaction with the external world, which necessarily implies that there are things happening in the brain that a driven, though not completely determined by, external physical circumstances. That, I believe, puts us in the world of William Powers, Behavior: The Control of Perception (1973). Language is among those circumstances; for humans it is one of the most important.

That, I think, will allow me to answer my question, though a proper answer will require more construction than I’m willing to undertake here. The relationship between the physical signifier and the signified is arbitrary. In particular, it is arbitrary with respect to the physical processes involved in both. It’s that arbitrary connection implies that language processes are not dominated by the internal physical dynamics of the brain. The basic language process is that of indexing, as Hays and I called it in Principles and Structure of Natural Intelligence. Newell and Simon called it designation.

Now, how can I formulate this idea so that it applies to large language models (in a useful way)? It seems to me that processes in LLMs are dominated by the statistics of the corpus that is modeled. That’s the nub of truth that’s captured by the otherwise reductive phrases, “stochastic parrots,” and “autocomplete on steroids.” The ideas that we require systems that both have access to the external world and that have robust symbolic capabilities, to a first approximation, those amount to freeing the system from being dominated by the statistics of some training corpus. Think about that, carefully.

More later.

The structured physical system hypothesis (SPSH), Polyviscous connectivity [The brain as a physical system]

Time to bump this to the top of the queue, 1) general principle (it's important), and 2) I'm thinking about this stuff at the moment.

* * * * *

Over in the discussion of Yann LeCun’s recent paper (A Path Towards Autonomous Machine Intelligence), Saty Chary has been arguing for something he calls the Structured Physical System Hypothesis (SPSH):

‘A structured physical system has the necessary and sufficient means for specific intelligent response’. By structured physical system, I mean, an analog design, e.g. a Rube Goldberg apparatus, or a Braitenberg (!) vehicle, etc. This is in contrast to this: PSSH - Physical Symbol System Hypothesis - 'A physical symbol system has the necessary and sufficient means for general intelligent action'.

I would add the slide rule as another example. From Wikipedia:

The slide rule is a mechanical analog computer, which is used primarily for multiplication and division, and for functions such as exponents, roots, logarithms, and trigonometry. It is not typically designed for addition or subtraction, which is usually performed using other methods. Maximum accuracy for standard linear slide rules is about three decimal significant digits, while scientific notation is used to keep track of the order of magnitude of results. [...]

At its simplest, each number to be multiplied is represented by a length on a pair of parallel rulers that can slide past each other. As the rulers each have a logarithmic scale, it is possible to align them to read the sum of the numbers' logarithms, and hence calculate the product of the two numbers.

My father used a slide rule for his entire career as an engineer. I learned to use one in my teens – everyone did back that – but never had any use for one. They’ve been replaced by cheap electronic calculators and PCs.

But that’s a digression. It’s the more general Structured Physical System Hypothesis that interests me, as Saty has been argued that the brain is such a system I agree (see this recent post, Once more around the merry-go-round: Is the brain a computer?). Here’s my reply to Saty:

Hi, Saty. I like your hypothesis – Structured Physical System Hypothesis (SPSH) – a lot. I think that a lot about the brain is consistent with it. For example, we know that mappings from one area to another as we move from the sense organs (or muscles), to the subcortex, and into the cortex tend to preserve topological relations between neurons. That’s of obvious value in the visual and motor systems. But there are subtleties. In the cortical visual system we have a so-called What-system and a so-called Where-system beyond the primary cortex. The What-system tracks location in space while the Where-system identifies objects. I assume that the Where-system has links to the hippocampus and I’d expect it can deal with both ego-centric and geocentric coordinates (this must be in the literature). But I’d think the What-system deals in object-centered coordinates. And so forth and so on.

In thinking about Freeman’s results (HERE and HERE), and others, I’ve coined a phrase: polyviscous connectivity. Thus I say that the cortical network as a whole exhibits polyviscous connectivity. What do I mean? Some connections are highly resistant to change, and thus have high viscosity. Others change quite readily, and have low viscosity. There is a literature on long-term (LTP) and short-term potentiation (STP) of neural connectivity that is certainly relevant here, but I’ve not looked at it in quite a while.

Consider Freeman’s results. He’s measuring neural activity with an 8 by 8 array of electrodes mounted on the cortical surface. They’re going to detect activity of neurons at varying levels of viscosity. Let’s a assume that the patterns of connectivity encoding odorants that rat already recognizes have a relatively high viscosity. Let’s further assume that the neurons most susceptible to learning new odorants have a relatively low viscosity.

Once they’ve formed a stable response to the new odorant, that will result in a new pattern of neural activity for the ensemble. But it is also going to change the patterns exhibited by already learned odorants even though the high-viscosity connections haven’t changed. The high viscosity connections maintain the overall integrity of the ensemble. In time, if the new odorant continues to be encountered, the connections registering it will increase in viscosity. So, polyviscous connectivity allows a structured connectionist physical system to maintain its overall integrity while adding new items to its repertoire.

More later.

* * * * *

Note: Looking around on my hard-drive I found an article which is about what I have called polyviscosity, though it doesn’t use that term:

Poonam Mishra and Rishikesh Narayanan, Stable continual learning through structured multiscale plasticity manifolds, Current Opinion in Neurobiology 2021, 70:51–63, https://doi.org/10.1016/j.conb.2021.07.009

Abstract: Biological plasticity is ubiquitous. How does the brain navigate this complex plasticity space, where any component can seemingly change, in adapting to an ever-changing environment? We build a systematic case that stable continuous learning is achieved by structured rules that enforce multiple, but not all, components to change together in specific directions. This rule-based low-dimensional plasticity manifold of permitted plasticity combinations emerges from cell type–specific molecular signaling and triggers cascading impacts that span multiple scales. These multiscale plasticity manifolds form the basis for behavioral learning and are dynamic entities that are altered by neuromodulation, metaplasticity, and pathology. We explore the strong links between heterogeneities, degeneracy, and plasticity manifolds and emphasize the need to incorporate plasticity manifolds into learning-theoretical frameworks and experimental designs.

* * * * *

Note: I’d previously been using the terms “hyperciscosity” or “hyperviscous”, and you’ll find them in my posts and notes on this topic going back to 2013 (when I wrote about From Associative Nets to the Fluid Mind). But I have reluctantly decided to coin a new term since “hyperviscosity” is already being used. 

* * * * *

Addendum, 8.13.22: On the Structured Physical System Hypothesis, see this post where I feature remarks by Rodney Brooks, Has the computer metaphor for the mind run out of steam? As the title suggests, Brooks is wondering whether or not it makes sense to think about nervous systems in terms of computation. Thus he wonders:

Is information processing the right metaphor there? Or are control theory and resonance and synchronization the right metaphor? We need different metaphors at different times, rather than just computation. Physical intuition that we probably have as we think about computation has served physicists well, until you get to the quantum world. When you get to the quantum world, that physical intuition about stuff and place gets in the way.

The mind as a polyviscous fluid

About a year ago I uploaded a post with a typically ungainly title, The structured physical system hypothesis (SPSH), Polyviscous connectivity [The brain as a physical system]. It’s that word, “polyviscous,” that’s got my present attention. Since then I’ve done a number of posts using that idea, whatever it is. This is another of those ideas.

So, viscosity. Honey is more viscous than, say, water. It flows more slowly, much more. What happens if you drop a lump of honey into a tumbler of water? It sinks to the bottom in a continuous lump and flattens out along the bottom. It will begin to diffuse into the water along the boundary, but I don’t know how long, if ever, it will take to mix completely. Now, put a stick down into the tumbler until it extends into the honey. Give is a stir or three, but no more. Now you’ll have gobs and threads of honey mixed in with water in a complex and somewhat irregular and ragged way. That’s a simple polyviscous fluid. It has regions of relatively high viscosity and other regions of relatively low viscosity. Now imagine a fluid with 5, 10, 27, 48, and more different levels of viscosity, from all but solid like cold tar through the wispiest whatever. Polyviscosity.

As the title of the year-old post indicates, I was thinking in terms of connectivity:

Thus I say that the cortical network as a whole exhibits polyviscous connectivity. What do I mean? Some connections are highly resistant to change, and thus have high viscosity. Others change quite readily, and have low viscosity.

OK. Now let’s shift our thinking just a bit and think of the mind as a polyviscous fluid. The mind, as the saying goes, is what the brain does. And that is very complex.

Imagine that you’re watching a movie, make it a Hong Kong martial arts movie. Your mind is entrained to the images on the screen. During a fight scene the level of mental viscosity is relatively how. The fight is over and the hero rests, contemplating the sunset, let’s say. The viscosity is somewhat higher.

Yet, while you’re entrained by the film, you’re not completely absorbed into. While the hero contemplates the sunset, you take a bit of popcorn. And maybe you were munching furiously during the fight. So, even as you were watching the film you slipped in some mental popcorn “frames” among the film frames. Very slippery, low viscosity.

When I wrote my book on music, Beethoven’s Anvil, I talked of the mind as neural weather. Thus (p. 72):

If the functional proclivities of a patch of neural tissue are not relevant for a current activity, those neurons will not be firing very often, but they will still generate some output. The only neuron that does not generate any output is a dead one. Neurons that are firing at low intensity one moment may well be recruited to more intense activity the next. As Walter Freeman has said, a low level of activity is still a means of participating in the evolving mental state.

The mind, in this view, is thus like the weather. The same environment can have very different kinds of weather. And while we find it natural to talk of weather systems as configurations of geography, temperature, humidity, air pressure etc., no overall mechanism regulates the weather. The weather is the result of many processes operating on different temporal and spatial scales.

At the global level and on a scale of millennia we have the long-term patterns governing the ebb and flow of glaciers which, in one commonly accepted theory, is a function of wobble and tilt in the earth’s spin axis and the shape of the earth’s orbit. At the global level and operating annually we have the succession of seasons, which is caused by the orientation of the earth with respect to the sun as it moves through the year. We can continue on, considering smaller and smaller scales until we consider the wind ripping through the twin towers of the World Trade Center or even the breeze coming in through your open window and blowing the papers off your desk.

Weather is regular enough that one can predict general patterns at scales of hours, days, and months, but not so regular that making such predictions is easy and routinely reliable. Above all, there is no central mechanism governing the weather. It just happens.

I develop that idea further in a couple of posts, The Mind is What the Brain Does, and Very Strange, and Neural Weather, an Informal Defense of Psychoanalytic Ideas.

So, neural weather, polyviscous fluid. Perhaps we’re getting somewhere. The mind IS what the brain does, and what the brain does is complex and varies along a wide range of time scales. The brain’s overall physical structure is relatively constant throughout life, barring injury and disease. But the connectivity changes over a variety of time scales from seconds through hours and days and even longer (think of cortical plasticity). There is much, perhaps most, millisecond to millisecond, activity that produces no synaptic change at all. A very fluid phenomenon, over multimer time scales.

Consciousness, reorganization and polyviscosity, Part 1: The link to Powers

The nature of consciousness is one of the big mysteries of contemporary thought. The best account of consciousness I know of is that offered by William Powers in Behavior: The Control of Perception (1973). That’s what this post is about. My objective is simple, to link Powers’s account of consciousness to the concept of polyviscosity that I offered about a week ago, The structured physical system hypothesis (SPSH), Polyviscous connectivity [The brain as a physical system]. Unfortunately, Powers’s concept, while basically simple, is simple only in the context of his overall model of mind, and that is not something that can readily be conveyed in a single blog post. Thus this post is mostly for my own benefit.

Powers’ model consists of two components: 1) a stack of servomechanisms – see the post In Memory of Bill Powers – regulating both perception and movement, and 2) a reorganizing system. The reorganizing system is external to the stack, but operates on it to achieve adaptive control, an idea he took from Norbert Wiener. Powers devoted “Chapter 14, Learning” to the subject (pp. 177-204). Reorganization is the mechanism through which Powers achieves learning.

Here’s an extensive passage that gets at the heart of the present matter (pp. 199-201):

To the reorganizing system, under these new hypotheses, the hierarchy of perceptual signals is itself the object of perception, and the recipient of arbitrary actions. This new arrangement, originally intended only as a means of keeping reorganization closer to the point, gives the model as a whole two completely different types of perceptions: one which is a representation of the external world, and the other which is a perception of perceiving. And we have given the system as a whole the ability to produce spontaneous acts apparently unrelated to external events or control considerations: truly arbitrary but still organized acts.

As nearly as I can tell short of satori, we are now talking about awareness and volition.

Awareness seems to have the same character whether one is being aware of his finger or of his faults, his present automobile or the one he wishes Detroit would build, the automobile’s hubcap or its environmental impact. Perception changes like a kaleidoscope, while that sense of being aware remains quite unchanged. Similarly, crooking a finger requires the same act of will as varying one’s bowling delivery “to see what will happen.” Volition has the arbitrary nature required of a test stimulus (or seems to) and seems the same whatever is being willed. But awareness is more interesting, somehow.

The mobility of awareness is striking. While one is carrying out a complex behavior like driving a car through to work, one’s awareness can focus on efforts or sensations or configurations of all sorts, the ones being controlled or the ones passing by in short skirts, or even turn to some system idling in the background, working over some other problem or musing over some past event or future plan. It seems that the behavioral hierarchy can proceed quite automatically, controlling its own perceptual signals at many orders, while awareness moves here and there inspecting the machinery but making no comments of its own. It merely experiences in a mute and contentless way, judging everything with respect to intrinsic reference levels, not learned goals.

This leads to a working definition of consciousness. Consciousness consists of perception (presence of neural currents in a perceptual pathway) and awareness (reception by the reorganizing system of duplicates of those signals, which are all alike wherever they come from). In effect, conscious experience always has a point of view which is determined partly by the nature of the learned perceptual functions involved, and partly by built-in, experience-independent criteria. Those systems whose perceptual signals are being monitored by the reorganizing system are operating in the conscious mode. Those which are operating without their perceptual signals being monitored are in the unconscious mode (or preconscious, a fine distinction of Freud’s which I think unnecessary).

This speculative picture has, I believe, some logical implications that are borne out by experience. One implication is that only systems in the conscious mode are subject either to volitional disturbance or reorganization. The first condition seems experientially self-evident: can you imagine willing an arbitrary act unconsciously? The second is less self-evident, but still intuitively right. Learning seems to require consciousness (at least learning anything of much consequence). Therapy almost certainly does. If there is anything on which most psychotherapists would agree, I think it would be the principle that change demands consciousness from the point of view that needs changing. Furthermore, I think that anyone who has acquired a skill to the point of automaticity would agree that being conscious of the details tends to disrupt (that, is, begin reorganization of) the behavior. In how many applications have we heard that the way to interrupt a habit like a typing error is to execute the behavior “on purpose”—that is, consciously identifying with the behaving system instead of sitting off in another system worrying about the terrible effects of having the habit? And does not “on purpose” mean in this case arbitrarily not for some higher goals but just to inspect the act, itself?

That, then, is consciousness as Powers conceives it. It is correlated with reorganization. If we are to reorganize a perception or action, we must be aware of it. The fact that we spend most of our lives in some state of consciousness implies that we are always learning or, perhaps, maintaining ourselves in a state of readiness to learn.

What has this to do with polyviscosity? Here I am thinking of neural connectivity. It is polyviscous in that some connections are highly resistant to change while others change readily. Reorganization, that is to say learning, requires that neural connectivity change. Connections of various levels of viscosity are likely to be intermingled in any given volume of cortical tissue.

Now, consider this passage from a 1988 paper by Fodor and Pylyshyn, Connectionism and Cognitive Architecture: A Critical Analysis (pp. 22-23):

Classical theories are able to accommodate these sorts of considerations because they assume architectures in which there is a functional distinction between memory and program. In a system such as a Turing machine, where the length of the tape is not fixed in advance, changes in the amount of available memory can be affected without changing the computational structure of the machine; viz by making more tape available. By contrast, in a finite state automaton or a Connectionist machine, adding to the memory (e.g. by adding units to a network) alters the connectivity relations among nodes and thus does affect the machine’s computational structure. Connectionist cognitive architectures cannot, by their very nature, support an expandable memory, so they cannot support productive cognitive capacities. The long and short is that if productivity arguments are sound, then they show that the architecture of the mind can’t be Connectionist. Connectionists have, by and large, acknowledged this; so they are forced to reject productivity arguments.

Physically, the nervous system appears to be connectionist in character. And so adding new items to the system is physically problematic. That’s the problem that is solved by polyviscous connectivity – see my post, Physical constraints on computing, process and memory, Part 1 [LeCun] (Note: in that post I use the term “hyperviscous” rather than “polyviscous”). Some connections must remain stable while others change. The stable connections maintain the overall structural integrity of the network while the changing connections introduce new items into that structure.

Here's a recent article that’s relevant, though it doesn’t use the term “polyviscious”: Poonam Mishra and Rishikesh Narayanan, Stable continual learning through structured multiscale plasticity manifolds, Current Opinion in Neurobiology 2021, 70:51–63, https://doi.org/10.1016/j.conb.2021.07.009

Abstract: Biological plasticity is ubiquitous. How does the brain navigate this complex plasticity space, where any component can seemingly change, in adapting to an ever-changing environment? We build a systematic case that stable continuous learning is achieved by structured rules that enforce multiple, but not all, components to change together in specific directions. This rule-based low-dimensional plasticity manifold of permitted plasticity combinations emerges from cell type–specific molecular signaling and triggers cascading impacts that span multiple scales. These multiscale plasticity manifolds form the basis for behavioral learning and are dynamic entities that are altered by neuromodulation, metaplasticity, and pathology. We explore the strong links between heterogeneities, degeneracy, and plasticity manifolds and emphasize the need to incorporate plasticity manifolds into learning-theoretical frameworks and experimental designs.