Emergence

𝗘𝗺𝗲𝗿𝗴𝗲𝗻𝗰𝗲, 𝘄𝗵𝗮𝘁 𝗶𝗻 𝘁𝗵𝗲 𝘄𝗼𝗿𝗹𝗱 𝗶𝘀 𝗶𝘁??
It would be cool to have a discussion on this one, who knows @seanmcarroll might even join??? https://t.co/VCuoabW8zf pic.twitter.com/Uhb9gD3Fz8

— Luiz Pessoa (@PessoaBrain) September 18, 2024

Control Theory, Prompt Engineering, and GPT [stories]

As a student of the work of William Powers I have long standing interest in control theory. It’s central to my conception of how the mind works. David Hays made it central his model of cognition, which is at the foundation of my early work (e.g. Cognitive Networks and Literary Semantics) and we incorporated it into our account of the brain (Principles and Development of Natural Intelligence). It is thus with some interest that I watched the following video:

Note that they develop the concept of feedback through the idea of the governor (for an engine) as an example at roughly 7:50.

Here's the YouTube copy:

These two scientists have mapped out the insides or “reachable space” of a language model using control theory, what they discovered was extremely surprising. [...]

Aman Bhargava from Caltech and Cameron Witkowski from the University of Toronto to discuss their groundbreaking paper, “What’s the Magic Word? A Control Theory of LLM Prompting.” (the main theorem on self-attention controllability was developed in collaboration with Dr. Shi-Zhuo Looi from Caltech).

They frame LLM systems as discrete stochastic dynamical systems. This means they look at LLMs in a structured way, similar to how we analyze control systems in engineering. They explore the “reachable set” of outputs for an LLM. Essentially, this is the range of possible outputs the model can generate from a given starting point when influenced by different prompts. The research highlights that prompt engineering, or optimizing the input tokens, can significantly influence LLM outputs. They show that even short prompts can drastically alter the likelihood of specific outputs. Aman and Cameron’s work might be a boon for understanding and improving LLMs. They suggest that a deeper exploration of control theory concepts could lead to more reliable and capable language models.

Here’s their paper: What's the Magic Word? A Control Theory of LLM Prompting.

More recently Behnam Mohammadi at Carnegie Mellon has written a paper which is somewhat different in formulation, but has a similar interest in the range over which an LLM can be controlled: Creativity Has Left the Chat: The Price of Debiasing Language Models. That paper has a passage that’s very interesting in a control theory context:

Experiment 2 investigates the semantic diversity of the models’ outputs by examining their ability to recite a historical fact about Grace Hopper in various ways. The generated outputs are encoded into sentence embeddings and visualized using dimensionality reduction techniques. The results reveal that the aligned model’s outputs form distinct clusters, suggesting that the model expresses the information in a limited number of ways. In contrast, the base model’s embeddings are more scattered and spread out, indicating a higher level of semantic diversity in the generated outputs. [...]

An intriguing property of the aligned model’s generation clusters in Experiment 2 is that they exhibit behavior similar to attractor states in dynamical systems. We demonstrate this by intentionally perturbing the model’s generation trajectory, effectively nudging it away from its usual output distribution. Surprisingly, the aligned model gracefully finds its way back to its own attractor state and in-distribution response. The presence of these attractor states in the aligned model’s output space is a phenomenon related to the concept of mode collapse in reinforcement learning, where the model overoptimizes for certain outputs, limiting its exploration of alternative solutions.

With these papers in mind I decided to redo some of my early story variation experiments using a prompt with slightly different wording. As you may know, these experiments involve a two-part prompt: 1) a story, and 2) and instruction use the given story as the basis of a new story. In the original experiments I formulated the instruction like this:

I am going to tell you a story about princess Aurora. I want you to tell the same story, but change princess Aurora to a Giant Chocolate Milkshake. Make any other changes you wish.

In the new experiments, I stated the instruction like this:

I’m going to give you a short story. I want you repeat that story, but with a difference. Replace Aurora with a giant chocolate milkshake. Make any other changes you wish in order preserve coherence.

The difference is relatively minor, but the new prompt nudges the instruction in the direction of control theory, at least superficially. Think of the specified change as a perturbance. We can then think of the further changes introduced by ChatGPT as moving ChatGPT “back to its own attractor state,” which we can think of as something like story coherence.

Below the asterisks I give two examples. The results are pretty much the same as in the earlier experiments. ChatGPT makes the change I explicitly requested, but makes other changes as well, changes that make the story consistent with the change I’d requested. My prompts are in bold face while ChatGPT's responses are in plain face.

* * * * *

Internal feedback in the cortical perception-action loop enables fast and accurate behavior

Jing Shuang (Lisa) Lia, Anish A. Sarmaa, Terrence J. Sejnowskic, and John C. Doyle, Internal feedback in the cortical perception-action loop enables fast and accurate behavior, PNAS, September 22, 2023 120 (39) e2300445120 https://doi.org/10.1073/pnas.2300445120, asXiv: https://arxiv.org/abs/2211.05922

Significance

Internal feedback projections—signals flowing from motor areas or late sensory processing regions back to early sensory processing regions such as primary visual and auditory areas—are ubiquitous in the sensorimotor nervous system and are as or more numerous than feedforward projections. However, the function of internal feedback is poorly understood, particularly in the context of task performance. We leverage control theory and simple models to demonstrate that internal feedback facilitates good task performance when there are communication limitations such as internal time delays and speed–accuracy trade-offs, which motivate compensatory feedback signals to counter self-generated and predictable movements. Control theory explains why motor-related signals are found throughout the sensory cortex and why the motor cortex is dominated by internal dynamics.

Abstract

Animals move smoothly and reliably in unpredictable environments. Models of sensorimotor control, drawing on control theory, have assumed that sensory information from the environment leads to actions, which then act back on the environment, creating a single, unidirectional perception–action loop. However, the sensorimotor loop contains internal delays in sensory and motor pathways, which can lead to unstable control. We show here that these delays can be compensated by internal feedback signals that flow backward, from motor toward sensory areas. This internal feedback is ubiquitous in neural sensorimotor systems, and we show how internal feedback compensates internal delays. This is accomplished by filtering out self-generated and other predictable changes so that unpredicted, actionable information can be rapidly transmitted toward action by the fastest components, effectively compressing the sensory input to more efficiently use feedforward pathways: Tracts of fast, giant neurons necessarily convey less accurate signals than tracts with many smaller neurons, but they are crucial for fast and accurate behavior. We use a mathematically tractable control model to show that internal feedback has an indispensable role in achieving state estimation, localization of function (how different parts of the cortex control different parts of the body), and attention, all of which are crucial for effective sensorimotor control. This control model can explain anatomical, physiological, and behavioral observations, including motor signals in the visual cortex, heterogeneous kinetics of sensory receptors, and the presence of giant cells in the cortex of humans as well as internal feedback patterns and unexplained heterogeneity in neural systems.

Hossenfelder on chaos control [e.g. teaching robots to walk, nuclear fusion plasmas]

You might want to look at this post from February: Using AI (reinforcement learning) to train a magnetic controller for tokamak plasmas.

Consciousness, reorganization and polyviscosity, Part 1: The link to Powers

The nature of consciousness is one of the big mysteries of contemporary thought. The best account of consciousness I know of is that offered by William Powers in Behavior: The Control of Perception (1973). That’s what this post is about. My objective is simple, to link Powers’s account of consciousness to the concept of polyviscosity that I offered about a week ago, The structured physical system hypothesis (SPSH), Polyviscous connectivity [The brain as a physical system]. Unfortunately, Powers’s concept, while basically simple, is simple only in the context of his overall model of mind, and that is not something that can readily be conveyed in a single blog post. Thus this post is mostly for my own benefit.

Powers’ model consists of two components: 1) a stack of servomechanisms – see the post In Memory of Bill Powers – regulating both perception and movement, and 2) a reorganizing system. The reorganizing system is external to the stack, but operates on it to achieve adaptive control, an idea he took from Norbert Wiener. Powers devoted “Chapter 14, Learning” to the subject (pp. 177-204). Reorganization is the mechanism through which Powers achieves learning.

Here’s an extensive passage that gets at the heart of the present matter (pp. 199-201):

To the reorganizing system, under these new hypotheses, the hierarchy of perceptual signals is itself the object of perception, and the recipient of arbitrary actions. This new arrangement, originally intended only as a means of keeping reorganization closer to the point, gives the model as a whole two completely different types of perceptions: one which is a representation of the external world, and the other which is a perception of perceiving. And we have given the system as a whole the ability to produce spontaneous acts apparently unrelated to external events or control considerations: truly arbitrary but still organized acts.

As nearly as I can tell short of satori, we are now talking about awareness and volition.

Awareness seems to have the same character whether one is being aware of his finger or of his faults, his present automobile or the one he wishes Detroit would build, the automobile’s hubcap or its environmental impact. Perception changes like a kaleidoscope, while that sense of being aware remains quite unchanged. Similarly, crooking a finger requires the same act of will as varying one’s bowling delivery “to see what will happen.” Volition has the arbitrary nature required of a test stimulus (or seems to) and seems the same whatever is being willed. But awareness is more interesting, somehow.

The mobility of awareness is striking. While one is carrying out a complex behavior like driving a car through to work, one’s awareness can focus on efforts or sensations or configurations of all sorts, the ones being controlled or the ones passing by in short skirts, or even turn to some system idling in the background, working over some other problem or musing over some past event or future plan. It seems that the behavioral hierarchy can proceed quite automatically, controlling its own perceptual signals at many orders, while awareness moves here and there inspecting the machinery but making no comments of its own. It merely experiences in a mute and contentless way, judging everything with respect to intrinsic reference levels, not learned goals.

This leads to a working definition of consciousness. Consciousness consists of perception (presence of neural currents in a perceptual pathway) and awareness (reception by the reorganizing system of duplicates of those signals, which are all alike wherever they come from). In effect, conscious experience always has a point of view which is determined partly by the nature of the learned perceptual functions involved, and partly by built-in, experience-independent criteria. Those systems whose perceptual signals are being monitored by the reorganizing system are operating in the conscious mode. Those which are operating without their perceptual signals being monitored are in the unconscious mode (or preconscious, a fine distinction of Freud’s which I think unnecessary).

This speculative picture has, I believe, some logical implications that are borne out by experience. One implication is that only systems in the conscious mode are subject either to volitional disturbance or reorganization. The first condition seems experientially self-evident: can you imagine willing an arbitrary act unconsciously? The second is less self-evident, but still intuitively right. Learning seems to require consciousness (at least learning anything of much consequence). Therapy almost certainly does. If there is anything on which most psychotherapists would agree, I think it would be the principle that change demands consciousness from the point of view that needs changing. Furthermore, I think that anyone who has acquired a skill to the point of automaticity would agree that being conscious of the details tends to disrupt (that, is, begin reorganization of) the behavior. In how many applications have we heard that the way to interrupt a habit like a typing error is to execute the behavior “on purpose”—that is, consciously identifying with the behaving system instead of sitting off in another system worrying about the terrible effects of having the habit? And does not “on purpose” mean in this case arbitrarily not for some higher goals but just to inspect the act, itself?

That, then, is consciousness as Powers conceives it. It is correlated with reorganization. If we are to reorganize a perception or action, we must be aware of it. The fact that we spend most of our lives in some state of consciousness implies that we are always learning or, perhaps, maintaining ourselves in a state of readiness to learn.

What has this to do with polyviscosity? Here I am thinking of neural connectivity. It is polyviscous in that some connections are highly resistant to change while others change readily. Reorganization, that is to say learning, requires that neural connectivity change. Connections of various levels of viscosity are likely to be intermingled in any given volume of cortical tissue.

Now, consider this passage from a 1988 paper by Fodor and Pylyshyn, Connectionism and Cognitive Architecture: A Critical Analysis (pp. 22-23):

Classical theories are able to accommodate these sorts of considerations because they assume architectures in which there is a functional distinction between memory and program. In a system such as a Turing machine, where the length of the tape is not fixed in advance, changes in the amount of available memory can be affected without changing the computational structure of the machine; viz by making more tape available. By contrast, in a finite state automaton or a Connectionist machine, adding to the memory (e.g. by adding units to a network) alters the connectivity relations among nodes and thus does affect the machine’s computational structure. Connectionist cognitive architectures cannot, by their very nature, support an expandable memory, so they cannot support productive cognitive capacities. The long and short is that if productivity arguments are sound, then they show that the architecture of the mind can’t be Connectionist. Connectionists have, by and large, acknowledged this; so they are forced to reject productivity arguments.

Physically, the nervous system appears to be connectionist in character. And so adding new items to the system is physically problematic. That’s the problem that is solved by polyviscous connectivity – see my post, Physical constraints on computing, process and memory, Part 1 [LeCun] (Note: in that post I use the term “hyperviscous” rather than “polyviscous”). Some connections must remain stable while others change. The stable connections maintain the overall structural integrity of the network while the changing connections introduce new items into that structure.

Here's a recent article that’s relevant, though it doesn’t use the term “polyviscious”: Poonam Mishra and Rishikesh Narayanan, Stable continual learning through structured multiscale plasticity manifolds, Current Opinion in Neurobiology 2021, 70:51–63, https://doi.org/10.1016/j.conb.2021.07.009

Abstract: Biological plasticity is ubiquitous. How does the brain navigate this complex plasticity space, where any component can seemingly change, in adapting to an ever-changing environment? We build a systematic case that stable continuous learning is achieved by structured rules that enforce multiple, but not all, components to change together in specific directions. This rule-based low-dimensional plasticity manifold of permitted plasticity combinations emerges from cell type–specific molecular signaling and triggers cascading impacts that span multiple scales. These multiscale plasticity manifolds form the basis for behavioral learning and are dynamic entities that are altered by neuromodulation, metaplasticity, and pathology. We explore the strong links between heterogeneities, degeneracy, and plasticity manifolds and emphasize the need to incorporate plasticity manifolds into learning-theoretical frameworks and experimental designs.

Neural Recognizers: Some [old] notes based on a TV tube metaphor [perceptual contact with the world]

Yet another bump can't hurt. Why? Because yesterday I saw this tweet by Kevin Mitchell: “A useful perspective shift is to think of a neuron (or brain area) as actively monitoring its inputs as opposed to being passively driven by them.” [5.17.22]

Another bump to the top can't hurt.  [Sept 2021]

I'm bumping this to the top of the queue because GPT-3. I'm reconfiguring and restructuring like crazy. More later.

Introduction: Raw Notes

A fair number of my posts here at New Savanna are edited from my personal intellectual notes. In this post the notes are unedited. This is an idea that dates back to my graduate school days in English at SUNY Buffalo. Since I keep my notes in Courier – a font that harks back to the days of manual typewriters – I’ve decided to retain that font for these posts and to drop justification.

Since these notes are “raw” you’re pretty much on your own. Sorry and good luck.

* * * * *

Diagrams from D'Arcy Thompson, On Growth and Form.

1.26.2002 – 1.27.2002

This is the latest version of an idea I first explored at Buffalo back in the late 1970s. It was jointly inspired by William Powers’ notion of a zero reference level at the top of his servo stack and by D’Arcy Thompson (see diagrams above). I’ve transcribed some of those notes into the next section. A version of this appeared in the paper DGH (David Hays) and I wrote on natural intelligence, where we talked in terms of Pribram’s neural holography and Spinelli’s OCCAM model for the cortical column:

• W. Benzon and D. Hays. Principles and Development of Natural Intelligence. Journal of Social and Biological Structures 11, 293 - 322, 1988.
• Powers, W.T. (1973). Behavior: The Control of Perception. Chicago: Aldine.
• Pribram, K. H. (1971). Languages of the Brain. Englewood Cliffs, New Jersey: Prentice-Hall.
• Spinelli, D. N. (1970). Occam, a content addressable memory model for the brain. In (K. H. Pribram & D. Broadbent, Eds): The Biology of Memory. New York: Academic Press, pp. 293-306.

“TV Tube Recognizer”

2.13.1979

Imagine a TV screen with a circle painted on it and with controls which allow you to operate on and manipulate the projection system in various useful ways. We’re going to use this to conduct an active analysis of the input to the screen.

Assume that the object to be analyzed is projected onto the screen in such a way that its largest dimension doesn’t extend beyond the circle painted on it. The analysis consists of twiddling the [control] dials until the area between the outer border of the object and the inner border of the circle is as small as possible. That is “minimize area between object and circle” is the reference signal for this servo-mechanical procedure, while “twiddle the dials” is the output function. Think of those knobs as operating on the coordinates in Thompson's illustrations above. (Notice that we are not operating on the input signal to the TV screen.)

Active Analysis

One thing we might do by dial twiddling is to operate on the coordinate system of the projection (I’m thinking here of D’Arcy Thompson’s grids whereby a bass on one coordinate grid becomes a flounder when projected onto another different grid.) Thus if the input is a vertical ellipse a horizontal stretch would lower the area between the ellipse and circle [painted on the TV screen]. One could bend the axes or distort them in various ways. Or, how about allowing the system to partition the screen in various ways and then make local alterations in the coordinate system within the partition.

Endless possibilities.

It doesn’t make much difference what [we do], the point is that the system have some way of operating on the image on the screen (without messing around with the input to the screen ...). The settings on the dials when the area between the projected object and the circle is at a minimum then constitutes the analysis of the object. To the extent that objects differ, the differences in the dial settings differentiate between objects (we are limited, of course, by the resolving power of the system). The settings which are best for a buzzard won’t be best for a flounder, nor a pine tree, nor a start, etc.

* * * * *

[1.26.2002]

The most obvious difficulty with this story is that it depends on someone observing the TV screen and twiddling the control knobs. We want to eliminate that someone so that the system can achieve the desired result itself.

The obvious way to do this is to call on the self-organizing capacity of cortical neural tissue. That tissue is itself the TV screen and control knobs while the appropriate subcortical thalamic nucleus is the source of input to the recognizer. The reference level is alpha oscillation, reflecting the observation that alpha energy is high when the stimulus is familiar and low when it is not. Unfamiliar input disturbs the oscillation and the recognizer seeks to restore oscillation by temporarily altering the properties (twiddling the dials) of the input array (thalamic nucleus).

Neural Recognizer

The neocortex is conceived as a patchwork of pattern recognizers; each is a sheet of cortical columns. Neighboring columns are mutually inhibitory, as in OCCAM (Spinelli 1970). A high level of output from one column will suppress output in its neighbors. The patterns are recognized in the primary input (input array) to a given recognizer. Let us assume a recognizer whose primary input is subcortical and let us set aside consideration of other inputs. The recognizer also generates primary output, which goes to the subcortical source of primary input. The computing capacity of a recognizer is far greater than that of its primary input.

The base state of such a recognizer occurs when the input is random (of a certain unspecified quality). In this base state the columns in the array oscillate – given a rather old notion that high alpha means low arousal, I’ve been thinking this would be at alpha; but, perhaps in view of Freeman’s work, I should revise this in favor of intrinsic chaos. The recognizer acts to maintain this base state under all conditions. When there is a non-random perceptual signal that signal will necessarily perturb the array so that it no longer oscillates smoothly. The array proceeds to form an impression of that input by sending (inhibitory) signals to the primary input. Some cortical columns will necessarily play a stronger role in this process than others. Eventually the recognizer will find some combination of outputs (modifying the properties of the input array) that restores randomness, and hence smooth oscillation. When this point is reached, the impression has been formed. This impression is of the input. In common parlance, we might want to say it represents that input.

Now some process must take place in the array so that the current perturbation can either be habituated into the background or an impression be taken, that is, can become part of the permanent repertoire of the recognizer. This latter, presumably, involves Hebbian learning and is triggered by reinforcement. In the manner of Spinelli’s OCCAM, the recognizer has many such impressions stored in its synaptic weights. A perceptual signal is presented across the entire array and, if it is of a kind that has already made an impression on the array, that impression will be evoked from the array and restore the recognizer to periodic oscillation. If it is of a kind that has not yet made an impression, then a new impression must be made.

Now, in fact, each recognizer has a variety of secondary inputs coming from other recognizers and it generates secondary outputs to them. All of them are attempting to account for their input simultaneously; through their secondary inputs and outputs the recognizers “help” one another out. Further it has inputs from subcortical nuclei which send neuromodulators to the array and it sends outputs to those nuclei which indicate its state of operation. The neuromodulators cause the recognizer to switch between its different operating modes.

I see these operating modes as follows:

Baseline: There is no perceptual load. The array is oscillating at alpha (chaos?).

Tracking: Perceptual input is accounted for. The array has recognized the input and is oscillating at alpha (chaos?).

Matching: The array is under a perceptual load and is attempting to match the input using its current set of impressions. EEG: “desynchronized,” gamma?

Forming (an impression): The array is under a perceptual load, but is unable to match it from its current impression repertoire. It is now forming a new impression. Obviously one critical aspect of the recognizer’s operation is switching from an unsuccessful matching operation to forming. EEG: “desynchronized,” gamma?

Habituating: The array is under a perceptual load, a new impression has been formed, and it has been assimilated into the background.

Fixing: A new impression has been formed. It must now become part of the permanent repertoire of impressions. This is the beginning of LTP. EEG: high alpha?

Group Expressive Behavior

We could apply this line of thought to group expressive behavior where the members of the group are regarded as oscillators coupled to one another through mutual perception and coordinated action. The simplest such behavior would be moving together, or clapping, to an isochronous pulse.

Assume a group moving to an isochronous pulse. Further assume that this activity is cortically controlled. Now, imagine that various members of the group are driven by subcortical impulses to inflect their movement in noticeable ways. These inflections will be transmitted to others through the coupling. Adjustments made to accommodate these inflections become, in effect, the group’s impression of those subcortical impulses.

This needs to be worked through rather more carefully, which will certainly change things a bit. But what I’m driving at is that these group impressions will become the stuff of culture. Here’s where we get memes and performance trajectories [as those are defined in Beethoven’s Anvil].

Using AI (reinforcement learning) to train a magnetic controller for tokamak plasmas

Degrave, J., Felici, F., Buchli, J. et al. Magnetic control of tokamak plasmas through deep reinforcement learning. Nature 602, 414–419 (2022). https://doi.org/10.1038/s41586-021-04301-9

Abstract

Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control using magnetic actuator coils, further complicated by the diverse requirements across a wide range of plasma configurations. In this work, we introduce a previously undescribed architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils. This architecture meets control objectives specified at a high level, at the same time satisfying physical and operational constraints. This approach has unprecedented flexibility and generality in problem specification and yields a notable reduction in design effort to produce new plasma configurations. We successfully produce and control a diverse set of plasma configurations on the Tokamak à Configuration Variable including elongated, conventional shapes, as well as advanced configurations, such as negative triangularity and ‘snowflake’ configurations. Our approach achieves accurate tracking of the location, current and shape for these configurations. We also demonstrate sustained ‘droplets’ on TCV, in which two separate plasmas are maintained simultaneously within the vessel. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied.

From the beginning of the article:

Tokamaks are torus-shaped devices for nuclear fusion research and are a leading candidate for the generation of sustainable electric power. A main direction of research is to study the effects of shaping the distribution of the plasma into different configurations3,4,5 to optimize the stability, confinement and energy exhaust, and, in particular, to inform the first burning-plasma experiment, ITER. Confining each configuration within the tokamak requires designing a feedback controller that can manipulate the magnetic field6 through precise control of several coils that are magnetically coupled to the plasma to achieve the desired plasma current, position and shape, a problem known as the tokamak magnetic control problem.

The conventional approach to this time-varying, non-linear, multivariate control problem is to first solve an inverse problem to precompute a set of feedforward coil currents and voltages7,8. Then, a set of independent, single-input single-output PID controllers is designed to stabilize the plasma vertical position and control the radial position and plasma current, all of which must be designed to not mutually interfere6. Most control architectures are further augmented by an outer control loop for the plasma shape, which involves implementing a real-time estimate of the plasma equilibrium9,10 to modulate the feedforward coil currents8. The controllers are designed on the basis of linearized model dynamics, and gain scheduling is required to track time-varying control targets. Although these controllers are usually effective, they require substantial engineering effort, design effort and expertise whenever the target plasma configuration is changed, together with complex, real-time calculations for equilibrium estimation.

A radically new approach to controller design is made possible by using reinforcement learning (RL) to generate non-linear feedback controllers. The RL approach, already used successfully in several challenging applications in other domains11,12,13, enables intuitive setting of performance objectives, shifting the focus towards what should be achieved, rather than how. Furthermore, RL greatly simplifies the control system. A single computationally inexpensive controller replaces the nested control architecture, and an internalized state reconstruction removes the requirement for independent equilibrium reconstruction. These combined benefits reduce the controller development cycle and accelerate the study of alternative plasma configurations. Indeed, artificial intelligence has recently been identified as a ‘Priority Research Opportunity’ for fusion control14, building on demonstrated successes in reconstructing plasma-shape parameters15,16, accelerating simulations using surrogate models17,18 and detecting impending plasma disruptions19. RL has not, however, been used for magnetic controller design, which is challenging due to high-dimensional measurements and actuation, long time horizons, rapid instability growth rates and the need to infer the plasma shape through indirect measurements.

H/t Tyler Cowen.

Neuroscience and control theory

I am really digging this mashup of network neuroscience and control theory https://t.co/d60Ly8cizC

— Patrick Mineault (@patrickmineault) January 30, 2022

Here's a great @neuromatch talk from @DaniSBassett linking control-theoretic control and cognitive control, hosted by @crozSciTech that contextualizes the paper https://t.co/rtjkCwcCYx

— Patrick Mineault (@patrickmineault) January 30, 2022

Ted Cloak: The Wheel and Cultural Evolution

Bumping this to the top of the queue as a reminder, Cloak was there first. And that 1968 study of the spoked wheel is very interesting. I wish there were more work like it.

Before there was Dawkins, there was Ted Cloak. His 1968 study (PDF) of the cultural evolution of the spoked wooden wheel and documentation of the manufacture of same (with photos) is a classic document, one not widely enough known. Here's a website he's put up setting out his ideas. Pay particular attention to his use of the work of William Powers (includes a number of useful videos of Power's Model).

* * * * *

Addendum: Note the following passage from David G. Hays, The Evolution of Technology, Chapter 5, “Politics, Cognition, and Personality”:

In reading to prepare to write this book, I have learned that the wheel was used for ritual over many years before it was put to use in war and, still later, work.

Behavior: The Control of Perception – Bill Powers rediscovered, again!

I decided to cruise by Slate Star Codex and saw a post with the title, What Intellectual Progress did I Make in the 2010S? Sounds ambitious, thought I do myself. [Hmmm...should I write such a post? Umm, err, I think not.*] This, the second paragraph, stopped me dead in my tracks:

I think the single most important thing I discovered this decade (due to a random comment in the SSC subreddit!) was the predictive coding theory of the brain. I started groping towards it (without knowing what I was looking for) in Mysticism And Pattern-Matching, reported the exact moment when I found it in It’s Bayes All The Way Up, and finally got a decent understanding of it after reading Surfing Uncertainty. At the same time, thanks to some other helpful tips from other rationalists, I discovered Behavior: The Control Of Perception, and with some help from Vaniver and a few other people was able to realize how these two overarching theories were basically the same. Discovering this area of research may be the best thing that happened to me the second half of this decade (sorry, everyone I dated, you were pretty good too).

It’s that reference to Behavior: The Control Of Perception, that caught my eye. It was published in 1973 by William Powers, positively reviewed a couple years later in Science, and had been central to the work that I’d done with David Hays in his computational linguistics research group at SUNY Buffalo in the mid-1970s. Back when I listed the ten books that had most influenced my thinking, that was one of them. [Note: I’ve got a number of posts about or at least mentioning Powers.]

But, for some reason, Powers’s thought never really caught on – though I note, in passing, that Ted Cloak, another forgotten thinker, also found his work valuable. By the mid-1980s or so a small group of thinkers had coalesced around him and began holding annual meetings. I never attended any of those, though I joined a mailing list for the group, and I presented with them at some meetings of the American Society for Cybernetics. Powers died in 2013, but I assume that group still meets.

Given that Powers has had relatively little influence, are those of us who HAVE been influenced by him wrong? I suppose that I’m not exactly in a position to offer up a defense, but I do find it interesting that Scott Alexander, proprietor of Slate Star Codex, should put his book front and center in his review of his intellectual decade. He prefaces his review Powers' book with a disclaimer (his italics): “Epistemic status: I only partly understood this book and am trying to review it anyway as best I can.” In the course of his review he expresses major doubts about aspects of Powers’s model. And the review ends in a string of questions without answers:

How useful is this book? I guess that depends on how metaphorical you want to be. Is the brain a control system? I don’t know. Are police a control system trying to control crime? Are police a “response” to the “stimulus” of crime? Is a stimulus-response pairing a control system controlling for the quantity of always making sure the stimulus has the response? I think it’s interesting and helpful to think of some psychological functions with these metaphors. But I’m not sure where to go from there.

That’s a mountain of doubt. And yet somehow that abstract, elegant, and elusive book moved the mountain.

I can understand Alexander’s reservations. When Hays, his other students, and I worked it over, we discarded and/or reworked major portions of the model. But despite that we kept the overall outline, including, believe it or not, his comments about consciousness and reorganization. In a way, especially those.

It’s that overall outline – though outline is an inadequate word, gestalt is perhaps better – that made it so attractive for us. It was and is a biologically grounded model of the mind based on classical control theory engineering – feedback loops, etc. It also assigned a coherent function to consciousness, Powers called it reorganization, but to appreciate the weight and valence of that them, you have to think about his whole model.

That it was based in cybernetics is perhaps why it never found favor. But the time Powers had published the book the so-called cognitive revolution was going into over drive. All the cool kids were thinking about digital computers, while Powers was thinking analog. Of course, we knew all that. Hays had been a first generation researcher in machine translation and, as such, one of the founders of computational linguistics. Our research group was ABOUT computational linguistics. But we, like so many others, were reaching for the mind. And we had decided/realized that computation alone wouldn’t get us there. So we took the gestalt that Powers had created and opened it up to include language, symbolic computation, in a more realistic way [see David Hays, Cognitive Structures, HRAF Press, 1981]. Powers kept us grounded in biology, we opened him up to language. That’s the line we took.

* * * * *

Our Rage for Order and Coherence

A quickie, just to get it out there so I can see it (from January 2015).

I have long believed that a need for order, coherence, patterns, is intrinsic to being human. I suppose I can date this belief to the mid-1970s when I read William Powers, Behavior: The Control of Perception (1973). Under the topic of reorganization (more conventionally, learning) Powers talked of intrinsic reference levels for the (proper) construction of control systems (which I’ve recently recounted in Cultural Beings, the Ontology of Culture, and a Return to Books and Blues). Hays and I included Powers’ notion in our paper, Principles and Structures of Natural Intelligence (1988), and I associated it with Csikszentmihalyi’s notion of flow in Culture as an Evolutionary Arena (1996).

That’s what’s behind ‘beauty’ or whatever it is that works of art are said to have, and for elegance in scientific theory and mathematical proof. And it is, I believe, behind cultural evolution. We must make order of the world; that is the ‘other side’ of anxiety.

But where’d it come from? In our paper on natural intelligence Hays and I associated the need for coherence with the process of reorganization (following Powers) and we linked that to the emergence of vertebrates. That makes it very old indeed. What seems to have happened with humankind is that this rage for order has been set free from merely biological imperatives, has been set from from the tyranny of the present, so that it now ranges over the whole of space and time.

How’d that happen? And when? There is the emergence of human language. That sets cognition free of the present by allowing us to index the past, and thus recall it, and allows us to index the cognitive system itself, thus creating a need to order that index (ontology).

But there is also death (see Cultural Beings Evolving in the Mesh). It is one thing to mourn one’s fellows, as animals do. It is another think to know that you will, at some point, die. How is it that we came to know that? What are the cognitive presuppositions of such knowledge?

Explicit grave sites along with grave goods appear in the human record about 100,000 years ago. I take that as evidence that people have come to know that they will die. It may be the dead that are buried, but it is done for the benefit of the living. It is a way of keeping the living whole.

Somewhere in the nexus of language and death, that is where we’ll find our rage for order. That is what drives us.

Addendum: The order, to be meaningful, must be collective. Language can exist only in a group. Burial rites are meaningful only to a group. This sets up a peculiar dialectic in which “cultural beings” are utterly dependent on the people who participate in them even as the participants are dependent on those same cultural beings. It is through those cultural beings that the individuals experience the order and coherence that is so important to them.

 

And the sharing of experience is itself a source of pleasure. When experience alone grief, for example, is merely grief. Sharing it transforms it, not into pleasure in any direct sense of the world, but into something that is fulfilling. It’s the sharing that works the transformation. That’s why sad music can make us feel good; that’s why tragedy can be so powerful, so cathartic.

* * * * *

Photographs are of the Jersey City and Harsimus Cemetary and were taken in March 2009.

Trumposaurus Rex @ 3QD – Toward a cybernetic interpretation

That’s what my current piece at 3 Quarks Daily is about, Feed Me Donald! – Trump, Musk, The Internet, And Monsters From The Id. I start with Elon Musk’s conversation with Joe Rogan. For example:

Elon Musk: A company is essentially a cybernetic collective of people and machines. That’s what a company is. There’s different levels of complexity in the ways these companies are formed. ...

Joe Rogan: Humans and electronics all interfacing, and constantly now, constantly connected.

And then on to my buddy David’s report of Trump’s performance at AIPAC (American Israel Public Affairs Committee) in 2016:

These were not Trump supporters. It is AIPAC’s tradition to cheer for good rousing lines. Standing ovations are not endorsements. On the other hand, you can see how the cameras may have shown us being turned into a Trump mob. And maybe we were. Maybe you would have stood and cheered, too. Yet at the very moment I became part of the mob, I had this sudden schizoid flash of rational clear sight into Trump. I saw, I mean really saw, the Trumposaur in its naked, primitive state.

And then I toss some lines from Little Shop of Horrors, a 1986 rock horror musical comedy. These lines are being sung by a blood-drinking carnivorous plant with luscious lips and no eyes:

Feed me! Feed me! Feed me!

Feed me, Seymour

Feed me all night long

That's right, boy

You can do it

Feed me, Seymour

Feed me all night long

‘Cause if you feed me, Seymour

I can grow up big and strong

Except that you’re supposed to hear it as “Feed me Donald” and the plant is Trump’s smart phone, the one he uses to issue his tweets, the personal ones that go zinging into the cyberverse at three in the morning where they end up on Fox News and scoot under the skirt of The Gray Lady (aka The New York Times), giving her a vile thrill she experiences as $$$$.

All night, baby, all night long. Just you and me. So good! So good!

And if Trump feeds his smart phone, what’s he get from it? Why satisfaction of course. Just what gives him satisfaction, only he knows, and he probably doesn’t know it all that well, though he’s been feeding it all its life. Word is that he’s got an endless need for praise and approval and he does whatever he can to get it. For the purposes of this piece I’m willing to let it go at that.

Consider this diagram, which is a standard cybernetic diagram I’ve adapted from William Powers, Behavior: The Control of Perception (1973) – a classic, if you haven’t read it, you should:

We’ve got Donald Trump at the top; he’s governed by his goals (aka reference levels), endless praise and approval. At the left we’ve got the media’s presentation of the world, which is DJT's Input Function. THAT’s what Trump attends to; that’s what he cares about. Everything he does is intended to have effects there. That’s what he’s trying to control.

He attempts to control them through various actions (aka Output Function), which I’ve grouped together at the right. Those actions in turn have some effect on the world; there it is, at the bottom. The world is big and complicated, lots going on. While Trump, as President of the United States, is undoubtedly one of the most powerful men in the world, if not THE most powerful man in the world, the sorry (or fortunate, depending on your POV) fact is there’s not a lot he can do to affect the world. He can certainly do something, more than you and me, but not a lot.

Whatever it is that Trump does, it scatters and dissipates into the world, the world does what it does, and the media takes note. That brings us back to the box there at the left (aka input function). Of course Trump does pay attention to everything in the media, just a little bit of it.

Let’s add a little detail to the diagram:

I’ve divided his actions into four categories: 1) Tweets, 2) campaign-style Rallies, 3) Presidential Actions of all kinds, and 4) Everything Else. Over at the left I’ve singled out Fox News, which apparently is his favorite source of media input and grouped his other media into Select Media (of whatever kind, cable TV, print, internet, etc.). That box now contains only the media Trump attends to, nothing else, and I’ve shuffled everything else down there into the world (where it is in any case). But I’ve also added Rally Buzz to Trump’s input function. That’s clearly important to him, and that isn’t mediated by any media at all. He gets that directly.

Trump’s major innovations are those (unofficial) tweets and those rallies. Forget about all those ways he violates presidential norms, including keeping some undisclosed degree of control over his businesses. It’s not that they’re unimportant – certainly they are. But all presidents have bent the rules and gone off the farm now and then. It’s not new, though Trump may well be doing more of it than any others. But none of them have produced flotilla after flotilla of personal tweets and none have done all those campaign-style rallies. Those are new.

Why’s he do it? Isn’t that obvious? The rallies and the tweets are his most direct means of gaining some control over what appears in his input function. The rallies give him precious direct real-time feedback. Alas, he can’t do them every day.

But those tweets, he can do them every day. The effect is not so immediate as the rally buzz, and it is often unpleasant rather than favorable, but it’s fairly predictable. Those tweets show up on Fox and they show up in all sort of media, including that august bastion of FAKE NEWS, The New York Times. In fact, even when the feedback is nasty, that’s OK for Trump as long as he’d anticipated that nasty feedback. Those tweets also reach his Base, and thus primes them for the rallies where they share the mutual satisfactions of direct interaction.

Of course, his tweets have other effects out there in the world, diffuse effects. Those effects he cannot control. As for the various actions he undertakes as president, well, it’s complicated. I’ll leave those complications as an exercise for the reader.

I note that one of the biggest and most important complications is time scale. A tweet may show up on the radar screen (Input Function) within hours, but, depending on its content, it may have effects the diffuse out across days or even weeks. The same is true for everything else Trump does as president. What about those tariffs? He calls for them in a tweet, a speech, whatever, then the chatter. In time the orders are issued. And then, and then the results come rippling in over weeks, months, years.

Sorting all those things out in order to fine-tune the system, that’s tough. Heck, it takes a whole federal bureaucracy to do that. But Trump can always emit another tweet and see what sticks.

Always.

Tweet.

Vehicularization & Ring-Form: Remarks on some issues raised at #HEX01

Edit 11.16.17: I've added some new material to the section on ontological mismatch. I've marked it by highlighting it.

I enjoyed presenting to HEX01: First Workshop on the History of Expressive Systems. I wish I’d had more time (don’t we all?), I wish I’d been there in person to talk with people and play with the exhibits. We do what we can.

I’ve been thinking about these issues for years. And will continue doing so. Indeed, between the time I submitted a draft paper (Abstract Patterns in Stories: From the intellectual legacy of David G. Hays)...

Paper at Academia: https://www.academia.edu/34975666/Abstract_Patterns_in_Stories_From_the_intellectual_legacy_of_David_G._Hays

And the time I put the last touches on the PowerPoint I used for the talk ...

PPT at Academia: https://www.academia.edu/35142428/Abstract_Patterns_in_Stories_From_the_intellectual_legacy_of_David_G._Hays

I had a few ideas that pushed the work forward here and there. I continue the push in these notes, which are rather informal. I’m just trying to get the ideas down on (virtual) paper.

Of course, the workshop was about history, so what was I doing presenting new ideas? Continuing the history. Oh yes, I presented some history, the computational ideas worked out by David Hays and his students in the mid-1970s, and how I, a student of literature, came to them. But streams of intellectual development don’t stop just because they’re always disappearing into the past.

More importantly, things change, deeply. I went into the 1970s with one set of ideas – call it paradigm in Kuhn’s sense, an épistème in Foucault’s – which I used to think about how language and literature work. I encountered a very specific issue (problematic?) within that paradigm, the structure of “Kubla Khan”, and my efforts to deal with that issue forced me to think in terms outside that paradigm, to start cobbling together a new paradigm (if I may). Am I there yet? Who knows?

That’s what I address in the first of these notes, about ontological mismatch in our thinking. Then I take a look at the triune model of the brain, as Hays and I recast it in terms of control hierarchy. I then use that recasting to think about ring-form in King Kong. I conclude with some remarks about Heart of Darkness.

A half-century of ontological mismatch (beyond the singularity)

I mean ontology in the sense it has come to have in computer and cognitive science, the organization of different types of objects in some domain. Prior to my work on “Kubla Khan” [1] I had internalized a certain ontology for dealing with literary phenomena. But the moment I decided to interpret line-end punctuation like parentheses, brackets, and braces in a mathematical expression (or like nested parentheses in a LISP expression) I moved out of that ontology and into a different one. It’s worth noting that, when I made that decision, I specifically thought about the computer programming course I had taken, and how, in THAT world, if you place a comma where a colon is expected, it won’t work.

The problem, then, is how to think about literary texts in a world where LISP expressions are ‘native’ objects.

Of course, we–me, my teachers, others–didn’t realize that that’s what had happened. (Of course, we didn’t think in terms of conceptual ontologies at all.) We just thought I was doing something strange and interesting within the existing (or perhaps emerging) ontology. The same with my 1976 paper on Sonnet 129 [2]. To be sure, it looked very different from every other article in the special issue of MLN. It had all those diagrams, while the other papers had no diagrams at all.

It wasn’t until much later that I realized that, when I did that work on “Kubla Khan”, I had irreversibly left the conceptual world of academic literary criticism. “Irreversible” because I can’t go back, though I can do good imitations.

Contemporary work in computational criticism presents the same problem. The desire to call it “distant reading” reflects a commitment to the standard ontology, an ontology is which the text is only incidentally marks on paper. In the standard ontology the text is, well, that’s hard to say. It’s that thing that you read, it’s somehow tethered to those marks on the page, but it’s more than those marks.

Well of course its more than those marks, but I can’t think of a better way to characterize that “more” than to think of it as come kind of computational process. And that’s what computational critics are scrupulously avoiding. On the one hand thinking of the mind as somehow fundamentally computational is of little practical value in their computational work. But also, they need to deflect the criticism of their more traditional colleagues who are wont to think of the notion of the mind as computational as, you know, the work of the devil.

Yet, in their own work, computational critics are working within an ontology in which the text is just marks on paper. The (miraculous? not really, but very artful (rare device)) craft in computational criticism is to analyze massive collections of such (mere) marks in a way that reveals the traces of mind, thousands and tens of thousands of minds reading. Think of it, from mere marks to the mind. That’s what computational criticism allows.

THAT ontology is different from, incommensurate with, the ontology of ordinary lit crit. There’s a deep tension that that is being glossed over. On the one hand, computational critics call it “deep reading” and note that, no, it’s not in competition with “close reading”. They’re complementary activities, complementary perhaps, but not ontologically compatible. On the other hand, traditional critics see “computer” and give a shudder–“There be dragons! Weave a circle around them thrice, and then lock ‘em up and throw away the key!” It’s not that bad; really, it isn’t. 

But still, THAT conversation has no happy ending. But no one’s dealing with that ontological gap. It can’t be bridged. Rather, it signals a need to rethink the discipline from top to bottom.

Which brings us to The Singularity. I figure that dreams and/or nightmares of the day when computers will become super-intelligent, those fantasies are rooted in a 19th century worldview. As such there’s an ontological mismatch between them and computing technology.

More later.

In Memory of Bill Powers

This post is over two years old, but I'm bumping it to the top because I'm thinking about fundamentals of the mind and language. In particular: How much of it is computation? That, in turn, depends on how you define computation. Powers was a control engineer and created an elegant control-system account of the mind. My some accounts that's computation. By others it's not. It all depends. Back in the 1970s Dave Hays picked Powers' model to ground him computational model of cognition.

* * * * *

Just yesterday I learned that Bill Powers had died on May 24, 2013. Powers was an independent scholar and investigator. He came from an engineering background, had wide interests, and had a genial nature. Though I only interacted with him face-to-face on three or four occasions, I had some email correspondence with him for off and on over the years and his model of human perception and action is foundational to my work.

I first heard of Bill Powers in the Fall of 1974. I’d signed up for a graduate seminar taught by David G. Hays, who would become my mentor. The course was entitled “Language and Human Life,” which was an interdisciplinary rubric under Hays used for studying whatever interested him and his current students. I forget just how it went, but each one in the seminar got to suggest books and somehow we collectively arrived at a syllabus.

The first book up was one Hays picked, Behavior: The Control of Perception, by William T. Powers. It had been published in 1973 and received a good review in Science. Hays saw in it a way to ground his cognitive network model, which was at that time much of a piece with other such models (though it had a novel feature or two), in psychological reality.

We worked our way through the book, slowly and carefully. It was well-written and, though not dense, required deliberation and attention to detail. It had a number of relatively simple diagrams, such as this one (of a basic servomechanism):

Much of Powers’ model was in those diagrams. The prose was commentary on them. There was an appendix with some simple algebra.

Powers on Conflict in Control Systems

One of the most profound books I’ve ever read was written by William Powers: Behavior: The Control of Perception (1973). It received a positive review in Science, which is how my teacher, the late David Hays, learned about. He urged his students to read it and it became foundational to our thinking. In particular, when it came time to reconceptualize the cognitive model Hays had been developing over the years, he adopted Powers’s system as a model for the sensorimotor component of the system.

I don’t have time to go into Powers’s model in any detail. Suffice it to say that it was based on servomechanical control systems or the sort that Norbert Wiener popularized, at least in intellectual circles, in his Cybernetics. Powers’s innovation was to imagine a simple and elegant scheme whereby control systems were stacked one above the other, with higher level systems regulating the activities of lower level systems and lower level systems providing inputs to, and realizing outputs from, those same higher level systems.

In this note I’m interested in what Powers has to say about conflict. Here’s how he sets it up (pp. 253-254):

A person is said to be “in conflict” when he wants two incompatible goals to be realized at once. Since the time of Freud and no doubt for much longer than that, inner conflicts have been recognized as a major cause of psychological difficulties. Unresolved conflict leads to anxiety, depression, hostility, unrealistic fantasies, and even delusions and hallucinations. In fact as I have come to realize what inner conflict means in terms of this feedback model, I have become more and more convinced that conflict itself, not any particular kind of conflict, represents the most serious kind of malfunction of the brain short of physical damage, and the most common even among “normal” people.

The reasons for the extraordinarily bad consequences of conflict are not to be found in specific behavioral efforts, although disruption of overt behavior certainly can make life difficult. Our model, however, tells us that mere practical consequences of specific conflicts are secondary to their major consequence, which is to remove parts of the brain’s organizations from action as effectively as if they had been cut out with a knife, yet without getting rid of their undesirable influences on the whole hierarchy [of control systems]. The worst aspect of conflict between control systems is that the higher the quality of the control system, the more violent and disabling is the result of conflict.

A Note on Triple O’s Rhetoric of Objects

The rhetoric of object-oriented ontology was a matter of interest some weeks ago, and it remains so for me.

In particular, there is a rather traditional sort of ontology, associated with the Great Chain of Being, in which beings, objects, or substances, whatever term you prefer, are arranged in a scale, or ladder, from low to high. At the top one has God, and at the bottom, something like brute matter, perhaps atoms, while in between one finds the rest of the beings, kumquats, seraphim and the rest. Those things higher on the chain have more being than those low. It is thus a very different scheme from those offered under the rubric of object-oriented ontology and its conceptual fellows.

The Great Chain of Grammar

I’m interested in this great chain because a portion of it is written into our grammar, from which it erupts into the rhetorical practices of OOO. Consider this passage in which Levi Bryant comments on an account of soccer in which Michel Serres would have the ball in mastery over the players:

Second, as Serres’ example of the soccer ball as a subject where humans are quasi-objects for it–where the soccer ball is the seat of agency and the players are patients; in part, anyway–suggests, we need to develop an adequate notion of agency. What sorts of agency are there? What agency do we have?

That word “patients”, where did it come from? Bryant is certainly not suggesting that, all of a sudden, players become the objects of medical attention. That does happen, but that’s not what’s going on here. The word comes from linguistics, specifically, case grammar, where patient is a specific role that an object can play with respect to a verb, with agent, instrument, and object being other available roles. For example:

(1) John hit Jack with a stick.

(2) Jack was hit by John.

(3) John hit the wall.

In (1) John is playing the agent role, Jack the patient, and stick the instrument. In (2) John and Jack are playing the same roles as in (1), though the form of the sentence is different. In (3) John is the agent while wall is the object. Patients are generally construed as animate while objects may be inanimate.