Imitation: Beyond those Freakin' Mirror Neurons

On the Relationship between the Visual Perception of and Motor Regulation of Motion

Imitation is important in human behavior, and perhaps in animal behavior as well. But it is one thing to think about imitation in the abstract and another to ask: How is it done, physically, by the brain and body? You see someone doing something, and you do it, just like that. Well, you do it just like that if the behavrior is within your repertoire; otherwise you've got work to do.

These days you're likely to hear a canned answer to the question: mirror neurons. From the Wikipedia: "A mirror neuron is a neuron that fires both when an animal acts and when the animal observes the same action performed by another." That there are such neurons tells us something, but not much. How does the neuron "know" that the action observed (vision) and the action executed (motor) are the same?

That's the question before us, one of them anyhow.

Moving and Seeing Motion

Two days ago I posted a recent discussion the regulation of motor behavior and appended some commentary; the work was done in the laboratory of Michael Riley and was specifically concerned with the interataction of two people. The theoretical approach was based on the work of Nikolai Bernstein from the middle of the last century. Bernstein was concerned about the computational program of controlling bodies with many "degrees of freedom" (independent vectors of control) and hypothesized that the brain performed this feet by coupling the the motions of different parts of the system (that is, joints) so as to "compress" the system's degrees of freedom.

Yesterday I posted two videos about the work of Gunnar Johansson, a Swedish researcher interested in motion perception, including the perception of biologial motion (e.g. people in motion). What Johansson did was simple: He'd present subjects with a short video of simple objects in motion, dots and lines, and ask them what they saw. Given the many different ways one could "assemble" the moving objects into a perceptible something, the problem is computationally difficult (as researchers in computationhal vision discovered long ago). And yet humans have no difficulty at all resolving the relative motions of two-dozen dots into (the mental image of) two people walking or dancing, and can do so in a second or less.

How do they do that? Well, the brain does have lots of computing power available to it, but that computinig power is not infinite and when it comes to infinities the world outpaces the brain every time. Computing is a physical activity and works against limited resources, of which time is one. Compared to electronic circuts neurons are slow, very slow. But they solve this horrendus computational problem in under a second.

Johansson's answer was that the brain imposes constraints on the problem. Those constraints represent dimensional compression. The brain doesn't consider all possible interpretations of the dots and lines in the visual display. It considers only some of them, those that involve the motion of rigid bodies, such as limbs.

We now ready to think about what those mirror neurons are doing.

Common Coding Theory and Diagonalization

These days researchers talk of common coding theory, an idea that has been traced back to William James:

Common coding theory is a cognitive psychology theory describing how perceptual representations (e.g. of things we can see and hear) and motor representations (e.g. of hand actions) are linked. The theory claims that there is a shared representation (a common code) for both perception and action. More important, seeing an event activates the action associated with that event, and performing an action activates the associated perceptual event.

Some years ago when we were thinking about brains and intelligence in a very abstract way, David Hays and I hit upon the same idea and called in diagonalization, one of five proposed principles of natural intelligence. Thus:

William L. Benzon and David G. Hays. Principles and Development of Natural Intelligence. Journal of Social and Biological Structures 11, 293 - 322, 1988.

We used the work of Bernstein and Johansson as our prime case in discussing diagonalization. Here's the opening of that discussion (p. 298):

The technique of diagonalization was first employed by the mathematician Georg Cantor in his proofs that (1) the rational numbers are countable, and (2) the real numbers are not countable (Hermes & Markwald, 1974). The technique allowed Cantor to structure infinite sets in a way which made them tractable. The nervous system faces a similar problem. Sensory input is often ambiguous, with many interpretations possible and no obvious way of choosing among them. Hence the need for sensory coherence.

Definition. Diagonalization applies information from one channel to resolve ambiguity and impose structure in another channel.

Perhaps the best way to grasp the principle is to consider a specific example first. We can then examine an explicit account of the principle. Gunnar Johansson's work on the perception of motion (1973, 1975) is suggestive. His basic technique has been to generate two-dimensional visual displays of dots in motion and ask his subjects what they see. One experiment involved images generated by moving people. Lights were attached to shoulders, elbows, wrists, hips, knees and ankles; and then films were made. These films show patterns of movement among 12 spots of light against a dark background. Subjects had little difficulty identifying the nature of the images, often doing so in a tenth of a second (the time required to project two frames of film). In fact, subjects performed better on this task than they did with mathematically simpler images derived from simple deformations and rotations of elementary geometric figures. 

There is nothing surprising in the fact that we should be very sensitive to moving patterns generated by our fellow humans. Such sensitivity could be easily explained by evolutionary considerations. What is remarkable is the mechanism which realizes such sensitivity. There are infinitely many structures and motions which could generate the observed pattern of moving dots, yet the nervous system settles on one, the motion of a figure having members and joints arranged as humans do, and moving in the ways which humans do.

One way to interpret this phenomenon is to see it as an interaction between visual and kinesthetic space. Following the work of the Russian psychologist N. Bernstein (1967), Karl Pribram (1971) has argued that the motor cortex stores images of trajectories which it uses to regulate the activity of lower brain centres in generating movement. These images take the form of a Fourier transform of the trajectory (see also Gallistel, 1980). While Johansson uses vector geometry, it is clear that his results could be recast in Fourier terms, or that Bernstein's could be recast in vector geometrical terms. Hence, the same neural coding can be used for both the perception of organismic motion in the visual field and the generation of such motion in the kinesthetic field. The principle is that the analysis of information in one sensory channel must be consistent with the analysis of information in other sensory channels; and, perhaps more importantly, the analysis of information in one channel can only be made coherent through consideration of information in other channels. Diagonalization is a technique for achieving this coherence.

For those who are curious about such things, Hays and I suggested that diagonalization give the nervous system its ontology, as that term is understood by students of ontological cognition, where the term refers to how we think about and conceputalize the world, not to how the world really is.

Should you be curious about our whole argument on natural intelligence, here's the abstract:

Abstract: The phenomena of natural intelligence can be grouped into five classes, and a specific principle of information processing, implemented in neural tissue, produces each class of phenomena. (1) The modal principle subserves feeling and is implemented in the reticular formation. (2) The diagonalization principle subserves coherence and is the basic principle, implemented in neocortex. (3) Action is subserved by the decision principle, which involves interlinked positive and negative feedback loops, and resides in modally differentiated cortex. (4) The problem of finitization resolves into a figural principle, implemented in secondary cortical areas; figurality resolves the conflict between pro-positional and Gestalt accounts of mental representations. (5) Finally, the phenomena of analysis reflect the action of the indexing principle, which is implemented through the neural mechanisms of language.

These principles have an intrinsic ordering (as given above) such that implementation of each principle presupposes the prior implementation of its predecessor. This ordering is preserved in phylogeny: (1) mode, vertebrates; (2) diagonalization, reptiles; (3) decision, mammals; (4) figural, primates; (5) indexing, Homo sapiens sapiens. The same ordering appears in human ontogeny and corresponds to Piaget's stages of intellectual development, and to stages of language acquisition.