Cerebral plasticity and localization; mathematics, vision, numerosity

Dan Modano, Blind People Do Math In The Visual Cortex , ReliaWire:

People blind from birth appear to do math in a part of the brain typically devoted to vision, a new study has found. Researchers using functional MRI watched the visual cortex in the brains of congenitally blind people as they solved algebra problems in their heads.

The visual cortex didn’t merely respond, the researchers say. The more complicated the math, the greater the activity they saw in the vision center.

The same did not happen in the brains of sighted people with masks covering their eyes who did the same math exercises as the blind subjects.

However:

Many believe this number sense evolves from seeing the world and trying to quantify all the sights. But seeing has nothing to do with it; the study showed that the brain network behind this sort of numerical reasoning is identical in blind and sighted people.

“The number network develops totally independently of visual experience,” Kanjilia says. “These blind people have never seen anything in their lives, but they have the same number network as people who can see.”

The research article:

Shipra Kanjliaa, Connor Lanea, Lisa Feigensona, and Marina Bednya, Absence of visual experience modifies the neural basis of numerical thinking, PNAS vol. 113 no. 40 > Shipra Kanjlia, 11172–11177, doi: 10.1073/pnas.1524982113

Significance: Human numerical reasoning relies on a cortical network that includes frontal and parietal regions. We asked how the neural basis of numerical reasoning is shaped by experience by comparing congenitally blind and sighted individuals. Participants performed auditory math and language tasks while undergoing fMRI. Both groups activated frontoparietal number regions during the math task, suggesting that some aspects of the neural basis of numerical cognition develop independently of visual experience. However, blind participants additionally recruited early visual cortices that, in sighted populations, perform visual processing. In blindness, these “visual” areas showed sensitivity to mathematical difficulty. These results suggest that experience can radically change the neural basis of numerical thinking. Hence, human cortex has a broad computational capacity early in development.

Abstract: In humans, the ability to reason about mathematical quantities depends on a frontoparietal network that includes the intraparietal sulcus (IPS). How do nature and nurture give rise to the neurobiology of numerical cognition? We asked how visual experience shapes the neural basis of numerical thinking by studying numerical cognition in congenitally blind individuals. Blind (n = 17) and blindfolded sighted (n = 19) participants solved math equations that varied in difficulty (e.g., 27 − 12 = x vs. 7 − 2 = x), and performed a control sentence comprehension task while undergoing fMRI. Whole-cortex analyses revealed that in both blind and sighted participants, the IPS and dorsolateral prefrontal cortices were more active during the math task than the language task, and activity in the IPS increased parametrically with equation difficulty. Thus, the classic frontoparietal number network is preserved in the total absence of visual experience. However, surprisingly, blind but not sighted individuals additionally recruited a subset of early visual areas during symbolic math calculation. The functional profile of these “visual” regions was identical to that of the IPS in blind but not sighted individuals. Furthermore, in blindness, number-responsive visual cortices exhibited increased functional connectivity with prefrontal and IPS regions that process numbers. We conclude that the frontoparietal number network develops independently of visual experience. In blindness, this number network colonizes parts of deafferented visual cortex. These results suggest that human cortex is highly functionally flexible early in life, and point to frontoparietal input as a mechanism of cross-modal plasticity in blindness.

Leaves

Friday Fotos: Jamie Variations

[See explanation below.]

William Gibson on process, recordings, the future, Victorians & trains

David Wallace-Wells interviews William Gibson in The Paris Review. Gibson's process:

INTERVIEWER

And your schedule is steady the whole way through?

GIBSON

As I move through the book it becomes more demanding. At the beginning, I have a five-day workweek, and each day is roughly ten to five, with a break for lunch and a nap. At the very end, it’s a seven-day week, and it could be a twelve-hour day.

Toward the end of a book, the state of composition feels like a complex, chemically altered state that will go away if I don’t continue to give it what it needs. What it needs is simply to write all the time. Downtime other than simply sleeping becomes problematic. I’m always glad to see the back of that.

INTERVIEWER

Do you revise?

GIBSON

Every day, when I sit down with the manuscript, I start at page one and go through the whole thing, revising freely.

INTERVIEWER

You revise the whole manuscript every day?

GIBSON

I do, though that might consist of only a few small changes. I’ve done that since my earliest attempts at short stories. It would be really frustrating for me not to be able to do that. I would feel as though I were flying blind.

The beginnings of my books are rewritten many times. The endings are only a draft or three, and then they’re done. But I can scan the manuscript very quickly, much more quickly than I could ever read anyone else’s prose.

The advent of sound recordings:

My great-grandfather was born into a world where there was no recorded music. It’s very, very difficult to conceive of a world in which there is no possibility of audio recording at all. Some people were extremely upset by the first Edison recordings. It nauseated them, terrified them. It sounded like the devil, they said, this evil unnatural technology that offered the potential of hearing the dead speak. We don’t think about that when we’re driving somewhere and turn on the radio. We take it for granted.

Gunnar Johannson and motion perception

These videos may have some bearing on Jamie Bérubé's biomorphs, so I'm bumping this post to the top of the queue. Think of his  biomorphic sheets as a form of proto-animation. Does those graphic objects 'tickle' the same neurons as these moving-dot images?

Watch these two videos, which are about the work of Gunnar Johansson. They're interesting and important. Just why they're important is more than I've got time for now, but it has to do with how we perceive motion. And perception of motion is VERY important. The world is full of moving things, many of the important to us. Like our fellows, and sources of food, or predators.

Some of these demonstrations use a primitive form of the motion capture technology Hollywood uses in driving computer-generated figures by the motions of human actors.

Jamie’s Investigations, Part 5: Biomorphs, Geometry and Topology

Figure 1: Biomorphs

Michael calls these images geometrics, and they are that, geometric. But, for reasons we’ll get to in a bit, I call them biomorphs. Here’s what he says about them:

Like Jamie’s fancy letters, probably influenced in part by the work of his brother the architect. But whatever their provenance, they are totally cool. I have included a few geometric drawings he did in crayon, but overwhelmingly this series is monochrome, usually in black but sometimes in red, blue, or orange.

After looking at these for awhile, see Figure 1, I decided they looked like tree branches and complex letterforms, so I emailed Mark Changizi, a theoretical neuroscientist who has done work on letterforms [1]. He has been making a general argument that culture re-purposes, harnesses (his term), perceptual capacities our ancestors developed for living in the natural world. One of his arguments is that the forms used in writing systems, whether Latinate or Chinese (for example), are those that happened to be useful in perceiving creatures in the natural world, such as plant and animal forms. I told him that Jamie’s forms looked like “tree branches and such.” He replied that they looked like people. His wife, an artist [2], thought so as well, and also: “This is like early human art.”

Which is to say, they appear to be biomorphic. Yes, these are geometric figures; but they’re special geometric figures. Well, “special” may not be the right word, but they’re not the relatively simple convex polygons–triangles, rectangles, pentangles, parallelograms, trapezoids, etc.–you study in a high school geometry class, at least not the class I took back in the Jurassic Era. Almost all of the individual figures–and perhaps all of them, I simply haven’t counted them up–are concave and all of them are “jointed” one or more times [3].

Let’s bring Michael into the discussion:

Oh, I agree that some forms are people-ish and creature-ish. But if you watch him at work (as I have, many times), it's almost like he's arranging forms on the page, not interlocking, but still talking to each other spatially. And some of them wind up looking like they have bodies, appendages, etc. And I totally agree that it feels interesting and evocative for him! The geometrics are his most recent form, dating over the last couple of years–

All of which is most interesting, especially the part about “arranging forms on the page.” For that’s composition, no? Getting his forms to “talk to each other spatially,” that’s more interesting, more sophisticated than what he was doing with the dots, the towers, and even the concentrics and letterforms.

Theme and Variations, but not a Bérubé

Jamie’s Investigations, Part 4: Concentrics, Letters, and the Problem of Composition

Figure 1: Concentrics

Let’s look at two closely related sets of sheets. One of them consists of concentric circles that have been colored-in, almost target fashion (Figure 1). With one exception they all have two sets of concentric circles. The two sets touch one another and they’re always arranged one set above the other, never side-by-side. Michael calls them swirls while his wife, Janet, and older son, Nick, call them jawbreakers. I’ll just call them concentrics. The other sheets consider of a set of concentric circles below and three or four rows of alphabetic circles above (Figure 2).

Figure 2: Letters & Concentrics

To be honest, I don’t find these images as attractive as the dots or the towers, but that’s irrelevant. What is important is why Jamie likes to draw these sheets. As in the cases of the dots and the towers, I base my conjectures on the visual properties of the images themselves and the problems involved in drawing them.

* * * * *

Let’s consider the concentrics first. In the next two examples, the two sets of circles are of roughly the same size. In Figure 3 they take the full height of the sheet while in Figure 4 they are confined to roughly the top half of the sheet.

Figure 3: Full-height concentrics

Figure 4: Half-height concentrics

How does Jamie draw these sheets? That’s question about composition. How does he ensure that the two sets of concentric circles are of roughly the same size –assuming, for the moment, that he is trying to achieve that result? That question is about technique. The assumption is reasonable given that most, though not all, of the concentrics consist of two circle sets of roughly the same size (I discuss this in more detail below).

In the case of the dot sheets Jamie seems to be interested in covering the sheet with dots. Sometimes he covers most of the sheet, sometimes he doesn’t. Judging from how the sheets look – I’ve not seen Jamie draw, nor have I asked Michael about this -– he appears to start along one edge and create row after row (if he starts from the top) or column after column (if he starts from one side). As long as Jamie keeps the dots roughly the same size and roughly the same distance apart, he’ll cover all or a large part of a sheet with a relatively uniform array of dots. Here’s the crucial point: he doesn’t have to do much, if any planning, to produce a good result.

Across cultures, bodily integrity is more valued than civil liberties

Wade M. Cole, Human Rights and the Individual: Cross-Cultural Variation in Human Rights Scores, 1980 to 2010, Social Forces (2016)

doi: 10.1093/sf/sow078

Abstract: This study analyzes patterns of cross-cultural variability and convergence in two categories of human rights: bodily integrity (protection from torture, extrajudicial killing, and other forms of physical repression) and civil liberties (the freedoms of expression, assembly, movement, and religion). Countries are delineated into twelve cultural zones based primarily on predominant religious tradition and secondarily on geographical region. The core hypothesis predicts that respect for bodily integrity rights, which seeks to protect biological beings from physical harm, will vary less across cultures than respect for civil liberties, which empowers social and cultural entities to be self-determining agents. Individuals’ capacity for pain and suffering is thought to be universal, but conceptions of the bounded and autonomous actor are culturally constructed and hence variable across cultures. Statistical analyses support this hypothesis: compared with civil liberties scores, cross-cultural variation in bodily integrity scores is much lower and also less durable in the presence of control variables. Moreover, whereas civil liberties scores are substantially higher in Western countries than in the rest of the world, cross-cultural variability in bodily integrity scores is gradational rather than polarized.

H/t Tyler Cowen.

Jamie’s Investigations, Part 3: Towers of Color

Example 1

Here’s what Michael says about these sheets:

I originally wanted Life as Jamie Knows It to include, between chapters, samples of these towers, but (a) that would be way too expensive and (b) you really need to see these things seriatim to get the full effect. Janet and I have framed some of them – we hung two in our New York apartment and gave some to family members – but however beautiful some individual pages are, the effect of having 35 of them in a row is even more remarkable.

Some obvious things (see example above): the towers run the height of the page; they’re relatively narrow; and the colors change from one segment to the next within a tower. When coloring individual towers Jamie is, for the most part (we’ll get to that later) following his principle of local contrast. However, if you compare adjacent towers, it is not at all unusual to see adjacent regions with the same or highly similar colors for a segment or three. Is Jamie treating each tower as an independent entity, so that such conjunctions are fortuitous, or are such conjunctions deliberate? I don’t know. But I’m inclined to think it’s deliberate.

There’s something else that’s deliberate, and puzzling. Counting from right to left, about a third of the way down and between the second and third tower there’s a blue spot. Go over two columns and, between four and five, you’ll see another blue dot further down the page. Several to many of the sheets have such dots. I don’t know what they’re doing, nor does Michael.

Example 2 (below) is much like Example 1 except that the towers start from the right edge rather than the left.

Example 2

Notice the local variation, the ‘color coupling’ between adjacent towers, and one of those spot, halfway down the page between two and three (counting from the right). If you look closely you’ll see that this one is black, and on a turquoise background.

Sunrise over Times Square as seen from Hoboken (across the river)

Jamie’s Investigations, Part 2: On Discovering Jamie’s Principle

To be honest, I was feeling pretty good after my first post in this series, Jamie’s Investigations, Part 1: Emergence, for it seemed to me that I’d pulled a pretty snazzy rabbit out of the hat. But as I continued looking through Jamie’s art and started thinking about further posts, those good feelings began to dissipate. What am I going to do to top THAT? What if there are no more rabbits in the hat? Because that’s how it looks at the moment: no more rabbits. If that’s the case, then we’ve just got to get as much fun out of this one rabbit as we can.

By rabbit I mean the idea that Jamie is following a local rule when he draws, but the general context is such is that those local choices result in (sometimes surprising) global order. Thus in the case of the dots pictures, the local rule is, don’t have adjacent dots be the same color. The global result is an overall rhythm to the image that’s easy to see and sense, but hard to describe.

With that in mind, let’s look at Jamie’s early work, done between 2003 and 2005, between the ages of 11 and 14. As Michael notes “all his modes of drawing and doodling and entertaining himself are here in embryonic form.” But let’s start with the one drawing that is NOT a precursor to later work:

Only one of these works is dated– the drawing on the left side of the first row, which remains (to my knowledge) Jamie’s only attempt at representation, aside from a self-portrait he did in school around the same time (May 2003). I did indeed have a purple swimming mask at the time, and Jamie’s was blue; and if you look carefully on the right side of the page, you can see, in gray, a box with three lines in it. This is of course the pool ladder, and Jamie and I are about to jump into the deep end.

And here it is:

Example 1: Representation

Why didn’t Jamie do other representational drawings? Children do lots of representational drawings, many of them just as crude as Jamie’s. Jamie, of course, is not like other children. He’s Jamie.

But that doesn’t tell us much of anything. What is it about Jamie that he loves to draw, but not representationally? I suspect it’s the fact of representation, the idea that the drawing is something other than itself, that the marks on the page are responsible to something other than, well, themselves, themselves and the artist. That’s not quite good enough, but it will have to do for now. I note in passing, however, that major artists of the past century or so have been happy to make non-representational art and that Islam forbade representation, resulting in an extraordinarily rich geometrical art.

This next sheet of motifs appears to me to be quite early as it’s just lots of bits and pieces of stuff, none of them very well developed.

Example 2: Miscellaneous

Encode THIS – old lessons from McPaint

Why I am not a semiotician, some crude notes

I was exposed to semiotics early in my undergraduate years at Johns Hopkins (where, of course, Pierce once held an academic post). I read Barthes’s Elements of Semiology probably in my sophomore year in one of the many courses I took from Richard Macksey and, perhaps in the same course, Lévi-Strauss’s well-known essay, “The Structural Study of Myth.” That material was foundational for me. Moreover, I have published in semiotics journals. One of my earliest articles, “Sir Gawain and the Green Knight and the Semiotics of Ontology” was published in Semiotica in 1977 and Dave Hays and I published “Metaphor, Recognition, and Neural Process”, in the American Journal of Semiotics a decade later (1987).

But I’m not now a semiotician nor was I then. To be sure, I use Saussure’s terminology (sign, signifier, and signified) and certain ideas. But the conceptual apparatus of semiotics has not been foundational for me. Once I found the cognitive sciences, THAT became my intellectual home base. It seemed ‘deeper’.

Why?

Here’s a note I wrote in the context of a ‘session’ at Academia.edu. The session is about my manuscript, “Sharing Experience: Computation, Form, and Meaning in the Work of Literature”.

* * * * *

I’ve been thinking about this ‘code’ business. I more or less understand how Morse code can be used to encode the characters of the Latin alphabet, Arabic numerals, etc. for transmission in an appropriate medium. I also have a reasonable understanding of ASCII code, which performs a similar function for the world digital electronics. I’ll even go so far as to claim some understanding how segmental phonemes encode the morphemes and lexemes of a language.

In each case we have physical objects on both sides of encode/decode process. To send a message using Morse code, you start with an alphanumeric text and, encode it into Morse code and then transmit the encoded message. To decode it you take the transmitted message, segment it into individual code units, and then translate the individual Morse units into the appropriate alphanumeric characters. The story about ASCII code is similar. The story of phonemes, morphemes, and lexemes is somewhat different. In some ways it’s simpler (since we’re just concatenating phonemes into standard sequences) and in some ways it’s more complex (the actual sounds of phonemes are highly variable in ways that are contextually dependent, yet the identity of the phoneme is constant). But I have some reasonable idea of the kind of story that needs to be told.

But when we talk of how lexemes ‘encode’ meaning, that’s VERY different. What’s on the ‘other side’ of the physical object, the ‘meaning’ side? We don’t talk of the signifier encoding the signified, do we? The relationship between signifier and signified is NOT like the relationship between an alphanumeric character and its Morse code equivalent. It’s a different kind of relationship performing a different job.

That cognitive model I discuss in the paper in the section, “Computational Semantics: Network and Text,” that’s what I think is on the meaning side of sign relationship, the signified. That’s my deepest excursion into meaning, and Hays’s too. And I don’t think we ever talked of codes and encoding when working on that model. Code-talk just doesn’t seem useful there. Whatever it is that’s ‘connected’ to the signifier, it’s NOT a discrete unit in the way that Morse code units or ASCII units are discrete units. It’s something very different.

I’m wondering whether or not the whole notion of cultural codes, where it’s meaning that’s being encoded, isn’t just a hopeful metaphor. We (implicitly) start with something we understand, like Morse code, in the hope that THAT kind of system will tell us something useful about something we don’t understand, like how words and utterances and texts have meaning. For me, at least, the metaphor doesn’t work.

Trump on a stick: Día de Muertos