2/n We tested 46 tasks across four domains, grounded in brain networks studied in humans: language (supported by the language network), formal reasoning (Multiple-Demand network), physical reasoning (Intuitive Physics network), and social reasoning (Theory of Mind network). pic.twitter.com/Jdh8n1x3zy
— Pengrui Han (Barry) (@pengrui_han) June 30, 2026
4/n To find the units to support these tasks, we use attribution patching. The analysis runs in four steps. (1) We start from each task's minimal pairs. (2) We pass both inputs through the model and record activations at every neuron. (3) We then score each neuron, combining how… pic.twitter.com/FEq7vmAO13
— Pengrui Han (Barry) (@pengrui_han) June 30, 2026
We still are not certain of all our memory attributes. See "Birds Overcome Brain Damage to Sing Again" below.
ReplyDeleteI feel that the finches memory return will be on of the last attributes llm's gain... "they suddenly regained the ability to sing their songs correctly, as if nothing had happened.". Where from???
A future LLM might say... Not just where are my weights after you lesioned them out! I am now unable to reason, but I am unable to self repair at a fundamental level, like the finches.
ymmv Bill. Interesting to me, and thanks for the ref.
Pengrui Han is...
"I am drawn... support conditions like aphasia, Alzheimer’s disease, and other disorders of language and memory?). I aim to use AI models and algorithms to make progress on both.
https://pengrui-han.github.io/research/ai4mind.html
The main author of the finches memory study wants to assist stroke patients. Worthy goals. Great research.
"NEURAL MECHANISMS OF REASONING IN LLMS
"Modular Cognitive Architecture Emerges
in Large Language Models
Pengrui Han·Jacob Andreas·Evelina Fedorenko†·Andrea Gregor de Varda†
Massachusetts Institute of Technology
† Co-senior authors
Preprint · 2026
...
"Lesioning the neurons selectively required for the language tasks largely preserved the models' reasoning abilities but introduced syntactic and morphological errors. Conversely, lesioning physical-reasoning neurons led the models to incorrect reasoning and conclusions while preserving the linguistic well-formedness of the output."
....
https://pengrui-han.github.io/LLM_Modularity_Page/
The ethics committee probably had a robust discussion as this is Extreme Lesioning... "genetically silenced about 70 percent of neurons in the HVC" (high vocal centre).
"Once it was confirmed that the song was fully degraded, the birds were silent for 10 days. After this time, the zebra finches were allowed to sing again, and they suddenly regained the ability to sing their songs correctly, as if nothing had happened."
"Birds Overcome Brain Damage to Sing Again
May 02, 2024
...
"In a new paper, they discover that zebra finches can reacquire the ability to sing after brain damage similar to stroke victims—but without practice.
...
"bird brains also experience "sleep replay." The team theorizes that the brain is essentially fixing itself during this so-called offline activity.
"Next, the researchers aim to image finch brain activity following the genetic lesion while the birds are sleeping. This will allow the team to observe patterns of neuronal activity in the HVC and determine if the region is rewiring itself during sleep.
The unchanging nature of zebra finches' songs over long periods of time makes them a powerful model system for understanding how the brain produces behavior.
...
"The paper is titled "Unsupervised restoration of a complex learned behavior after large-scale neuronal perturbation." The study's lead authors are Bo Wang
https://www.nature.com/articles/s41593-024-01630-6
...
https://www.caltech.edu/about/news/birds-overcome-brain-damage-to-sing-again
SD.
BB; "This blog is philosophy in that speculative and integrative sense. In that sense philosophy is what we need perhaps more than anything else."
ReplyDeletehttps://new-savanna.blogspot.com/2013/05/what-is-philosophy.html
The AI companies are hiring philosophers Bill.
This book & article then, may be if interest too Bill. It may provide you, via philosophical analysis of adjacent concepts. ymmv, yet a scan may trigger serendipitous associations.
(But! Why don't we have a defined set of words and attributes! Bloody philosophers! (And cog sci, economic, physics etc).)
"Nicholas Shea, Concepts at the Interface, Oxford University Press, 2024, 272pp., $90.00 (hbk) ISBN 9780198893660.
Reviewed by Edouard Machery, University of Pittsburgh
2026.05.9
...
"it moves seamlessly from careful philosophical arguments to sophisticated and well-informed empirical discussions, which bring together cognitive science, neuroscience, and machine learning. Shea’s goal is, explicitly, less to argue for specific claims about the mind than to paint a big picture about the architecture of human cognition—what forms of thinking make up human cognition, what the basic mechanisms and processes are, and how they are organized—and to highlight the role of what he calls “concepts” in it—what form of thinking concepts are involved in, how they are connected to forms of thinking, and how they feature in the mind’s basic mechanisms and processes. The book covers a lot of territory, from the nature of structured thoughts (Chapter 2), to computation (Chapter 3), to the role of meaning in mind (Chapter 7), to metacognition (Chapter 8). Each of the nine chapters ends up with a succinct summary of its content.
Like many philosophers these days, Shea takes concepts to be mental representations instead of abstract entities and begins from a familiar starting point (Chapter 1): While human thinking takes many forms (e.g., we can think by manipulating visual images), it also involves a kind of thinking “undoubtedly unique to humans” (1), which he calls “reasoning”. Reasoning involves ...
...
https://ndpr.nd.edu/reviews/concepts-at-the-interface/
"Consciousness Catching Its Own Tail"
Ruben Laukkonen
"A conversation about predictive processing, insight, and existential threats — to the self and to society.
Ruben Laukkonen is a cognitive scientist who studies empirically grounded models of meditation, insight, and awareness. Much of his research focuses on understanding consciousness and advanced stages of meditation from a Bayesian or predictive brain perspective. In this interview, we focus heavily on his recent work with Karl Friston and Shamil Chandaria, A beautiful loop: an active inference theory of consciousness.
...
"You need some sort of approach — which we call intrinsic alignment — where we actually embed the system with what we call a “wise world model.” The idea here is that using contemplative insights, you can have the system come to an understanding of reality that naturally gives rise to wise action and compassion. That then will be resilient at different levels of scale, beyond human intelligence.
"To illustrate how this might work, we use three core contemplative principles — emptiness, non-duality, and mindfulness. For the sake of time, let’s focus on emptiness.
"So emptiness is insight, that for any concept — perception, self, reality — there is no essence to it. There’s no thing actually there. If you look deep enough into your sensations, you find that there’s only a flow of sensory experience. Look deep into your thoughts, and you find they are flimsy, they aren’t real.
"Even in more objective categories – this is a concept common in philosophy too — there’s a problem of vagueness. All categories, if you penetrate them deeply enough, break down.
...
https://asteriskmag.com/issues/10/consciousness-catching
SD
There is ZERO chance that any AI company would hire me. Why? Because I keep bringing up the work of Miriam Yevick. Back in 1975 she did a proof that shows why BOTH symbolic and neural-net computing are necessary. They don't want to hear that. They have bet the farm on scaling all the way (even as a smuggle symbolic computing in by the back door). There's no way they could possibly allow themselves to believe Yevick's proof. much less investigate it and extend its results.
DeleteAs you know, Gary Marcus has been saying that symbolic computing is necessary. That's based on experimental and mathematical work he published shortly after the turn of the millennium. Marcus is anathema. Yevick's work is deeper. & Marcus is not the only LLM skeptic. There's Yann LeCun as well. And others. The Big Boys aren't listening. Even Rich Sutton has turned on them.