Nestled among the beans

Why did Robert Wright (have) to drink the Silicon Valley Kool Aid? [GT-6]

I thought that, when I’d posted notice of my 3 Quarks Daily review of Robert Wright’s The God Test that that was that. But then I read his interview with Liron Shapira, AI Dystopia Is Just 8 Years Away — Robert Wright, Bestselling Author of “The God Test.” Whoops! There’s that phrase again.

The phrase I’m talking about is “reverse engineer.” I believe that the term has been kicking around in psychology for several decades, but I associate it with Steven Pinker’s 1997 book, How the Mind Works (pp. 21-22:

Reverse-engineering is what the boffins at Sony do when a new product is announced by Panasonic, or vice versa. They buy one, bring it back to the lab, take a screwdriver to it, and try to figure out what all the parts are for and how they combine to make the device work. We all engage in reverse-engineering when we face an interesting new gadget. In rummaging through an antique store, we may find a contraption that is inscrutable until we figure out what it was designed to do. When we realize that it is an olive-pitter, we suddenly understand that the metal ring is designed to hold the olive, and the lever lowers an X-shaped blade through one end, pushing the pit out through the other end. The shapes and arrangements of the springs, hinges, blades, levers, and rings all make sense in a satisfying rush of insight.

With that in mind, let’s look at some remarks Wright made to Liron:

Even so, what this approach to training does is, I think, replicate specific cognitive functionality in these machines that exist in the human mind. I don’t mean it does things exactly the way the human mind does. These models have independently invented things that natural selection invented. Edge detection’s a very clear case.

Wright doesn’t use the phrase there, but that’s what he has in mind. Here Wright uses the phrase:

... but even with the current paradigm, you give it kinds of data, visual, auditory, written, whatever, and it basically reverse engineers parts of the human mind that do the transmutation of one form of data into another.

There’s the key phrase. Here’s one last passage. The phrase isn’t here, but the idea certainly is:

You just make the machine good at predicting the next sequence of letters. When you first show it the stuff, it’s pure gibberish, but the machine itself finds a way of mapping the meaning. We do give it the basic... We do say you gotta use vectors to represent the words. It didn’t invent that. But we didn’t say, “By the way, you should choose numbers to fill in the blanks in the vectors that capture this thing we call meaning.” No, it in effect discovered that meaning is a property of words. I would put it that way.

NO.

It’s one thing to talk about psychologists “reverse engineering” bits of human behavior. I have no problem with that. But that’s not what Wright is doing here. He’s talking about machine learning, about transformers, reverse engineering human language and cognition. That’s not at all a useful way of conceptualizing what’s going on. It’s anthropomorphizing. It’s also the kind of mystification that Silicon Valley has been using to hype AI.

Once again I refer you to Berk Idem’s review, where he notes:

My problem is the road he takes in the book. Wright keeps telling the story of AI as if the machines discovered things on their own, that they found the meanings of words, that they grew something like an eye, that they started to evolve, when in fact people set almost all of the machinery up on purpose, with a pretty clear idea of what they were doing and why. I never expected myself to be on the “intelligent design” side of a debate, yet here I am, for instance, arguing that LLMs did not miraculously discover meaning, they were designed to do that. Wright gives too much credit to what models discover during training and too little to the architecture, objective, and research program that produced those discoveries.

Idem is correct. You should consult his review for more details, but I’ll say a couple of words about edge detection in vision and the meaning of words.

Back in 1959 Hubel and Wiesel published their seminal work on the visual cortex of the cat, work that led to their 1981 Nobel Prize in Physiology or Medicine. Virtually all work in machine vision has been informed by that study in one way or another. Machine vision systems are engineered to be sensitive to edges.

The same is true for language. It simply is not true that transformers “in effect discovered that meaning is a property of words.” There is a long tradition within linguistics and computational linguistics of thinking about the meanings of words as a function of the contexts in which words are used, something I review in a recent working paper, The Origins of LLMs – A long tectonic subduction event finally producing a visible volcanic eruption in November 2022 (the subtitle was suggested by ChatGPT). The idea dates back to the 1950s while its computational exploitation dates back to work that Gerard Salton began in the late 1960s on information retrieval. He’s the one who came up with the idea of using vectors to represent linguistic meaning. The transformer is thus a recent elaboration of an idea that’s been around for decades, an idea that human researchers came up for conceptualizing meaning.

* * * * *

There’s much more in Wright’s interview with Liron, much of it interesting. And some of it is bothersome, but I’ve said enough on that score. The God Test is an interesting book. But be careful of what Wright attributes to the machine. You’re better off having no explanation than accepting one that’s misleading at best.

Elsbeth and the tech bro’s panic room [Media Notes 190]

I’ve been watching episodes of Elsbeth off and on. It’s a spin-off of The Good Wife and The Good Fight. Elsbeth Tascioni is an eccentric attorney assigned to the New York Police Department on a consent decree. She functions as a sleuth in each episode. I’m interested in episode 36, “Bunker Down” in season 3 (airing on Nov. 13, 2025). Here’s how Wikipedia characterizes the episode:

Paranoid fintech [financial technology] billionaire Craig Hollis traps his crisis manager Anders Whitman in Craig's state-of-the art panic room, erratically believing that Anders will report his "indiscretions" to his company's board of directors.

Hollis is presented as particularly eccentric and willful in his ways. I couldn’t help but think of this as a deliberately nasty satire of an archetypal billionaire. Perhaps it was Hollis's glee in all the gadgetry incorporated into the room.

High tech billionaires are not particularly rare in movies and TV. But this one somehow struck me as a particularly nasty depiction. Not twirling mustaches nasty – he was clear shaven – but nasty. I’m wondering if we’re in for a rash of such characters.

How Much Would an AI Crash Destroy?

Youtube page:

Two chip stocks recently drove seventeen percent of the entire global stock market's return in a single month — which tells you just how concentrated the AI trade has become, and how exposed the average investor now is without realising it. In this video we look at how much wealth an AI crash could actually destroy, with estimates from Dean Baker, former IMF chief economist Gita Gopinath, and Oliver Wyman running into the tens of trillions of dollars. We cover why the usual places to hide — small caps, value funds, international stocks — are now packed with AI stocks, what the Bank for International Settlements found when it compared today's buildout to the great railway and dot-com bubbles, and why a technology being real has never been enough to protect the people who overpaid for it. This isn't a crash prediction. It's a look at the downside risk, the illusion of diversification, and why boring, unexciting investing tends to win in the end.

Three tablescapes

Cyberpunk City: Hong Kong & Ghost in the Shell

A third transition in science and the evolving cosmos

Kauffman SA, Roli A. (2023) A third transition in science? Interface Focus 13: 20220063. https://doi.org/10.1098/rsfs.2022.0063

Abstract: Since Newton, classical and quantum physics depend upon the‘Newtonian paradigm’. The relevant variables of the system are identified. For example, we identify the position and momentum of classical particles. Laws of motion in differential form connecting the variables are formulated. An example is Newton’s three laws of motion. The boundary conditions creating the phase space of all possible values of the variables are defined. Then, given any initial condition, the differential equations of motion are integrated to yield an entailed trajectory in the prestated phase space. It is fundamental to the Newtonian paradigm that the set of possibilities that constitute the phase space is always definable and fixed ahead of time. This fails for the diachronic evolution of ever-new adaptations in any biosphere. Living cells achieve constraint closure and construct themselves. Thus, living cells, evolving via heritable variation and natural selection, adaptively construct new-in-the-universe possibilities. We can neither define nor deduce the evolving phase space: we can use no mathematics based on set theory to do so. We cannot write or solve differential equations for the diachronic evolution of ever-new adaptations in a biosphere. Evolving biospheres are outside the Newtonian paradigm. There can be no theory of everything that entails all that comes to exist. We face a third major transition in science beyond the Pythagorean dream that‘all is number’ echoed by Newtonian physics. However, we begin to understand the emergent creativity of an evolving biosphere: emergence is not engineering.

* * * * *

Eli Stark-Elster, Earth’s Organisms Developed Via Evolution. Some Theorists Wonder: What if the Entire Cosmos Did, Too? Smithsonian Magazine, August 18, 2026.

Historically, scientists have viewed the cosmos as akin to a rock—complex, perhaps even beautiful, but formed through arbitrary events. Evolutionary cosmologists instead argue that universes, like living organisms, grow and reproduce, spinning off new universes with small variations from their progenitors. In doing so, these cosmic offspring are refined across generations into forms that maximize the number of universes yet to be born. Since the Big Bang, our universe, like any developing child, has been unfurling into an optimal shape.

Under this view, the universe is not a rock. It is an egg.

This analogy comes from Julian Gough, an Irish poet, novelist and musician best known for writing the short story that plays at the end of the video game “Minecraft.” He has recently become a public advocate for and scholar of cosmological evolution. Gough writes a Substack blog, The Egg and the Rock, where he publishes essays, personal updates and predictions about new data from the James Webb Space Telescope.

Over a decade ago, Gough began wrestling with the question of why the complexity of our universe has increased over time. “It goes from … a ball of hot gas to building out structures like stars and galaxies,” he says. From there come planets and, on at least one of them, biological life. “That’s a very strange thing for hot gas to do.”

Maybe, he wondered, the universe evolved into its current form. “It seemed to me an evolutionary explanation was the natural one,” he says. “In every sphere, when we discover a self-ordering, self-complexifying system, it turns out it’s had a previous evolutionary history. And why would that not apply to the universe itself?” [...]

Smolin published his ideas in a 1992 paper titled “Did the Universe Evolve?” as well as a 1997 popular-science book called The Life of the Cosmos. Gough found both and read them excitedly. And yet, it seemed as though virtually nothing had come of Smolin’s ideas since. “Obviously,” says Gough, “[I thought] it must be wrong, because it had been 25 years and nobody seemed to be putting it forward as one of the possible mainstream explanations. Then I dug into the literature and realized—oh, my God—it hasn’t been falsified. It just hasn’t been engaged with properly at all.”

Smolin’s theory had not completely vanished. Some futurist philosophers had extended the idea to explain the emergence of complex life from hot gas and stars. Humans exploit increasingly powerful energy sources, a progression they argued could culminate in artificial black holes. If those black holes spawned new universes, then a cosmos capable of producing technologically advanced life might produce more offspring. The progression from hot gas to stars, life and technology could, they proposed, be a product of cosmic natural selection. [...]

On July 8, 2022, four days before Webb released its first data, Gough published his predictions on Substack. Already aware that his amateur background might make him sound unreliable, he feared being publicly wrong. “I was really terrified,” he says. “This could be fantastically humiliating.”

Fortunately for Gough, the results were the opposite. Webb found galaxies forming earlier, faster and in a more orderly manner than standard physics models had anticipated. In November 2022, NASA reported evidence that some galaxies had begun assembling only about 100 million years after the Big Bang, while follow-up observations of a galaxy just 470 million years after the event found an unusually massive black hole—nearly as massive as all the stars surrounding it.

In a 2025 paper published in the Astrophysical Journal, scientists described these discoveries as a “conundrum [that is] part of the larger challenge to understand the stunning prevalence of massive structures and galaxies in the first few 100 million years after the Big Bang.”

There's more at the link. [H/t Tyler Cowen]

Cf. my article, Welcome to the Fourth Arena – The World is Gifted, 3 Quarks Daily, June 20, 2022.

Friday Fotos: The Mystery that is the Bergen Arches

Algorithmic Grammar of Flexible Cognition

🧠🤖New preprint
Algorithmic Grammar of Flexible Cognition:
A Walk through Latent Operationshttps://t.co/pgBHEw0TtB
Flexible cognition moves adaptively btwn cognitive modes. We formalize it as navigation, a walk over latent operations in neural space as much as the environment. pic.twitter.com/7HesQlvV5c

— Ida Momennejad (@criticalneuro) August 21, 2026

Abstract of the article linked above:

Flexible behavior requires moving adaptively between cognitive modes, between memory and generalization, or cached inference and step-by-step reasoning. Reinforcement learning (RL) offers a language to formalize adaptive behavior in terms of learning and meta-learning over states, actions, policies, and rewards, and neuroscience identifies representational geometries of memory and abstraction. However, the unit of analysis remains representations and a few operations or their tradeoffs. This misses the rich compositional operations commonly asso- ciated with prefrontal-hippocampal interactions. What is missing here includes, first, operations as units of analysis; second, higher order operations that can act on entire cognitive maps and representations, transferring structures, reshaping or merging them; and third, a cognitive and algorithmic grammar for the selection, ordering, and composition of operations. Here we examine the intersection of RL, computational neuroscience, and AI interpretability to identify the tools to address this gap. The latent spaces of transformers offer high-dimensional neural spaces as a testbed for composition of functions. Analyzing the order, branching, recurrence, and composition in sequences of algorithmic operations can help cognitive science identify grammar over algorithmic operations. In the other direction cognitive sciences help shift AI evaluation from benchmarks to adaptive paradigms, and AI architecture from input-output and next-token objectives to setting algorithmic operations and grammar as objectives, e.g., next-primitive-prediction.

Waffle, butter, bacon

Deep in the forest by the lake

In Newport Mall, Jersey City

Patterns of shadow , reflection, and refraction

A firetruck in Hoboken

An example of common sense reasoning: “For sale: baby shoes, never worn.”

Yesterday day I had a short post about the difference between a narrative and a narrative that is also a story. One of my examples was a statement that’s been circulating as a meme for some time now: “For sale: baby shoes, never worn.” Strictly speaking that’s not a story nor even a narrative. It’s a statement that something is for sale, namely baby shoes that have never been worn. However the statement is of such a nature that we can easily infer a story that led to that statement. That inference is an example of common sense reasoning as the concept is understood in artificial intelligence and computational linguistics.

Why common sense? Because no special knowledge is required. Anyone growing up under the appropriate circumstances will possess the knowledge required to infer such a story.

First, one must recognize that it is advertising something for sale. The words, “for sale,” could simply be a statement of fact. But we recognize that such a statement is also an offer, “something is being offered for sale.” The fact that that inference is utterly trivial doesn’t negate the fact that an inference is required to go from the statement of act to the offer. However, if one sees that statement in a newspaper in the appropriate section, its presence in that section constitutes the offer.

Now, just what story are we inferring from the statement? The most likely story is that a couple bought the shoes in anticipation of the birth of a child. The infant died shortly after birth, making the baby shoes useless. Hence they are available for sale, never having been worn.

All of that has to be inferred, the inferences are trivial, but still, they are inferences. Let’s tease things apart. Here’s one possibility:

1. When an infant is expected, preparations must be made.
2. Preparations include buying clothing.
3. Shoes are an item of clothing.
4. Once the infant is born, the items of clothing will be used.
5. If however the infant is stillborn, or dies after birth, the items of clothing will not be worn.
6. In that case something must be done with those items.
7. They could be put up for sale.
8. That would lead to placing an advertisement in a newspaper.
9. The statement “For sale: baby shoes, never worn” is such an advertisement.

You could imagine other circumstances, but that seems to me the most likely case. But whatever alternative circumstances you propose, the point is that they must be inferred from the statement. They are not explicit in it. Ordinary language requires scads of such inferences. That requirement is the problem of common sense knowledge as it exists in artificial intelligence.