Friday, June 26, 2026
A Meeting of Minds on Mars: Charles Babbage • John von Neumann • Geoffrey Hinton
This discussion is about the implementation of computing in matter. That was the topic of John von Neumann’s last book, The Computer and the Brain. He died before he finished it, so it was published posthumously in 1958. I don’t know when I first learned about it, perhaps sometime in the mid-1970s. Though I knew about von Neumann and his role in early computing, it dismissed the book itself, figuring that we’d learned so much about the brain since then, and the nature of computers had changed so much, that it must be obsolete.
I was wrong. When I finally read the book, probably in the early to mid-1980s I was stunned. This was a profound book and free of mathematics beyond some simple back-of-the-envelope calculations. For one thing von Neumann talked of both analog and digital computation; that contrast was central. When I first started reading about computers in the mid-to-late 1960s that contrast was at the beginning of every article or book. But once personal computers appeared and proliferated in the late 1970s and 1980s, analog computing was all but forgotten.
It was good to see it back. For one thing, I had been strongly influenced by the work of William Powers, whose 1973 book, Behavior: The Control of Perception, offered an elegant analog model of the brain. That was central to the work of my teacher and mentor, David Hays, who had been one of the founders of computational linguistics. In 1974 and 1975 he worked out a scheme in which cognitive networks were grounded in Powers’s analog model. This was years before anyone else was working on the problem, before Steven Harnad coined the term, “symbol grounding” in his 1990 paper on the problem in Physica D.
Now, with the success of artificial neural networks, there’s another aspect of physical implementation we’ve got to deal with, distributed representation. You’re looking at an apple. Where is that apple represented in your brain? There might be some one neuron or a tightly clustered group of neurons that represent that apple. That’s a local representation. But the representation might be distributed across a loosely linked population of neurons. That’s a distributed representation. We now know that that’s how the brain works, though I assume there’s some laggards still stuck in the previous century where local representation was widely favored. And that’s how artificial neural nets work. The concept of apple isn’t localized in one or a small group of weights, it’s smeared over a population of weights. The same for the concepts of truth, beauty, and justice, or, for that matter, neuron.
Consequently von Neumann’s 1958 discussion must now be extended to distributed representation. It is for that purpose that I asked Claude to convene a conversation between Charles Babbage, who conceived of an elaborate mechanical computer in the 19th century, von Neumann himself, and Geoffrey Hinton, one of the so-called godfathers of artificial intelligence – a bit of journalistic and pop culture slight-of-hand that obscures a long and complex history. Moreover, since these three gentleman did not co-exist in time, though there’s a bit of overlap between von Neumann and Hinton, I decided that I might also have Claude transport them to Mars for the occasion. Anything is possible in virtual reality.
Or is it?
Image by ChatGPT.
A sun-facing room on Mars. Three chairs arranged loosely, angled toward a broad window. Through it, the Martian plain stretches red and absolutely still, and the sun — recognizable but wrong, too small, too distant — hangs in a pale pink sky. Three men stir and look about them, bewildered but unhurt.
I. Awakening
Charles Babbage
I am not dead — I can confirm that much. My pulse is steady and my faculties appear intact. But this room is not any room I have entered by ordinary means. That light through the window — is that the sun? The colour is right but the disc is too small by half. And that plain outside... brick red, entirely without vegetation or track or any sign of habitation. Gentlemen, I should very much like an explanation.
John von Neumann
I was in Princeton not five minutes ago. I notice the gravity is reduced — perhaps sixty percent of normal. The sky has the ferric oxide colouration one would predict. The solar disc subtends roughly two-thirds the angle it does from Earth. I believe I know where we are, and I find the hypothesis almost too extraordinary to voice.
Geoffrey Hinton
Mars. We're on Mars. I was at a conference in Toronto. But that sky, the dust, the sun at that distance — I'd stake my reputation on it. He exhales slowly. I've recently left a position at Google partly so I could speak freely about alarming possibilities. I did not anticipate that the alarming possibilities would include this. May I ask what year it is, from each of your perspectives?
Charles Babbage
The year of Our Lord 1862. I am Charles Babbage, formerly Lucasian Professor of Mathematics at Cambridge, Fellow of the Royal Society. I have spent the better part of forty years attempting to build mechanical calculating engines, with results that have been — a pause — mixed, as regards the support of Her Majesty's government.
John von Neumann
John von Neumann. The year is 1955. I work at the Institute for Advanced Study in Princeton, and consult for various government and military bodies. Mr. Babbage — I know your name extremely well. Better, I suspect, than you know mine.
Geoffrey Hinton
Geoffrey Hinton. For me it is 2024. And I know both of you — Mr. Babbage, you designed the Difference Engine and the Analytical Engine. Dr. von Neumann, you are among the most formidable mathematicians of the twentieth century and you gave your name to the architecture that every conventional computer on Earth is built upon. You are, in a real sense, my ancestors. The field I work in — machine learning, artificial intelligence — descends directly from the problems both of you were grappling with.
Charles Babbage
A long pause, during which he stares at Hinton with an expression mixing hunger, pride, and something close to grief. The Analytical Engine. Did anyone build it?
Geoffrey Hinton
Not in your lifetime. The government never provided the funds. But the ideas were entirely right — and they were eventually built, first in relay and vacuum tube and then in silicon, by people who in some cases had read your work and in other cases had arrived at the same conclusions independently. You were approximately a century early.
Charles Babbage
Very quietly. A century. I had hoped twenty years would suffice. I petitioned the Chancellor. Three times.
II. The mill and the store
John von Neumann
Mr. Babbage, allow me to tell you what your Analytical Engine set in motion — because it bears directly on the work that has occupied all three of us. You made a distinction, in your design, between what you called the Mill and the Store. The Mill performs the operations — addition, subtraction, multiplication. The Store holds the numbers awaiting operation and the results of operations completed. That separation of active calculation from passive memory was the foundational insight.
Charles Babbage
It seemed to me the only sensible arrangement. The columns of number-wheels in the Store are passive — they merely hold values. The Mill acts upon them. To mix the two functions in the same mechanism would create hopeless confusion.
John von Neumann
And yet it is precisely that separation which I have spent the last years of my life questioning — not as an engineering choice, which was entirely sound, but as a principle of intelligence itself. My colleagues and I formalized your Mill-and-Store distinction into what is now called the stored-program architecture. The processor executes instructions sequentially; the memory holds both data and those instructions passively until called upon. It is a serial machine, one operation following another, orchestrated by a central clock. The computers being built in my era all follow this pattern.
Geoffrey Hinton
And in my era they still do, at bottom. But you've just described the tension at the heart of everything, Dr. von Neumann. Serial, precise, with a strict wall between computation and memory — that is the von Neumann architecture. And it is, in a sense, the architecture that human intelligence refuses to use.
Charles Babbage
You are saying the brain does not separate Mill from Store?
Geoffrey Hinton
Exactly. In the brain, every neuron is simultaneously a memory element and a processing element. It holds information in the strength of its connections to other neurons, and it also fires — it computes — based on what it receives. There is no central Mill. There is no passive Store. Computation and memory are fused at every node in the network, and the whole thing operates in parallel, millions of neurons active at once.
John von Neumann
This is precisely what troubles me — and I am glad to find the trouble is still alive in your era, Mr. Hinton, because it means I was not merely chasing a phantom. I have been writing a manuscript, unfinished I'm afraid, called "The Computer and the Brain." My central puzzle is this: where, in a neuron, is the Mill? A neuron receives signals, sums them, and if the sum exceeds a threshold, it fires. That threshold operation is a computation. But the synaptic weights — the strengths of the incoming connections — those are also the memory. The neuron is its own Mill and its own Store simultaneously. I have been calling such things active elements, to distinguish them from the passive memory elements of our conventional machines. But I confess I have not yet worked out the full implications.
Geoffrey Hinton
With genuine feeling. Dr. von Neumann, the full implications are what I have spent my career working out. And you had named the essential thing: active elements. That is exactly what the nodes of a neural network are. Each one holds a weight — that is its memory — and each one applies a non-linear function to its inputs — that is its computation. Fused. Inseparable. Replicated millions or billions of times, connected in layers, and trained by adjusting all those weights simultaneously until the network's outputs match the desired answers.
This very expensive AI infrastructure depreciates fast; if you don't make profits soon, you won't make them ever.
“If there is a deflation of the AI bubble, the optimists say that the new infrastructure will remain even if the companies do not — just as railways survived the 19th-century railway bust. However, this fails to reckon with the reality of depreciation (few pieces of silicon hold… pic.twitter.com/TaiMJqUzne
— Gary Marcus (@GaryMarcus) June 26, 2026
Thursday, June 25, 2026
Smoke gets in your eyes [Hannibal Marvin Peterson]
Now's The Time (1992)
Hannibal Marvin Peterson (trumpet)
John Hicks (piano)
Richard Davis (bass)
Tatsuya Nakamura (drums)
In 2025 OpenAI spent $35B and lost $21B (Whoops!)
OpenAI spent roughly as much last year as the US government spent on the FBI and NASA combined.
— Ed Elson (@edels0n) June 25, 2026
The return on all that? A $21 billion loss. 👇 https://t.co/kN8wuFvmb9 pic.twitter.com/PAIzJPXehl
“The way down is the way up” may work for mystics, but not for business.
Why do American's fear AI, but many other countries don't? [Homo economicus strikes again!]
Paul Kedrosky, There’s One Clear Reason Why Americans Are Gloomy About A.I., NYTimes, June 25, 2026.
Hating artificial intelligence may be the only thing about which Americans agree. But they are global outliers in their pessimism. A survey of 24,000 adults across 30 countries found that citizens of nearly all of those countries, rich or poor, see A.I. more favorably than Americans do. This is startling for citizens of a wealthy, advanced economy who are usually enthusiastic tech adopters of anything with a wall charger. [...]
Why isn’t it working? Because the theory is incomplete, at best. If American A.I. pessimism were merely cultural or informational, it would correlate with media consumption, education levels or political polarization. Instead, it cuts across all those categories. It correlates instead with labor market institutions.
Start with the global picture. Plot A.I. sentiment against income and labor market, and there is a pattern. Poorer countries are A.I. optimistic: Indonesia at 76 percent, Thailand at 77 percent and Mexico at 63 percent. Rich countries like the United States, the Netherlands and Belgium are not. What A.I. means depends, in large part, on where you sit economically.
In countries with largely informal economies — where large numbers of people work without contracts, benefits or legal protections — A.I. looks like a ladder to better economic outcomes previously available only to those with capital, education and formal employment. A small manufacturer in Guadalajara or a street vendor in Jakarta doesn’t have much to lose from A.I. disruption, and potentially a great deal to gain.
In rich countries with more formal labor markets, however, A.I. looks more like an ambush. It threatens what people already have: stable employment, predictable income and accumulated professional standing. [...]
But not all wealthy nations feel the same. Norway is more optimistic than France, and Germany more than Canada. Those countries have broadly similar income levels, so income alone doesn’t explain the variation.
So what does? In Norway, losing your job means receiving around 67 percent of your previous wages in unemployment benefits while you search for the next position. In France, it’s around 66 percent, and 60 percent in Germany. The insurance system treats unemployment as a temporary inconvenience and bridges you smoothly across.
The United States pays significantly less in unemployment benefits than many European countries do. [...]
There's more at the link.
Wednesday, June 24, 2026
We need new economic indicators
We have the stock market figures everyday. Maybe also we could have figures on the number of homeless people too.
— David Graeber Institute (@Graeber_social) June 24, 2026
And the media can report daily on those homeless figures. pic.twitter.com/Wgc2w9Bglb
AI is in a regime of theory-free R&D
New @lawhsw piece on what automating ML R&D gets you. As usual, worth your time. https://t.co/jr9ny6EcPf pic.twitter.com/qGTMKVSQ40
— Séb Krier (@sebkrier) June 24, 2026
Eliezer Yudkowsky, LessWrong, OpenAI & countercultures past & present
Claude 4.6 Medium summarizes a dialog I had with Gemini that started with an inquiry about
1) Eliezer Yudkowsky, his early ideas, & his early following,
2) then to his interactions with Peter Thiel, Elon Musk, and Sam Altman that got OpenAI started,
3) to my own sojourn on LessWrong and
4) concluded with the connection between 1960s counterculture and contemporary Silicon Valley computer culture.
For some reason Claude pretended to summarize the conversation in my voice.
* * * * *
A Conversation with Gemini About Eliezer Yudkowsky Bill Benzon, new-savanna.blogspot.com, June 24, 2026
I recently had an extended exchange with Gemini (accessed through the Google search interface) about Eliezer Yudkowsky — a figure I've been thinking about in the context of AI culture more broadly. What follows is a summary of where the conversation went.
It began with a query about Yudkowsky's 2007 paper Levels of Organization in General Intelligence (LOGI), which argues that recursive self-improvement could allow an AI to rapidly cycle through levels of cognitive architecture in ways that would break traditional training and testing boundaries. Gemini gave a competent account of the paper's significance for AI alignment theory.
I then offered my own assessment: reading LOGI years ago, I concluded it was the kind of work produced by a brilliant college sophomore who had figured out everything and decided to write it up. The sort of student you'd want to guide and nurture — but of course, that never happened with Yudkowsky, who is entirely self-educated. Gemini agreed this was a common reaction, and traced the characteristic features of his writing — grand scope, idiosyncratic jargon, overconfidence — to the absence of the standard academic filters that would normally shape a thinker. Without a thesis advisor to push back, he co-founded his own institutions (MIRI, LessWrong), creating an insulated subculture where he became the mentor rather than the student.
I offered a specific passage from LOGI as an example of what goes wrong. Yudkowsky dismisses semantic networks as "completely bankrupt" on the grounds that they're simple enough to write on paper. Gemini correctly identified this as a classic category error: confusing the notation with the mechanism. The diagram on the whiteboard is inert; what matters is the graph-traversal algorithms, the spreading activation, the interpreter running the data structure. Ironically, Yudkowsky later wrote extensively about the Map-Territory Fallacy — but as I put it to Gemini, he is constantly mistaking a map for the territory. His entire worldview treats clean theoretical proofs as if they dictate messy engineering realities.
From there the conversation turned to how Yudkowsky managed to build such a large following despite these intellectual weaknesses. Gemini confirmed that Harry Potter and the Methods of Rationality, his 660,000-word fanfiction, was openly designed as a recruiting tool — drawing technically minded young people into the Rationalist and AI safety ecosystems. Countless engineers and founders who later populated early AI labs first encountered his ideas through that story.
The crowning irony: Yudkowsky's warnings about AI helped convince Elon Musk, Peter Thiel, and Sam Altman that humanity needed a counterweight to closed corporate AI efforts — which led directly to the founding of OpenAI in 2015. Once OpenAI pivoted to the empirical, data-driven methodology of large language models, they completely bypassed the deductive logic "maps" Yudkowsky had spent decades drawing. Sam Altman acknowledged Yudkowsky's role in a 2023 tweet, noting that he had arguably done more to accelerate AGI than anyone else, and adding that he might someday deserve a Nobel Peace Prize.
I told Gemini that wouldn't be necessary. I also shared my own experience: I joined LessWrong around the time ChatGPT launched, initially as an anthropological participant-observer, but stayed for the conversation, which I found genuinely useful. There are very smart people there. But the insularity was unmistakable — and I described one telling episode: someone on the forum was trying to spread Rationalism in Japan and struggling. I pointed out that Japanese popular culture, from Osamu Tezuka's Astro Boy through the Ghost in the Shell franchise, has a long history of viewing robots and AI as fundamentally benevolent — an expression of Shinto techno-animism, in which kami can reside in machines as naturally as in rivers. They simply didn't know about this cultural background. Gemini observed that the Western doomer ethos is rooted in a Frankenstein complex with Judeo-Christian substrata: creating life is hubris, and the creation must turn on its master. The Japanese paradigm operates from entirely different premises. The LessWrong response to my observation? They noted it and moved on.
After a while I tired of the place. One small anecdote captures the texture of the experience: I frequently link out to other things I've written, and one LessWrong post linked to an essay-review I'd done of Benny Shannon's book on ayahuasca. I noticed a significant spike of traffic to my Academia page coming from LessWrong and pointing to that essay — which tells you something about the undercurrent of interest in altered states of consciousness running alongside the dry decision theory. They approach psychedelics with an engineer's curiosity: the brain as a computer, phenomenology as data.
The conversation ended with what I think is the most useful historical frame. LessWrong is, in structure and function, a counterculture — but centered on computers and AI, with Yudkowsky as guru rather than Timothy Leary. And there's a genuine genealogical link through San Francisco and transhumanism: Stewart Brand bridging the Merry Pranksters to personal computing, the Extropians of the 1990s who wanted to transcend the body via nanotechnology and cryonics rather than LSD, and then Yudkowsky emerging from that same Bay Area Transhumanist mailing-list culture. Fred Turner's From Counterculture to Cyberculture maps this lineage. The counterculture became the vanguard — but corporate reality, as I noted to Gemini, has not submitted. Bill Gates and his successors were never absorbed by the counterculture. Peter Thiel, who was an early funder of MIRI, has since publicly labeled Yudkowsky a Luddite and positioned AI safety concerns as obstacles to American technological dominance. The "well-run alternative universe" of LessWrong lost all leverage once scaling deep learning required billions of dollars in silicon, electricity, and data centers. The colorful intellectual vanguard warmed society up to the idea of AGI; then the massive engine of global capitalism took the steering wheel.
The subculture keeps its cozy, insular forum to debate the semantics of the map. The corporate empires plow ahead across the territory.
Near-optimal AI through symbolic learning
You don't need to wait until 2040. You can solve 2-6 OOM today. You already called out the specific error modes, it's undergrad algebra required to fix the errors, and some conceptual systems building. I have been using a prototype for the last 2 months. It works.
— Ryan McCormick (@RyanMcC35236715) June 23, 2026