The compute theory of everything

Samuel Albanie, Reflections on 2025, December 30, 2025. The first section, of three, is entitled "The Compute Theory of Everything." Here's an excerpt:

I have come to believe that every engineer must walk the road to Damascus in their own time. One does not simply adopt the Compute Theory of Everything by hearing others discuss it. You have to be viscerally shocked by the pyrotechnics of scale in a domain you know too well to be easily impressed.

For many senior engineers, that shock arrived in 2025. I have watched colleagues who were publicly sceptical through 2023 and 2024 quietly start to integrate these systems into their daily work. The “this is just a stochastic parrot” grimace has been replaced by the “this stochastic parrot just fixed my RE2 regex”. They still say “this can’t do what I do”, but the snorts of laughter have been replaced with a thoughtful silence and the subtle refreshing of their LinkedIn profile.

My own conversion came earlier. It is a privilege of my career that I was working in one of the first fields to get unceremoniously steamrollered by scaling: Computer Vision. During a glorious period at the VGG in Oxford, I spent months crafting bespoke, artisanal architectural inductive biases. They were beautiful, clever, and they had good names. And then, in early 2021, my approach was obliterated by a simple system that worked better because it radically scaled up pretraining compute1. I spent a full afternoon walking around University Parks in shock. But by the time I reached the exit, the shock had been replaced by the annoying zeal of a convert.

Returning to my desk, it did not take long to discover that the Compute Theory of Everything is 50 years old and has been waiting patiently in a Stanford filing cabinet since the Ford administration.

In 1976, Hans Moravec wrote an essay called “The Role of Raw Power in Intelligence“, a document that possesses both the punch and the subtlety of a hand grenade. It is the sort of paper that enters the room, clears its throat, and informs the entire field of Artificial Intelligence that their fly is down. Moravec’s central thesis is that intelligence is not a mystical property of symbol manipulation, but a story about processing power, and he would like to explain this to you, at length, using log scales and a tone of suppressed screaming.

He starts with biology, noting that intelligence has evolved somewhat independently in at least four distinct lineages: in cephalopods, in birds, in cetaceans, and in primates. He spends several pages on the brainy octopus covering the independent evolution of copper-based blood and the neural architecture of the arms, citing a documentary in which an octopus figures out how to unscrew a bottle to retrieve a tasty lobster from inside. One gets the impression he prefers the octopus to many of his colleagues. The evolutionary point is that intelligence is not a fragile accident of primate biology. It is a recurring architectural pattern the universe stumbles upon whenever it leaves a pile of neurons unattended. The octopus and the crow did not copy each other’s homework. Instead, they converged on the answer because the answer works. The question is: what is the underlying resource?

Moravec’s answer is: it’s the compute, stupid.

To make his point, he compares the speed of the human optic nerve (approximately ten billion edge-detection operations per second) to the PDP-10 computers then available at Stanford. The gap is a factor of more than a million. He calls this deficit “a major distorting influence in current work, and a reason for disappointing progress.” He accuses the field of wishful thinking, scientific snobbery, and (my favourite) sweeping the compute deficit under the rug “for fear of reduced funding.” It is the sound of a man who has checked the numbers, realized the Emperor has no clothes, and is particularly annoyed that the Emperor has neither a GPU nor a meaningful stake in God’s Chosen Company: Nvidia (GCCN).

This leads to his aviation aphorism that has become modestly famous, at least among the demographic that reads 1976 robotics working papers for recreational purposes: “With enough power, anything will fly.” Before the Wright brothers, serious engineers built ornithopters (machines that flapped their wings, looked elegant, and stayed resolutely on the ground). Most failed. Some fatally. The consensus was that AI was a matter of knowledge representation and symbolic reasoning, and that people who talked about “raw power” were missing the point and possibly also the sort of people who enjoy watching videos of monster truck rallies (a group that includes your humble author). Moravec’s point was that the Symbolic AI crowd were busy building ornithopters, obsessing over lift-to-drag ratios, while the solution was to strap a massive engine to a plank and give researchers the chance to brute-force the laws of physics into submission.

Twenty-two years later, he published an update. “When Will Computer Hardware Match the Human Brain?“ which opens with a sentence that has aged like a 1998 Pomerol:

“The performance of AI machines tends to improve at the same pace that AI researchers get access to faster hardware.”

He plots curves, whips up a Fermi estimate that human-level cognition requires on the order of 100 million MIPS, and predicts this capability will be available in affordable machines by the 2020s. The paper includes a chart in which various organisms and machines are arrayed by estimated computational throughput. The spider outperforms the nematode by a humiliating margin. Deep Blue appears as a reference point for what IBM’s R&D budget bought you in 1997, which was the ability to defeat Garry Kasparov at chess while remaining unable to recognise a photograph of a chess piece. The figure is instructive, but after staring at it for a few minutes, it can start to grate on one’s sensibilities. Perhaps because it treats the human soul as an arithmetic problem. Philosophy on two axes.

There's more on the compute theory of everything, which is worth your while.

Let me add that, for myself, sure, we need enough compute. That's necessary, but not sufficient. LLMs are a limited architecture. Throwing more compute at them isn't going solve all the problems. I've got a working paper that's relevant: What Miriam Yevick Saw: The Nature of Intelligence and the Prospects for A.I., A Dialog with Claude 3.5 Sonnet. Here's Claude's summary:

1. Miriam Yevick's 1975 work proposed a fundamental distinction between two types of computing: A) Holographic/parallel processing suited for pattern recognition, and B) Sequential/symbolic processing for logical operations. Crucially, she explicitly connected these computational approaches to different types of objects in the world.

2. This connects to the current debate about neural vs. symbolic AI approaches: A) The dominant view suggests brain-inspired (neural) approaches are sufficient. B) Others argue for neuro-symbolic approaches combining both paradigms, and C) Systems like AlphaZero demonstrate the value of hybrid approaches (Monte Carlo tree search for game space exploration, neural networks for position evaluation).

3. The 1988 Benzon/Hays paper formalized “Yevick's Law” showing: A) Simple objects are best represented symbolically, B) Complex objects are best represented holographically, and C) Many real-world problems require both approaches.

4. This framework helps explain: A) Why Chain of Thought prompting works well for math/programming (neural system navigating symbolic space), B) Why AlphaZero works well for chess (symbolic system navigating game tree, neural evaluation of positions), and C) The complementary relationship between these approaches.

5. These insights led to a new definition of intelligence: “The capacity to assign computational capacity to propositional (symbolic) and/or holographic (neural) processes as the nature of the problem requires.”

6. This definition has implications for super-intelligence: A) Current LLMs' breadth of knowledge doesn't constitute true super-intelligence, B) Real super-intelligence would require superior ability to switch between computational modes, and C) This suggests the need for fundamental architectural innovations, not just scaling up existing approaches.

The conversation highlighted how Yevick's prescient insights about the relationship between object types and computational approaches remain highly relevant to current AI development and our understanding of intelligence itself.