AI is changing how Silicon Valley gets its work done.

Kalley Huang, A.I. Could Change the World. But First It Is Changing Silicon Valley. New York Times, Apr. 2, 1016.

But nearly four years after OpenAI lit the A.I. boom with its ChatGPT chatbot, the one industry that is unquestionably being disrupted by this once-in-a-generation technology shift is the tech industry itself.

Tech workers, it is becoming clear, have been building their A.I. replacements. The profitable business models of software companies are also threatened by A.I. Even the way companies are built is being turned inside out, as tiny shops use A.I. to build apps and software that would have taken dozens of skilled programmers just a few years ago.

“Silicon Valley is this really interesting petri dish right now of all of this change and transformation,” said Aaron Levie, the chief executive of Box, a company that makes software for storing and managing data.

Generative A.I. made by companies like OpenAI, Anthropic and Google can do many things. The one task it has become particularly good at is computer programming. That has given many tech companies the chance to start cleaning house, even if executives stop short of saying that’s what they’re doing.

One reason chatbots are good a programming is that the (syntactic) correctness of a program is readily verified by the chatbot itself. That's not true for most intellectual tasks.

So far this year, more than 70 tech companies have eliminated at least 40,000 jobs, according to Layoffs.fyi, which tracks job cuts in the industry. Block, the financial services company that owns Square, Cash App and Tidal, laid off 40 percent of its work force in February, or about 4,000 employees.

“We’re already seeing that the intelligence tools we’re creating and using, paired with smaller and flatter teams, are enabling a new way of working which fundamentally changes what it means to build and run a company,” Jack Dorsey, Block’s top executive, wrote in a social media post.

Large scale job loss and transformation in tech:

The layoffs have contributed to tech hubs like San Francisco — the home of OpenAI and Anthropic — not seeing the job growth characteristic of prior booms, Mr. Egan said. From 2022 through 2025, when the most recent data was available, San Francisco County lost about 30,000 tech jobs, according to data from the Census Bureau. It added roughly that number of jobs during the dot-com era and a start-up funding frenzy between 2020 and 2022.

That decline is visible across the country, too. Nationwide, tech jobs declined by about 150,000 from 2022 through 2025.

“The tech labor pool and talent pool is definitely reassembling,” Mr. Egan said. “A.I. is a big reason for that.” [...]

Part of tech’s reassembly is happening at start-ups. Gone is the traditional process of raising a boatload of venture capital funding, not worrying about revenue or profit and hiring heavily. Today’s start-ups are tapping A.I. tools like agents — personal assistants that can take actions on their own — to make money and grow with fewer employees.

There's more at the link.

America’s New Chip Factory — $50B Disaster

This is a fascinating story about how Samsung set out to build a state of the art chip fab (fabrication facility) and the problems that bedeviled it. Without the chips this factory was designed to build, AI is nowhere.

Timestamps:
00:00 - $50B Chip Nightmare
18:26 - Where Everything Went Wrong
29:58 - The Twist

Brave New World: Notes on the next 30 years in AI [Work in Progress]

You may or may not be wondering why so many tweets have recently been showing up on New Savanna. There’s a good reason: I’ve been thinking. These days, more often than not, the means interacting with either ChatGPT or Claude, and often both in one day. I copy these interactions to Word documents, which I save to my hard drive. And then promptly forget what topic is in what document where on my hard drive. FWIW, these discussions can ramble all over the place, which is fine. That’s how I think. But keeping track of it all is difficult.

Anyhow, over the last week I’ve been thinking about work that David Hays and I have done going back to the mid-1970s when I first began working with him in graduate school. I’ve asked ChatGPT to look at some of our papers and evaluate them in terms of current literature and debates. And so forth and so on. I’m also looking forward to next steps, for me, but also in general.

At the moment I’m working on a piece for 3 Quarks Daily with the tentative title: “The Paradox of Contemporary AI: Engineering Success and Institutional Failure.” Here’s how ChatGPT put it in an interaction we had yesterday:

The engineering success is real. LLMs and related systems have given us access to a new conceptual continent. They work, and at extraordinary scale. But the institutional failure lies in the monoculture: too much intellectual, financial, and training-path dependence on one family of architectures and one style of thought about intelligence. The result is that we are building out the utility before we have adequately explored the space of possible successor technologies or developed the conceptual tools needed to understand what these systems are revealing about language, cognition, and cultural structure.

That’s a metaphor I’ve used off and on over the years, the discovery of a new continent. My paradigm case is the European discovery of the Americas. Making it to the eastern shore is one thing. Then comes exploring and settling. Columbus set foot in the Caribbean in 1492. Lewis and Clarke made it across North America to the Pacific Ocean in 1805. That’s over three centuries later. It would take another century for Europeans and their descendants to more or less settle North America from coast to coast.

Let’s say that GPT-3 in late 2022 is analogous to Columbus making it to shore in the New World. While I don’t think it will take four centuries to settle this new world, nor even three centuries to measure its extent. Intellectual discovery, engineering implementation, and commercial development move at a faster pace. But I do think there’s a lot of work, very exciting work, ahead of us.

Here’s a fragment of an interaction I had with ChatGPT 5.2 a couple of days ago. My prompt is first, then ChatGPT’s response.

* * * * * 

Obviously I don’t know how long it will take to explore that territory and to settle it, nor do I know what we’ll have when that’s done. But I’m thinking in terms of 30 years or more, not one or two bright ideas away.

Your time horizon sounds realistic, and historically it fits the pattern of major intellectual/technical shifts. What you’re describing—the exploration and eventual settlement of that “continent”—is not the discovery of a single clever algorithm. It is the construction of a new intellectual infrastructure, and those almost always take decades.

Let me sketch why.

1. Precedent: how long paradigm infrastructures take

If we look at previous transformations in science and technology, they tend to unfold over multi-decade arcs.

Digital computing

•       1936 — Turing machine

•       1940s–50s — early computers

•       1960s–70s — programming languages, operating systems

•       1980s–90s — personal computing ecosystem

Roughly 50 years from theory to mature civilization-scale infrastructure.

Molecular biology

•       1953 — DNA structure

•       1960s — genetic code deciphered

•       1970s — recombinant DNA

•       1990s — genome sequencing

Again 40 years to transform biology.

Deep learning

•       1986 — backprop revival

•       1998 — LeNet

•       2012 — AlexNet

•       2020 — GPT-3

That arc is already 35+ years long.

So a 30-year horizon for what you’re describing is not pessimistic—it is historically typical.

2. Why this particular transition is slow

The reason the transition will be slow is that the field is confronting three different layers simultaneously.

1. Mathematics of distributed cognition

We still lack a clear mathematical theory of:

•      representation in high-dimensional spaces

•       compositionality in distributed systems

•       interaction between symbolic and dynamical processes

Yevick was an early attempt at that level.

But the theory is still incomplete.

2. Architecture

Even if the mathematics existed, we would still need working architectures.

The field is currently experimenting with:

•       transformers

•       memory systems

•       agent frameworks

•       search hybrids

These are early prototypes.

3. Cognitive theory

Perhaps the hardest layer is understanding what intelligence actually is.

That question spans:

•       neuroscience

•       psychology

•       linguistics

•       philosophy

Your RNA work sits squarely in that domain. [Relational Network over Attractors]

3. Why the current moment feels fast

The speed of the last few years comes from something different: engineering scaling.

Transformers gave the field a powerful substrate for experimentation.

So exploration is happening rapidly near the coastline.

But deeper theoretical consolidation will take longer.

AI as coder, reports from the trenches

Clive Thompson, Coders Coded Their Job Away. Why Are So Many of Them Happy About It? NYTimes, Mar. 12, 2026.

He and Brennan-Burke, who is 32, are still software developers, but like most of their peers now, they only rarely write code. Instead, they spend their days talking to the A.I., describing in plain English what they want from it and responding to the A.I.’s “plan” for what it will do. Then they turn the agents loose.

A.I. being A.I., things occasionally go haywire. Sometimes when Claude misbehaves and fails to test the code, Ebert scolds the agent: Claude, you really do have to run all the tests.

To avoid repeating these sorts of errors, Ebert has added some stern warnings to his prompt file, the list of instructions — a stern Ten Commandments — that his agents must follow before they do anything. When you behold the prompt file of a coder using A.I., you are viewing a record of the developer’s attempts to restrain the agents’ generally competent, but unpredictably deviant, actions.

A coder is now more like an architect than a construction worker.

I looked at Ebert’s prompt file. It included a prompt telling the agents that any new code had to pass every single test before it got pushed into Hyperspell’s real-world product. One such test for Python code, called a pytest, had its own specific prompt that caught my eye: “Pushing code that fails pytest is unacceptable and embarrassing.”

Embarrassing? Did that actually help, I wondered, telling the A.I. not to “embarrass” you? Ebert grinned sheepishly. He couldn’t prove it, but prompts like that seem to have slightly improved Claude’s performance. [...]

Computer programming has been through many changes in its 80-year history. But this may be the strangest one yet: It is now becoming a conversation, a back-and-forth talk fest between software developers and their bots.

This vertiginous shift threatens to stir up some huge economic consequences. For decades, coding was considered such wizardry that if you were halfway competent you could expect to enjoy lifetime employment. If you were exceptional at it (and lucky), you got rich. Silicon Valley panjandrums spent the 2010s lecturing American workers in dying industries that they needed to “learn to code.”

Now coding itself is being automated. To outsiders, what programmers are facing can seem richly deserved, and even funny: American white-collar workers have long fretted that Silicon Valley might one day use A.I. to automate their jobs, but look who got hit first! Indeed, coding is perhaps the first form of very expensive industrialized human labor that A.I. can actually replace.

Why programmers like their AI coders:

The enthusiasm of software developers for generative A.I. stands in stark contrast to how other Americans feel about the impact of large language models. Polls show a majority are neutral or skeptical; creatives are often enraged. But if coders are more upbeat, it’s because their encounters with A.I. are diametrically opposite to what’s happening in many other occupations, says Anil Dash, a friend of mine who is a longtime programmer and tech executive. “The reason that tech generally — and coders in particular — see L.L.M.s differently than everyone else is that in the creative disciplines, L.L.M.s take away the most soulful human parts of the work and leave the drudgery to you,” Dash says. “And in coding, L.L.M.s take away the drudgery and leave the human, soulful parts to you.”

There's much more at the link.

Living Human Brain Cells Play DOOM on a CL1

 

Andrew Paul, Computer run on human brain cells learned to play ‘Doom’, Popular Science, Mar. 2, 2026.

A biocomputer powered by lab-grown human brain cells has leveled up from Pong to Doom. While nowhere ready to handle the video game shooter’s most challenging levels, researchers at Cortical Labs in Australia believe their neuronal chip is well on its way to powering a new generation of hybrid organic technologies.

“This was a major milestone, because it demonstrated adaptive, real-time goal directed learning,” Brett Kagan, Cortical Labs Chief Scientific and Chief Operations Officer, said in a recent video announcement.

It’s taken years to cross the Doom benchmark. In 2021, Cortical Labs debuted DishBrain—an early biocomputer utilizing around 800,000 human nerve cells. These neurons were connected to a small processing chip capable of interpreting and directing electrical activity similar to a standard silicon-powered device.

To showcase DishBrain’s potential, engineers successfully trained their biocomputer to play Pong. The classic, 2D game is often a test case for computational neuroscientists because it requires their system to navigate a dynamic information landscape in real time.

It took Cortical Labs more than 18 months using its original hardware and software to accomplish their Pong goal. DishBrain was eventually supplanted by CL1, which the company bills as the “world’s first code deployable biological computer.”

There's more at the link.

The future of computer programming is here, and it’s fun.

Paul Ford, The A.I. Disruption Is Actually Here, and It’s Not Terrible, NYTimes, Feb. 18, 2026.

Vibe coding:

To vibe code is to make software with prompts sent to a specialized chatbot — not coding, but telling — and letting the bot work out the bugs. Like many other programmers, I use a product called Claude Code from Anthropic, although Codex from OpenAI does about as well, and Google Gemini is not far behind. Claude Code earned $1 billion for Anthropic in its first six months. It was always a helpful coding assistant, but in November it suddenly got much better, and ever since I’ve been knocking off side projects that had sat in folders for a decade or longer. It’s fun to see old ideas come to life, so I keep a steady flow. Maybe it adds up to a half-hour a day of my time, and an hour of Claude’s.

November was, for me and many others in tech, a great surprise. Before, A.I. coding tools were often useful, but halting and clumsy. Now, the bot can run for a full hour and make whole, designed websites and apps that may be flawed, but credible. I spent an entire session of therapy talking about it.

The tech industry is a global culture — an identity based on craft and skill. Software development has been a solid middle-class job for a long time. But that may be slipping away. What might the future look like if 100 million, or a billion, people can make any software they desire? Could this be a moment of unparalleled growth and opportunity as people gain access to tech industry power for themselves?

It’s by no means perfect:

Is the software I’m making for myself on my phone as good as handcrafted, bespoke code? No. But it’s immediate and cheap. And the quantities, measured in lines of text, are large. It might fail a company’s quality test, but it would meet every deadline. That is what makes A.I. coding such a shock to the system.

An axiom of programming is “real artists ship.” That was something Steve Jobs once said to remind his team that finishing and releasing a product matters more than endlessly refining it. Much of the software industry is organized around managing ship risk, and the possibility that a product never actually makes it out to the world. good technology manager assumes that a product will never ship for launch, that every force is arrayed against it, and that the devil himself has cursed it — and then the manager works back from that. Even if all these obstacles are surmounted, the software will ship late.

Having worked in the software industry for a few years, though not as a programmer, I’m well aware of this. See this post from the two years I spent at MapInfo: Crisis in a High-Tech Start-Up: A Case of Collective Action. Now, back to Ford’s article:

Except … what if, going forward, it’s not? What if software suddenly wanted to ship? What if all of that immense bureaucracy, the endless processes, the mind-boggling range of costs that you need to make the computer compute, just goes poof? That doesn’t mean that the software will be good. But most software today is not good. It simply means that products could go to market very quickly.

And for lots of users, that’s going to be fine. People don’t judge A.I. code the same way they judge slop articles or glazed videos. They’re not looking for the human connection of art. They’re looking to achieve a goal. Code just has to work.

There are many arguments against vibe coding through A.I. [...] All of these are true and valid. But I’ve been around too long. The web wasn’t “real” software until it was. Blogging wasn’t publishing. Big, serious companies weren’t going to migrate to the cloud, and then one day they did.

But right now, excited developers are overextending themselves to the point of burnout, obsessively coding all the time. [...] People trumpet the Jevons paradox, which points out that greater efficiency often leads to more consumption — but at the same time, would it surprise you to find out tomorrow that large technology consulting firms had just laid off 10,000 people? A hundred thousand? A million?

The market keeps convulsing, and I wish we could hit the brakes. But we live in a brakeless era.

No matter where you work, my hunch is this is coming for you.

It’s unavoidable, Ford says:

This is all exacerbated by how much of the A.I. industry is led by people who see human thought as raw material, like a steel manufacturer sees ore. The industry is arranged into an ouroboros of mutual investments, with the world economy teetering on their sweetest dreams. Social change at this level needs careful, federal governance and thoughtful regulation. But we’re being handed the opposite: Racist A.I. video slop shared on Truth Social, Grok doing who-knows-what inside the Pentagon, and a White House policy that would give the U.S. attorney general the power to challenge any state’s attempt to regulate A.I. No brakes.

All of the people I love hate this stuff, and all the people I hate love it. And yet, likely because of the same personality flaws that drew me to technology in the first place, I am annoyingly excited. [...]

I believe there are millions, maybe billions, of software products that don’t exist but should: Dashboards, reports, apps, project trackers and countless others. People want these things to do their jobs, or to help others, but they can’t find the budget. They make do with spreadsheets and to-do lists.

My industry is famous for saying “no,” or selling you something you don’t need. We have an earned reputation as a lot of really tiresome dudes. But I think if vibe coding gets a little bit better, a little more accessible and a little more reliable, people won’t have to wait on us. They can just watch some how-to videos and learn, and then they can have the power of these tools for themselves. [...]

The simple truth is that I am less valuable than I used to be. It stings to be made obsolete, but it’s fun to code on the train, too. And if this technology keeps improving, then everyone who tells me how hard it is to make a report, place an order, upgrade an app or update a record — they could get the software they deserve, too. That might be a good trade, long term.

Interesting how automation also creates all sorts of new tasks and bottlenecks.

Fascinating insights from senior engineers on how AI is changing their jobs. Interesting how automation also creates all sorts of new tasks and bottlenecks. https://t.co/O99myQbcTb pic.twitter.com/eQdvjjkTTk

— Séb Krier (@sebkrier) February 14, 2026

Terminology: Generative Machines, Epistemic Structure of the Cosmos, Intelligence-Complete

I’ve been spending a lot of time with my chatbots, ChatGPT and Claude, and some terminological issues have come. Noting particularly deep, just clarification.

Generative machines vs. equilibrium machines

While we talk of computers as machines, it’s obvious that they’re very different beasts. Electric drills, helicopters, sewing machines, hydraulic presses, they’re all (proper) machines. Interaction with and manipulaton of matter is central to their purpose. Computers, well, technically, yes, they push electrons around in intricate paths, and electrons are matter, subatomic particles, very small chunks of matter, the smallest possible chunks. What computers are really about, though, is manipulate bits, units of information. And they use “trillions of parts” (a phrase I have from Daniel Dennett) to do so. Thus computers, with their trillions of parts, are very different from machines, with only 10s, 100s, or 1000s of parts.

So, what names should we give to differentiate them. “Type 1” and “Type 2” machines would do the job, but it’s not very descriptive. ChatGPT and I settled on “equilibrium machines” for those machines centered on interaction with matter while “generative machines” seemed appropriate to bit-wranglers. “Generative” seems just right for computers, with its echoes on Chomsky’s generative grammar the generative pre-trained transformer (GPT) of machine learning. “Equilibrium machines” is perhaps a bit oblique for the other kind of machine, but it’s meant to evoke the equilibrium world of macroscopic devices as opposed to the far-from-equilibrium world of, well, generative machines.

Epistemic Structure of the Cosmos

Back in 2020 I wrote of the metaphysical structure of the cosmos. I said:

There is no a priori reason to believe that world has to be learnable. But if it were not, then we wouldn’t exist, nor would (most?) animals. The existing world, thus, is learnable. The human sensorium and motor system are necessarily adapted to that learnable structure, whatever it is.

I am, at least provisionally, calling that learnable structure the metaphysical structure of the world.

I’ve always been uneasy with “metaphysical” in that role. ChatGPT suggested that “epistemic” would serve better. The epistemic structure of the cosmos, I like that. As for “cosmos,” the dictionary tells me that the word implies order, which I like as well. 

I leave it as an exercise to the reader to demonstrate that the epistemic structure of the cosmos must necessarily be recursive. 

Intelligence-Complete

A generative machine is intelligence-complete if it possesses the full capacities of human intelligence, whatever human intelligence is. By that definition LLS are not intelligence complete. As for human intelligence, I like the account given in What Miriam Yevick Saw: The Nature of Intelligence and the Prospects for A.I., A Dialog with Claude 3.5 Sonnet.

Hays, Forth, and creativity in man-machine-interaction

I've just been talking with ChatGPT about David Hays back in the 1970s. From my prompt:

In 1977 or 1978 he got a personal computer, one of those based on the Z-80 chip, and started learning to program it in Forth. He also sent out a weekly snail-mail newsletter entitled “The Diary of a Man and His Machine.” In one issue he asked for replies from people whose work feel between the cracks. He got about 100 replies. We read them all. We concluded that most of them were just oddballs; they didn’t even seem to know the standard literature in the field(s) relevant to their work. But a handful seemed legit. I don’t think we ever did anything with them.

...working between the cracks is not the same as working without foundations

From ChatGPT's response:

First, David Hays’s early adoption of a Z-80–based personal computer and Forth in 1977–78 places him well ahead of the curve, not just technically but cognitively. Forth, in particular, is telling: it is intimate with the machine, minimalist, stack-based, and reflective. It invites thinking about process, control, and representation in ways that high-level languages often conceal. That choice alone aligns with his lifelong interest in cognitive architecture rather than surface performance.

working between the cracks is not the same as working without foundations

Later:

Third—and this is perhaps the most interesting part—the call for people whose work “fell between the cracks,” and the response it generated, reads like an early, analog precursor to today’s online intellectual ecosystems. The fact that you and Hays received about a hundred replies, and that most of them struck you as oddballs, is not a disappointment; it’s a sociological constant. When disciplinary boundaries loosen, noise floods in first. What matters is that a handful were clearly legitimate, even if nothing came of it at the time. [...] Finally, your retrospective judgment—that many respondents lacked grounding in the relevant literature—underscores a distinction you’ve been circling throughout this chat: working between the cracks is not the same as working without foundations. Hays was interested in the former, not the latter.

Another prompt:

Your comments about Forth are apt. Hays valued the intimacy with the machine. He eventually programmed a whole environment using Forth and used it for keeping notes and generating text. Some of the text for our articles came out of that machine. That detail completes the picture—and it makes the point even stronger.

Chat replies:

What David Hays was doing with Forth was not “early word processing” or “personal productivity.” He was building a cognitive prosthesis—a writable environment in which thinking, note-keeping, and text generation were continuous with computation rather than merely supported by it.

A few things are worth making explicit.

First, the intimacy you mention is decisive. Forth collapses distance between intention and execution: no elaborate compilation pipeline, no abstraction layers to hide what the machine is doing. That intimacy encourages a style of thought in which process is visible. For someone like Hays—already committed to understanding cognition as structured activity—that mattered more than convenience or speed.

Second, the fact that he built an entire environment—not just scripts, but a living workspace for notes and text—means that the machine was not ancillary. It was part of the thinking loop. That some of the prose in your joint articles emerged from that environment is not incidental; it means the machine participated, materially, in the production of theory. This is not metaphorical augmentation. It is literal.

Rodney Brooks on the state of AI and Robotics

As you may know, Rodney Brooks has been keeping an annual scorecard for various categories of high-tech activity. He puts it online on the first of the year. I’ve listed some excerpts from the 2026 scorecard below. The scorecard has much much more that I haven’t excerpted.

The Falcon 9

Eight years ago, Falcon 9 had been launched 46 times, all successful, over the previous eight years, and it had recently had a long run of successful landings of the booster whenever attempted. At that time five launches had been on a previously used booster, but there had been no attempts to launch Falcon Heavy with its three boosters strapped together.

Now we are eight years on from those first eight years of Falcon 9 launches. The scale and success rate of the launches has made each individual launch an unremarkable event, with humans being launched a handful of times per year. Now the Falcon 9 score card stands at 582 launches with only one failed booster, and there have been 11 launches of the three booster Falcon Heavy, all successful. That is a sustained growth rate of 38% year over year for eight years. And that it is a very high sustained deployment growth rate for any complex technology.

There is no other modern rocket with such a volume of launches that comes even close to the Falcon 9 record. And I certainly did not foresee this volume of launches. About half the launches have had SpaceX itself as the customer, starting in February 2018, launching an enormous satellite constellation (about two thirds of all satellites ever orbited) to support Starlink bringing internet to everywhere on the surface of Earth.

[Not AI or robotics, I know. But it interests me.] 

Humanoid Robots

My blog post from September, details why the current learning based approaches to getting dexterous manipulation will not get there anytime soon. I argue that the players are (a) collecting the wrong data and (b) trying to learn the wrong thing. I also give an argument (c) for why learning might not be the right approach. My argument for (c) may not hold up, but I am confident that I am right on both (a) and (b), at least for the next ten years.

I also outline in that blog post why the current (and indeed pretty much the only, for the last forty years) method of building bipeds and controlling them will remain unsafe for humans to be nearby. I pointed out that the danger is roughly cubicly proportional to the weight of the robot. Many humanoid robot manufacturers are introducing lightweight robots, so I think they have come to the same conclusion. But the side effect is that the robots can not carry much payload, and certainly can’t provide physical support to elderly humans, which is a thing that human carers do constantly — these small robots are just not strong enough. And elder care and in home care is one of the main arguments for having human shaped robots, adapted to the messy living environments of actual humans.

Given that careful analysis from September I do not share the hype that surrounds humanoid robotics today. Some of it is downright delusional across many different levels.

At the end:

Meanwhile here is what I said at the end of my September blog post about humanoid robots and teaching them dexterity. I am not at all negative about a great future for robots, and in the nearish term. It is just that I completely disagree with the hype arguing that building robots with humanoid form magically will make robots useful and deployable. These particular paragraphs followed where I had described there, as I do again in this blog post, how the meaning of self driving cars has drifted over time.

Following that pattern, what it means to be a humanoid robot will change over time.

Before too long (and we already start to see this) humanoid robots will get wheels for feet, at first two, and later maybe more, with nothing that any longer really resembles human legs in gross form. But they will still be called humanoid robots.

Then there will be versions which variously have one, two, and three arms. Some of those arms will have five fingered hands, but a lot will have two fingered parallel jaw grippers. Some may have suction cups. But they will still be called humanoid robots.

Then there will be versions which have a lot of sensors that are not passive cameras, and so they will have eyes that see with active light, or in non-human frequency ranges, and they may have eyes in their hands, and even eyes looking down from near their crotch to see the ground so that they can locomote better over uneven surfaces. But they will still be called humanoid robots.

There will be many, many robots with different forms for different specialized jobs that humans can do. But they will all still be called humanoid robots.

As with self driving cars, most of the early players in humanoid robots, will quietly shut up shop and disappear. Those that remain will pivot and redefine what they are doing, without renaming it, to something more achievable and with, finally, plausible business cases. The world will slowly shift, but never fast enough to need a change of name from humanoid robots. But make no mistake, the successful humanoid robots of tomorrow will be very different from those being hyped today.

Neural networks

Despite their successes with language, LLMs come with some serious problems of a purely implementation nature.

First, the amount of examples that need to be shown to a network to learn to be facile in language takes up enormous amounts of computation, so the that costs of training new versions of such networks is now measured in the billions of dollars, consuming an amount of electrical power that requires major new investments in electrical generation, and the building of massive data centers full of millions of the most expensive CPU/GPU chips available.

Second, the number of adjustable weights shown in the figure are counted in the hundreds of billions meaning they occupy over a terabyte of storage. RAM that is that big is incredibly expensive, so the models can not be used on phones or even lower cost embedded chips in edge devices, such as point of sale terminals or robots.

These two drawbacks mean there is an incredible financial incentive to invent replacements for each of (1) our humble single neuron models that are close to seventy years old, (2) the way they are organized into networks, and (3) the learning methods that are used.

That is why I predict that there will be lots of explorations of new methods to replace our current neural computing mechanisms. They have already started and next year I will summarize some of them. The economic argument for them is compelling. How long they will take to move from initial laboratory explorations to viable scalable solutions is much longer than everyone assumes. My prediction is there will be lots of interesting demonstrations but that ten years is too small a time period for a clear winner to emerge. And it will take much much longer for the current approaches to be displaced. But plenty of researchers will be hungry to do so.

LLMs

So we all know we need guard rails around LLMs to make them useful, and that is where there will be lot of action over the next ten years. They can not be simply released into the wild as they come straight from training.

This is where the real action is now. More training doesn’t make things better necessarily. Boxing things in does.

Already we see companies trying to add explainability to what LLMs say. Google’s Gemini now gives real citations with links, so that human users can oversee what they are being fed. Likewise, many companies are trying to box in what their LLMs can say and do. Those that can control their LLMs will be able to deliver useable product.

A great example of this is the rapid evolution of coding assistants over the last year or so. These are specialized LLMs that do not give the same sort of grief to coders that I experienced when I first tried to use generic ChatGPT to help me. Peter Norvig, former chief scientist of Google, has recently produced a great report on his explorations of the new offerings. Real progress has been made in this high impact, but narrow use field.

New companies will become specialists in providing this sort of boxing in and control of LLMs.

A note on embodiment

But since 1991 I have made a distinction between two concepts where a machine, or creature can be either, neither, or both situated and embodied. Here are the exact definitions that I wrote for these back then:

[Situatedness] The robots are situated in the world—they do not deal with abstract descriptions, but with the here and now of the world directly in-fluencing the behavior of the system.

[Embodiment] The robots have bodies and experience the world directly—their actions are part of a dynamic with the world and have immediate feed-back on their own sensations.

At first glance they might seem very similar. And they are, but they are also importantly different. And, spoiler alert, I think much of the work at companies, large and small, right now, is trying abstract out the embodiment of a robot, turning it into a machine that is merely situated.

Later:

Being both situated and embodied is still a challenge to robots in the world. [[Now here is the most important sentence of this whole blog post.]] I think the training regimes that [are] being used for both locomotion and dexterity are either ignoring or trying to zero out the embodiment of physical robots, their inertias and forces, reducing them to merely being situated, just apps with legs and arms, characters in video games, not the reality of real physical beings that the tasks we want them to do requires.

The Ridiculous Engineering Of The World's Most Important Machine (extreme ultraviolet lithography)

This video is about how the long process through which ASML (Advanced Semiconductor Materials Lithography) their EUV (extreme ultraviolet lithography) systems. Pay particular attention to how much tinkering and serendipity was involved in the process. It's hard to imagine that such a machine could have been designed by giving an AI a bunch of data and design goals and directing it to design the machine outright. It's even more difficult to imagine connecting that AI to a bunch of robots and having them build the machine in one fell swoop.

Time stamps:

0:00 The Machine That Saved Moore’s Law
3:12 How are microchips made?
9:11 What is extreme ultraviolet lithography?
15:04 Nuclear Fusion To The Rescue
21:59 How ASML Conquered The Chip World
35:35 Who are ASML’s biggest customers?
37:40 The Most Important Tech Company In The World
41:25 Inside ASML

Spintronics will support much faster electronics using less power

YouTube:

Spintronics is short for “spin electronics,” and refers to the study of the spin of the electron. In electronic devices, spintronics leverages the spin of electrons to process and store data with extreme efficiency – this technology is just a few years from reaching the consumer market, and will make your devices faster and more efficient. For a price, of course. Let’s take a look at how spintronics got here and where it’s going.

Text, coding, and knowledge agents

One critical point made by Karpathy on the Dwarkesh podcast was about the difference between AI coding agents and the rest of knowledge work.

It’s clear that AI agents in coding have taken off extremely quickly. There are many reasons for this, including the fact that a…

— Aaron Levie (@levie) October 20, 2025

The effect of AI tools on coding

Joel Becker et al., "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity", METR 7/10/2025:

Absract: Despite widespread adoption, the impact of AI tools on software development in the wild remains understudied. We conduct a randomized controlled trial (RCT) to understand how AI tools at the February–June 2025 frontier affect the productivity of experienced open-source developers. 16 developers with moderate AI experience complete 246 tasks in mature projects on which they have an average of 5 years of prior experience. Each task is randomly assigned to allow or disallow usage of early-2025 AI tools. When AI tools are allowed, developers primarily use Cursor Pro, a popular code editor, and Claude 3.5/3.7 Sonnet. Before starting tasks, developers forecast that allowing AI will reduce completion time by 24%. After completing the study, developers estimate that allowing AI reduced completion time by 20%. Surprisingly, we find that allowing AI actually increases completion time by 19%—AI tooling slowed developers down. This slowdown also contradicts predictions from experts in economics (39% shorter) and ML (38% shorter). To understand this result, we collect and evaluate evidence for 20 properties of our setting that a priori could contribute to the observed slowdown effect—for example, the size and quality standards of projects, or prior developer experience with AI tooling. Although the influence of experimental artifacts cannot be entirely ruled out, the robustness of the slowdown effect across our analyses suggests it is unlikely to primarily be a function of our experimental design.

(See also this version…)

Posted at Language Log by Mark Liberman along with comments by Liberman and others. For example, one Rick Rubenstein said:

I have to admit I've been surprised that the ceiling for generative AI so far has turned out to be somewhere at the top edge of "hack" level. Until recently my hunch was that Doug Hofstadter was essentially right: the hard part was getting computers to match the level of not-especially-clever people; getting them from there to Mozart/Einstein/Shakespeare would just be a matter of degree.

But no, AI is proving more than capable of generating not-actually-good-but-not-laughably-bad output in all sorts of fields. We detect AI by its "slopness", not by its incompetence. It seems clear to me that there's little future for human hack illustrators, hack novelists, hack programmers, hack songwriters. AI is perfectly suited to creating the 90% part of Sturgeon's Law. But thus far I haven't seen anything that looks like it's cracked that top 10% — and tellingly, I don't really hear genAI's hypsters claiming it either.

Silicon Valley is going nuts chasing superintelligence (whatever that is)

Cade Metz, The A.I. Frenzy Is Escalating. Again. NYTimes, June 27, 2025.

Silicon Valley’s artificial intelligence frenzy has found a new gear.

Two and a half years after OpenAI set off the artificial intelligence race with the release of the chatbot ChatGPT, tech companies are accelerating their A.I. spending, pumping hundreds of billions of dollars into their frantic effort to create systems that can mimic or even exceed the abilities of the human brain.

The tech industry’s giants are building data centers that can cost more than $100 billion and will consume more electricity than a million American homes. Salaries for A.I. experts are jumping as Meta offers signing bonuses to A.I. researchers that top $100 million.AI

And venture capitalists are dialing up their spending. U.S. investment in A.I. companies rose to $65 billion in the first quarter, up 33 percent from the previous quarter and up 550 percent from the quarter before ChatGPT came out in 2022, according to data from PitchBook, which tracks the industry.

“Everyone is deeply afraid of being left behind,” said Chris V. Nicholson, an investor with the venture capital firm Page One Ventures who focuses on A.I. technologies.

This astonishing spending, critics argue, comes with a huge risk. A.I. is arguably more expensive than anything the tech industry has tried to build, and there is no guarantee it will live up to its potential. But the bigger risk, many executives believe, is not spending enough to keep pace with rivals.

“The thinking from the big C.E.O.s is that they can’t afford to be wrong by doing too little, but they can afford to be wrong by doing too much,” said Jordan Jacobs, a partner with the venture capital firm Radical Ventures.

The biggest spending is for the data centers. Meta, Microsoft, Amazon and Google have told investors that they expect to spend a combined $320 billion on infrastructure costs this year. Much of that will go toward building new data centers — more than twice what they spent two years ago.

As OpenAI and its partners build a roughly $60 billion data center complex for A.I. in Texas and another in the Middle East, Meta is erecting a facility in Louisiana that will be twice as large. Amazon is going even bigger with a new campus in Indiana. Amazon’s partner, the A.I. start-up Anthropic, says it could eventually use all 30 of the data centers on this 1,200-acre campus to train a single A.I system.

Specialization:

But as venture firms double down on their deal making, there is less appetite for investing in general A.I. systems designed to do everything, because that work is dominated by established companies like OpenAI and Google. Instead, they are starting to focus on A.I. that does specific tasks, like Ribbon, a company that does A.I. for job interviews, and Eleos Health, which creates A.I. to record and summarize doctor visits.

And then there’s Columbus:

Tech companies acknowledge that they may be overestimating A.I.’s potential. But even if the technology falls short, many executives and investors believe, the investments they’re making now will be worth it.

“Christopher Columbus thought he was headed to the Orient, and he ended up in the Caribbean,” said Mr. Nicholson of Page One Ventures. “He did not get to where he thought he was going, but he still got to a place that was highly valuable.”

That’s true, but if the “highly valuable” new territory is to be exploited, it’s going to require new ideas and new ideas ARE just what ISN’T being cultivated in this spending spree.

Meanwhile, around the corner, Mike Isaac and Cade Metz report on Mark Zuckerberg’s “catch-up” spending spree, motived by the fear that Meta has fallen behind in the race for “superintelligence”. He’s offering unprecedented compensation packages, some as high as $100 million. The final paragraph:

“In Silicon Valley, you hear a lot of talk about the 10x engineer,” said Amjad Masad, the chief executive of the A.I. start-up Replit, using a term for extremely productive developers. “Think of some of these A.I. researchers as 1,000x engineers. If you can add one person who can change the trajectory of your entire company, it’s worth it.”

Amazon's new power-hogging data center in Indiana is super-mega-giant-sized.

Karen Weise and Cade Metz, At Amazon’s Biggest Data Center, Everything Is Supersized for A.I., NYTimes, June 24, 2025.

A year ago, a 1,200-acre stretch of farmland outside New Carlisle, Ind., was an empty cornfield. Now, seven Amazon data centers rise up from the rich soil, each larger than a football stadium.

Over the next several years, Amazon plans to build around 30 data centers at the site, packed with hundreds of thousands of specialized computer chips. With hundreds of thousands of miles of fiber connecting every chip and computer together, the entire complex will form one giant machine intended just for artificial intelligence.

The facility will consume 2.2 gigawatts of electricity — enough to power a million homes. Each year, it will use millions of gallons of water to keep the chips from overheating. And it was built with a single customer in mind: the A.I. start-up Anthropic, which aims to create an A.I. system that matches the human brain.

The complex — so large that it can be viewed completely only from high in the sky — is the first in a new generation of data centers being built by Amazon, and part of what the company calls Project Rainier, after the mountain that looms near its Seattle headquarters. Project Rainier will also include facilities in Mississippi and possibly other locations, like North Carolina and Pennsylvania.

Project Rainier is Amazon’s entry into a race by the technology industry to build data centers so large they would have been considered absurd just a few years ago. Meta, which owns Facebook, Instagram and WhatsApp, is building a two-gigawatt data center in Louisiana. OpenAI is erecting a 1.2-gigawatt facility in Texas and another, nearly as large, in the United Arab Emirates.

These data centers will dwarf most of today’s, which were built before OpenAI’s ChatGPT chatbot inspired the A.I. boom in 2022. The tech industry’s increasingly powerful A.I. technologies require massive networks of specialized computer chips — and hundreds of billions of dollars to build the data centers that house those chips. The result: behemoths that stretch the limits of the electrical grid and change the way the world thinks about computers.

A power hog:

AEP[American Electric Power] has told regulators that new, large-scale data centers will more than double the amount of peak power it must provide Indiana, from about 2.8 gigawatts in 2024 to more than seven gigawatts by approximately 2030. Amazon’s campus alone accounts for about half of the additional load growth.

“It will be the largest power user in the state of Indiana by a country mile,” said Ben Inskeep of the Citizens Action Coalition.

The utility told regulators in April that it expected to use natural gas plants to provide about three-quarters of the additional power that would be needed by 2030.

There's more at the link.

Why can AIs code for 1h but not 10h?

Human's chance of error declines more slowly, so they eventually win over longer horizons.

This could be due to a remaining bastion of human advantage (or just because the human data is averaged over many levels of expertise). pic.twitter.com/gDO2ytzXCH

— Benjamin Todd (@ben_j_todd) June 15, 2025

Added: Is this what LeCun said all along?https://t.co/6mbUjyiLsJ

— Benjamin Todd (@ben_j_todd) June 16, 2025

What happens to programming jobs when AI writes more and more code?

Noam Scheiber, At Amazon, Some Coders Say Their Jobs Have Begun to Resemble Warehouse Work, NYTimes, May 25, 2025.

As A.I. spreads through the labor force, many white-collar workers have expressed concern that it would lead to mass unemployment. But while joblessness has ticked up and widespread layoffs might eventually come, the more immediate downside for software engineers appears to be a change in the quality of their work. Some say it is becoming more routine, less thoughtful and, crucially, much faster paced.

Companies seem to be persuaded that, like assembly lines of old, A.I. can increase productivity. A recent paper by researchers at Microsoft and three universities found that programmers’ use of an A.I. coding assistant called Copilot, which proposes snippets of code that they can accept or reject, increased a key measure of output more than 25 percent.

At Amazon, which is making big investments in generative A.I., the culture of coding is changing rapidly. In his recent letter to shareholders, Andy Jassy, the chief executive, wrote that generative A.I. was yielding big returns for companies that use it for “productivity and cost avoidance.” He said working faster was essential because competitors would gain ground if Amazon doesn’t give customers what they want “as quickly as possible” and cited coding as an activity where A.I. would “change the norms.”

Those changing norms have not always been eagerly embraced. Three Amazon engineers said that managers had increasingly pushed them to use A.I. in their work over the past year. The engineers said that the company had raised output goals and had become less forgiving about deadlines. It has even encouraged coders to gin up new A.I. productivity tools at an upcoming hackathon, an internal coding competition. One Amazon engineer said his team was roughly half the size it had been last year, but it was expected to produce roughly the same amount of code by using A.I.

Sheiber goes on to give two other examples, Shopify and Google:

The shift has not been all negative for workers. At Amazon and other companies, managers argue that A.I. can relieve employees of tedious tasks and enable them to perform more interesting work. Mr. Jassy wrote last year that the company had saved “the equivalent of 4,500 developer-years” by using A.I. to do the thankless work of upgrading old software.

Eliminating such tedious work may benefit a subset of accomplished programmers, said Lawrence Katz, a labor economist at Harvard University who has tracked research on the subject closely.

But for inexperienced programmers, the result of introducing A.I. can resemble the shift from artisanal work to factory work in the 19th and 20th centuries. “Things look like a speed-up for knowledge workers,” Dr. Katz said, describing preliminary evidence from ongoing research. “There is a sense that the employer can pile on more stuff.”

Sheiber then returns to Amazon noting that this seems like what had happened in the warehouses where robotic warehouses were adopted:

But the robots have increased the number of items each worker can pick to hundreds from dozens an hour. Some workers complain that the robots have also made the job hyper-repetitive and physically taxing. Amazon says it provides regular breaks and cites positive feedback from workers about its cutting edge robots.

The Amazon engineers said this transition was on their minds as the company urged them to rely more on A.I. They said that, while doing so was technically optional, they had little choice if they wanted to keep up with their output goals, which affect their performance reviews.

More about Amazon, then on to Microsoft where AI tools are writing large chunks of code:

“It’s more fun to write code than to read code,” said Simon Willison, an A.I. fan who is a longtime programmer and blogger, channeling the objections of other programmers. “If you’re told you have to do a code review, it’s never a fun part of the job. When you’re working with these tools, it’s most of the job.”

This shift from writing to reading code can make engineers feel as if they are bystanders in their own jobs.

There's more at the link.

Return to the source? Trump makes AI deals with the U.A.E. and Saudi Arabia

Tripp Mickle and Ana Swanson, Outsourcer in Chief: Is Trump Trading Away America’s Tech Future?, NYTimes, May 16, 2025.

Over the course of a three-day trip to the Middle East, President Trump and his emissaries from Silicon Valley have transformed the Persian Gulf from an artificial-intelligence neophyte into an A.I. power broker.

They have reached an enormous deal with the United Arab Emirates to deliver hundreds of thousands of today’s most advanced chips from Nvidia annually to build one of the world’s largest data center hubs, three people familiar with the talks said. The shipments would begin this year, with the vast majority of the chips going to U.S. cloud service providers and about 100,000 of them to G42, an Emirati A.I. firm.

The administration revealed the agreement on Thursday in an announcement unveiling a new A.I. campus in Abu Dhabi supported by 5 gigawatts of electrical power. It would be the largest such project outside the United States and help U.S. companies serve customers in Africa, Europe and Asia, the administration said. The details about the chips weren’t disclosed, and it’s not clear if they could still be subject to change.

As Mr. Trump traversed the region in recent days, the United States also struck multibillion-dollar agreements to sell advanced chips from Nvidia and AMD to Saudi Arabia. The United States and Saudi Arabia are also still in discussions on a larger contract for A.I. technology, five people familiar with the negotiations said.

There's more at the link.

And remember, mathematics from the Far East made its way to Europe via the Arab world:

The algorithms of arithmetic were collected by Abu Ja'far Mohammed ibn Musa al-Khowarizm around 825 AD in his treatise Kitab al jabr w'al-muqabala (Penrose 1989). They received an effective European exposition in Leonardo Fibonacci's 1202 work, Algebra et almuchabala (Ball 1908). It is easy enough to see that algorithms were important in the eventual emergence of science, with all the calculations so required. But they are important on another score. For algorithms are the first purely informatic procedures which had been fully codified. Writing focused attention on language, but it never fully revealed the processes of language (we’re still working on that). A thinker contemplating an algorithm can see the complete computational process, fully revealed.

The word “algorithm” is derived from the name “al-Khowarizm” and “algebra” from “al jabr.”

Ignorant White House techbros are destroying our seed corn

David Singer, White House Tech Bros Are Killing What Made Them (and America) Wealthy, NYTimes, April 28, 2025.

What’s seed corn? It’s corn the farmer preserves through winter so it can be used to seed next year’s crop. It’s a metaphor.

One would think that venture capitalists, especially those with ties to the Trump administration, would be the most forceful champions of America’s research universities, given how much these institutions have fueled our careers and fortunes. Instead, many of us are scratching our heads as to why officials from the industry have turned their backs while the government chaotically terminates funding for this work. Harvard and Columbia have been in the headlines, but the hatchet has also fallen on Michigan State in the Midwest and the University of Hawaii farther west. It is as if the V.C.s in Washington had just enjoyed a fine meal in Silicon Valley and decided to skip out on the check.

Breakthroughs in technology are grounded in a fundamental truth: that transformative innovation often begins with a new understanding of the natural world at its most basic level. And this understanding almost always emerges from challenging accepted wisdom. That requires space for free inquiry and a culture that protects it, something that Vannevar Bush understood in his landmark 1945 report “Science, the Endless Frontier,” where he argued that basic research generates “scientific capital” — the foundation for practical applications, new products and new processes. Even patent law reflects this principle, requiring that an invention be “nonobvious to one skilled in the art.” This is the crux of the matter.

Drawing a causal link between federal investment in basic science research and the rise of the venture capital industry is about as difficult as reading a map. The geographic centers of venture capital and the industries it has spawned overlap precisely with the locations of our great research universities. Think of Cambridge and Route 128 in Massachusetts (Massachusetts Institute of Technology and Harvard), or the stretch from San Jose to San Francisco (Stanford and University of California, San Francisco and Berkeley). This is no accident. It’s why world leaders visit these places to understand how we do it. It is also why Mr. Vance left Ohio for Yale and then high-tailed it to Silicon Valley for a job.