Dwarkesh's problems with LLMs: They can't learn! [the problem's architectural]

Dwarkesh Patel, Why I have slightly longer timelines than some of my guests, Dwarkesh Podcast, June 2, 2025.

Sometimes people say that even if all AI progress totally stopped, the systems of today would still be far more economically transformative than the internet. I disagree. I think the LLMs of today are magical. But the reason that the Fortune 500 aren’t using them to transform their workflows isn’t because the management is too stodgy. Rather, I think it’s genuinely hard to get normal humanlike labor out of LLMs. And this has to do with some fundamental capabilities these models lack.

I like to think I’m “AI forward” here at the Dwarkesh Podcast. I’ve probably spent over a hundred hours trying to build little LLM tools for my post production setup. And the experience of trying to get them to be useful has extended my timelines. I’ll try to get the LLMs to rewrite autogenerated transcripts for readability the way a human would. Or I’ll try to get them to identify clips from the transcript to tweet out. Sometimes I’ll try to get it to co-write an essay with me, passage by passage. These are simple, self contained, short horizon, language in-language out tasks - the kinds of assignments that should be dead center in the LLMs’ repertoire. And they’re 5/10 at them. Don’t get me wrong, that’s impressive.

But the fundamental problem is that LLMs don’t get better over time the way a human would. The lack of continual learning is a huge huge problem. The LLM baseline at many tasks might be higher than an average human’s. But there’s no way to give a model high level feedback. You’re stuck with the abilities you get out of the box. You can keep messing around with the system prompt. In practice this just doesn’t produce anything even close to the kind of learning and improvement that human employees experience.

The reason humans are so useful is not mainly their raw intelligence. It’s their ability to build up context, interrogate their own failures, and pick up small improvements and efficiencies as they practice a task.

There’s a reason why LLMs can’t learn, their architecture. In a standard digital computer we have a clean separation between processing and memory. Consequently it’s easy to add more memories to the system. Just find a chunk of “blank tape,” or attach or splice some “blank tape” in, and then pur your new memories there. While LLMs are themselves implemented on a standard (so-called von Neuman) architecture, LLM itself is quite different. Within the LLM memory and processing are not separate, making it very difficult to add new memories. Where do they go? How are they connected to existing memories? Those are profound issues. How does the brain deal with them? Like so much else about the brain, we don’t know. [Note: I discuss these issues in various posts on the Structured Physical Systems Hypothesis, which are tagged: SPSH.]

Dwarkesh continues:

LLMs actually do get kinda smart and useful in the middle of a session. For example, sometimes I’ll co-write an essay with an LLM. I’ll give it an outline, and I’ll ask it to draft the essay passage by passage. All its suggestions up till 4 paragraphs in will be bad. So I’ll just rewrite the whole paragraph from scratch and tell it, “Hey, your shit sucked. This is what I wrote instead.” At that point, it can actually start giving good suggestions for the next paragraph. But this whole subtle understanding of my preferences and style is lost by the end of the session.

Maybe the easy solution to this looks like a long rolling context window, like Claude Code has, which compacts the session memory into a summary every 30 minutes. I just think that titrating all this rich tacit experience into a text summary will be brittle in domains outside of software engineering (which is very text-based).

Note that in such cases we are not dealing with memory that has “settled” or “been distilled into the weights.” This is a live memory; it’s fluid. It’s for this reason that I’ve come to think of the brain as a polyviscous fluid. It’s all fluid, all the time. As Walter Freeman once pointed out to me, any living neuron is spiking; that’s what it means for a neuron to be alive. But inactive neurons simply don’t spike as often as fully active ones do. Hence, polyviscosity. Some regions of the fluid are very viscous, others not at all, with all gradations in between.

But the brain is an organic system, consisting mostly of water, which is held by somewhat permeable membranes (neuron walls), with varying concentrations of trace substances (neuotransmitters). How can we create such polyviscosity in a silicon-based system?

Dwarkesh continues:

If AI progress totally stalls today, I think [less than] 50% of white collar employment goes away. Sure, many tasks will get automated. Claude 4 Opus can technically rewrite autogenerated transcripts for me. But since it’s not possible for me to have it improve over time and learn my preferences, I still hire a human for this. Without progress in continual learning, I think we will be in a substantially similar position with white collar work - yes, technically AIs might be able to do a lot of subtasks somewhat satisfactorily, but their inability to build up context will make it impossible to have them operate as actual employees at your firm.

While this makes me bearish on transformative AI in the next few years, it makes me especially bullish on AI over the next decades. When we do solve continuous learning, we’ll see a huge discontinuity in the value of the models.

That, I submit, will require new architectures based on a different physical substrate. Who’s currently working on producing that substrate? I’m sure than someone is, though I can’t cough up a name at the moment, but how much funding do they have? I’ll tell you: Not nearly enough. 

 There’s much more at the link.