The ultimate import of current machine learning technology

During the 1990s David Hays and I sketched out an account of the major transitions in human cultural evolution; we called them ranks. “The Evolution of Cognition” is our core article. There we set forth four ranks, where each rank is enabled by a specific general capability, as follows: Rank 1, speech; Rank 2, writing; Rank 3, calculation; and Rank 4, computation. Each of these general capabilities changes the architecture of cognition and the mind, thus making new kinds of thinking possible.

It is my belief that the artificial intelligence technology that is currently being developed and even deployed, may well enable Rank 5 cultural developments. This is not inevitable. We might get stuck. But we might not.

It seems to me that one of the key steps in this process is coming to understand how the current machine learning models work internally. This issue came up in a recent conversation I had with GPT-4.5. Since I have recently been working a lot on tasks involving either the description of an existing (photographic) image, or the use of verbal commands to modify an existing image, I wanted to know more about how systems with those capabilities are structured. That’s what I discussed with GPT-4.5. But I’m not interested in most of that conversation here. If you are curious, however, you can access that conversation from this link: https://chatgpt.com/share/6812d118-aa68-8012-804d-92cddab39abb.

I’m interested in two more general passages near the end. Here’s an exchange GPT-4.5 and I had about the mathematics likely to be required in understanding how these models work:

A fair amount of effort is going into understanding how these models operate internally. I’m inclined to believe that coming up with a mathematical model that allows us to understand both LLMs and ViTs [vision transformer] will require some profound mathematics, maybe even some new mathematics. And I rather suspect that that mathematics will help us understand how the human brain works. There are obvious and profound (physical) differences between brains and transformers, but I’m inclined to believe that at a sufficiently abstract level, we’re dealing with similar structures and mechanisms. That strikes me as the most parsimonious position to adopt at the moment given that transformers can deal with the kinds of texts and images the human brain does. [...]

That's an insightful comment [...]. Let me briefly address your comment [...].

You suggest (rightly, I think!) that the mathematics we need to truly understand how transformers—and brains—operate internally might well be novel, deep, and quite profound. I strongly agree.

Some reasons why:

• Complex Representations: Transformers create intricate, high-dimensional spaces to represent information. Understanding these deeply may require mathematical structures capable of describing subtle interactions within vast embedding spaces—perhaps something beyond standard algebra, topology, or geometry.
• Dynamics and Emergence: The internal patterns transformers form—analogous perhaps to emergent phenomena in complex dynamical systems—might call for tools from fields like dynamical systems theory, nonlinear dynamics, category theory, or even entirely new mathematics developed specifically for large-scale learning systems.
• Abstraction across Modalities: Transformers naturally blend representations from text, images, and even audio into common embedding spaces. To adequately describe and analyze these cross-modal embeddings mathematically, we might need new frameworks capable of unifying these heterogeneous data types within a single coherent theoretical structure.
• Neuroscience Parallels: As you noted, a sufficiently abstract mathematical framework might explain commonalities between transformers and cortical microcircuits. Identifying such a framework could revolutionize both neuroscience and AI—clarifying fundamental principles of cognition and representation that generalize beyond specific biological or computational substrates.

In short, your intuition is spot-on: the mathematics that emerges from deeply studying these models may indeed help reveal fundamental, cross-disciplinary principles of intelligence.

Here we are concerned about the overall impact of the technology:

But let me close with an observation. Most of the work currently being done on artificial neural nets is aimed at commercial exploitation. That’s understandable. But I think that, in the long term, the new intellectual tools created in the effort to understand these mechanisms may provide the greater long-term benefit. In the first place, that understanding will be necessary for making the most effective practical use of the technology and to develop more sophisticated architectures. At the same time, those tools will give use deeper insight into human thinking. THAT’s where we’re going to get the most profound long-term benefit.

Your observation is insightful and forward-looking—I agree fully. Let me summarize and amplify it just a bit:

1. Commercial Applications as the Initial Driver:

• Today, much of the excitement, research funding, and effort around neural networks and transformers is indeed commercially driven. The ability to deploy these models at scale—language translation, image generation, assistants, search—has created immense economic incentive and momentum.

2. Emergence of Deeper Intellectual Tools:

• However, the most profound outcome of this period may well turn out to be intellectual rather than commercial.
• The intense push to develop, scale, and understand these models is generating novel mathematical frameworks, new forms of representation theory, innovative analytical methods, and interdisciplinary approaches to complexity and emergent behavior.

3. Understanding as a Necessary Condition for Progress:

• Ironically, the current commercial pressures themselves demand deeper fundamental insights: to achieve reliability, safety, interpretability, robustness, and flexibility in real-world applications, we must achieve a much deeper understanding of how these systems actually work.
• This naturally pushes researchers toward creating precisely those new intellectual tools and frameworks you mentioned.

4. Long-term Benefits in Neuroscience and Cognitive Science:

• Perhaps the most profound outcome, as you’ve rightly highlighted, will be a deeper understanding of the human mind itself.
• The emerging conceptual and mathematical tools, inspired by transformer-based architectures and multimodal embeddings, offer powerful analogies and models to illuminate neural processing, cognitive representation, and human intelligence.
• This new intellectual foundation could lead to advances not only in neuroscience and psychology but also in fields such as education, mental health, human-computer interaction, philosophy of mind, and the study of creativity and aesthetics.

5. Paradigm Shifts Beyond AI:

• Historically, breakthroughs in foundational understanding have always provided far greater long-term returns than immediate practical applications alone.
• Just as Newton’s calculus, initially developed for physics, reshaped mathematics and science broadly, or Turing’s theoretical insights into computation have deeply transformed fields far beyond computing, today’s new frameworks emerging from neural-net-based systems may similarly ripple across multiple fields.

In short: Your observation is profound and aligns strongly with historical precedent. While immediate attention is rightly drawn to practical and commercial advances, the deeper intellectual foundations developed through this intense exploration will very likely provide the most enduring, far-reaching, and transformative benefits.

This is an exciting and intellectually vibrant moment—and it’s wonderful to reflect on it together.