Into the digital wilderness and beyond LLMs

Alexander Doria has a recent post, The Model is the Product, that’s gotten me to think about future developments in a different way:

There were a lot of speculation over the past years about what the next cycle of AI development could be. Agents? Reasoners? Actual multimodality?

I think it’s time to call it: the model is the product.

All current factors in research and market development push in this direction.

• Generalist scaling is stalling. This was the whole message behind the release of GPT-4.5: capacities are growing linearly while compute costs are on a geometric curve. Even with all the efficiency gains in training and infrastructure of the past two years, OpenAI can’t deploy this giant model with a remotely affordable pricing.
• Opinionated training is working much better than expected. The combination of reinforcement learning and reasoning means that models are suddenly learning tasks. It’s not machine learning, it’s not base model either, it’s a secret third thing. It’s even tiny models getting suddenly scary good at math. It’s coding model no longer just generating code but managing an entire code base by themselves. It’s Claude playing Pokemon with very poor contextual information and no dedicated training.
• Inference cost are in free fall. The recent optimizations from DeepSeek means that all the available GPUs could cover a demand of 10k tokens per day from a frontier model for… the entire earth population. There is nowhere this level of demand. The economics of selling tokens does not work anymore for model providers: they have to move higher up in the value chain.

This is also an uncomfortable direction. All investors have been betting on the application layer. In the next stage of AI evolution, the application layer is likely to be the first to be automated and disrupted.

He then goes on to explain why he’s saying that. He then says:

...the current training ecosystem is very tiny. You can count all theses companies on your hands: Prime Intellect, Moondream, Arcee, Nous, Pleias, Jina, the HuggingFace pretraining team (actually tiny)… Along with a few more academic actors (Allen AI, Eleuther…) they build and support most of the current open infrastructure for training. In Europe, I know that at least 7-8 LLM projects will integrate the Common Corpus and some of the pretraining tools we developed at Pleias — and the rest will be fineweb, and likely post-training instruction sets from Nous or Arcee.

There is something deeply wrong in the current funding environment. Even OpenAI senses it now. Lately, there was some felt irritation at the lack of “vertical RL” in the current Silicon Valley startup landscape. I believe the message comes straight from Sam Altman and will likely result in some adjustment in the next YC batch but pinpoint to a larger shift: soon the big labs select partners won’t be API customers but associated contractors involved in the earlier training stage.

If the model is the product, you cannot necessarily build it alone. Search and code are easy low hanging fruits: major use cases for two years, the market is nearly mature and you can ship a new cursor in a few months. Now many of the most lucrative AI uses cases in the future are not at this advanced stage of development — typically, think about all these rule based system that still rule most of the world economy… Small dedicated teams with a cross-expertise and a high level of focus may be best positioned to tackle this—eventually becoming potential acquihire once the initial ground work is done. We could see the same pipeline in the UI side. Some preferred partner, getting exclusive API access to close specialized models, provided they get on the road for business acquisition.

Note I have been arguing that LLMs are a digital wilderness. In post from December 29, 2022, not long after ChatGPT was released, I quoted from an article by Ted Underwood:

In his penultimate paragraph Underwood notes:

I have suggested that approaching neural models as models of culture rather than intelligence or individual language use gives us even more reason to worry. But it also gives us more reason to hope. It is not entirely clear what we plan to gain by modeling intelligence, since we already have more than seven billion intelligences on the planet. By contrast, it’s easy to see how exploring spaces of possibility implied by the human past could support a more reflective and more adventurous approach to our future. I can imagine a world where generative models of culture are used grotesquely or locked down as IP for Netflix. But I can also imagine a world where fan communities use them to remix plot tropes and gender norms, making “mass culture” a more self-conscious, various, and participatory phenomenon than the twentieth century usually allowed it to become.

These digital wildness regions thus represent opportunities for discovery and elaboration. Alignment is simply one aspect of that process.

And by alignment I mean more than aligning the AI’s values with human values; I mean aligning its conceptual structure as well. That’s where “old school” symbolic computing enters the picture, especially language. Language – not the mere word forms available in digital corpora, but word forms plus semantics and syntactic affordances – is one of the chief ‘tools’ through which young humans are acculturated and through which human communities maintain their beliefs and practices. The full powers of language, as treated by classical symbolic systems, will be essential for “domesticating” the digital wilderness and developing it for human use.

What do you do with a wildness? You explore it, map it, enclose parts of it, and then develop them. That’s a job for “small dedicated teams with a cross-expertise and a high level of focus.”

I returned to the wilderness them in the report I recently posted on the year and-a-half I spent investigating ChatGPT: ChatGPT: Exploring the Digital Wilderness, Findings and Prospects. There I said:

To return to the metaphor with which I began this report, these LLMs, these so-called Foundation Models, they are a digital wildness. Wild and untamed, but rich in resources. Ongoing esearch in mechantistic interpretatility is one way to explore that wildnerness, to map it. I have been presenting a different, a complementary mode of exploration and mapping. We need to map the territory, settle it, and domesticate it. By that I mean develop symbolic models to operate on and in the territory. While we may start developing those models through hand-coding, we will have to develop programmatic techniques if we want to cover the territory, which, of course, will be ever expanding.

As far as I can tell, Doria is imagining that his small focused teams will be working within existing architectural and programmatic frameworks or obvious extensions of them. I’m imagining something a bit different, something that requires us to understand the internal operations of LLMs. Yet, I have little trouble imagining that here and there one or more of his small focused teams will begin making sense of those internal operations beyond that displayed in current interpretability research. Perhaps they will lay the foundations for the kind of programmatic techniques I am imagining.

H/t Tyler Cowen.

On the explicit construction of cognitive ontology: From “salt” to “sodium chloride”

I have long used the conceptual difference between “salt” and “sodium chloride” to illustrate the idea of conceptual ontology. Except for impurities in (samples of) salt, they are the same thing. But conceptually they are quite different. Salt hardly needs any formal definition at all; it's a basic taste and a common physical substance. But sodium chloride is expressed in conceptual terms that weren’t fully developed until the 19th century.

Lately I’ve been wondering what would be required to develop cognitive accounts ontological concepts in a rich and full way. I’ve been talking about conceptual ontology using the idea of the Great Chain of Being. But I’ve always thought of that as a stand-in for a more thorough treatment, one I’ve never gotten around to. What would that more thorough treatment be like?

It's fairly obvious what we need to do with salt. It’s a white granular substance. We know how to do that sort of thing with tools invented back in the 1970s and 1980s. Nor should there by much difficulty it explicitly accounting for texture, taste, and whatever odor there is. But what about sodium chloride?

That’s not so clear. Oh, there’s been lots of work on formal ontologies for informatic purposes. John Sowa has worked on this, and Barry Smith’s website has lots of material. But that’s not quite what I had in mind.

For example, chemical experimentation typically involves weighing substances very carefully. In the 18th and 19th centuries they might have used a mechanical analytical balance something like this one:

Photo by Sarcyn, licensed under a CCA by-SA 3.0 Unported License.

Such balances would have been used in the experiments used to identify the chemical elements, such as sodium and chlorine, identified in modern atomic theory. Since that is the case, the construction and operation of such balances is part of the conceptual web that supports the concept, /chemical element/, as is the mathematical used in analyzing these experiments. I wouldn’t expect that construction and operation to be directly implicated in the definition of chemical element, but there would be an explicit traceable linkage between the definition and that constructure and those operations. There would also be traceable links to reports in formal journals. Those reports would have specific weights and calculations, etc.

THAT’s the kind of thing I have in mind when I talk about a “thorough treatment” of conceptual ontology. On the one hand we have the sensorimotor processes involved in make observations and conducting experiments. That’s at the bottom layer, if you will, the foundation, of this cognitive constructure. Those objects and processes are going to be bound into complex patterns over which abstractions are made and those abstractions will end up as the terms directly involved in, in this case, 19th century atomic theory and its elaboration in chemistry.

I’m pretty sure that, if you ask your favorite chatbot about these things, it will tell you about salt, sodium chloride, sodium and chlorine, atoms and elements, analytical balances, solutions, gases, arithmetic, and so forth and so on. All of that stuff is there. But I haven’t the foggiest idea of what kinds of connections are latent in the model. It is by no means that all of the connections implied in my previous paragraph would be there in LLMs.

Thus, when I talk about LLMs as digital wilderness, I am implying that it is there to be explored, mapped, and ultimately “domesticated.” What do I mean by domestication? I mean development a rich and full symbolic cognitive account of some intellectual domain. In order to do that, we’re going to need to know how LLMs work internally. That’s just the beginning. I figure different intellectual communities will take responsibility for different regions of the digital wilderness. Getting the whole thing domesticated? That’s the work of intellectual generations. The idea that one day we’ll achieve the magical AGI which will then lead to AI-takeoff in which everything is all worked out in a matter of hours, days, weeks, or months at the most, that’s pure foolishness.

The NYTimes is doing better on next-word prediction, much better [Here be dragons.]

Cade Metz, Why Do A.I. Chatbots Tell Lies and Act Weird? Look in the Mirror. NYTimes, Feb 26, 2023.

After some setting the stage, including introducing Terry Sejnowski, Metz tells us this:

As it analyzes that sea of good and bad information from across the internet, an L.L.M. learns to do one particular thing: guess the next word in a sequence of words.

It operates like a giant version of the autocomplete technology that suggests the next word as you type out an email or an instant message on your smartphone. Given the sequence “Tom Cruise is a ____,” it might guess “actor.”

As you might imagine, my initial reaction was: No! No! No! A thousand times NO! and then some. After all, I’ve been objecting to this characterization for a while now. After I’d peeled myself from the ceiling I decided to click on that link, and found something interesting, indeed.

This a bit of animation, which I can’t show here. It is showing us next-word generation starting with this question: Who is LeBron James? The AI has already replied: LeBron James is an American professional... The animation steps through the addition of new words and, shows us the shifting probability distribution as it moves along. We get these short paragraphs as we move down the page:

When artificial intelligence software like ChatGPT writes, it considers many options for each word, taking into account the response it has written so far and the question being asked.

It assigns a score to each option on the list, which quantifies how likely the word is to come next, based on the vast amount of human-written text it has analyzed.

ChatGPT, which is built on what is known as a large language model, then chooses a word with a high score, and moves on to the next one.

The model’s output is often so sophisticated that it can seem like the chatbot understands what it is saying — but it does not.

Every choice it makes is determined by complex math and huge amounts of data. So much so that it often produces text that is both coherent and accurate.

Yes! This is just the kind of thing we need. And we need more of it. Much more. And it gets better as we go on. 

“This is terra incognita,” Dr. Sejnowski said. “Humans have never experienced this before.”

Let’s go back to Cade Metz. He continues on:

When you chat with a chatbot, the bot is not just drawing on everything it has learned from the internet. It is drawing on everything you have said to it and everything it has said back. It is not just guessing the next word in its sentence. It is guessing the next word in the long block of text that includes both your words and its words.

The longer the conversation becomes, the more influence a user unwittingly has on what the chatbot is saying. If you want it to get angry, it gets angry, Dr. Sejnowski said. If you coax it to get creepy, it gets creepy.

The alarmed reactions to the strange behavior of Microsoft’s chatbot overshadowed an important point: The chatbot does not have a personality. It is offering instant results spit out by an incredibly complex computer algorithm.

I suppose that makes sense even if you either knew about shifting probability distributions already or if you linked out to that wonderful animation. But the more you know, the better.

Metz goes on:

But there’s a caveat to this reassurance: Because chatbots are learning from so much material and putting it together in such a complex way, researchers aren’t entirely clear how chatbots are producing their final results. Researchers are watching to see what the bots do and learning to place limits on that behavior — often, after it happens.

Microsoft and OpenAI have decided that the only way they can find out what the chatbots will do in the real world is by letting them loose — and reeling them in when they stray. They believe their big, public experiment is worth the risk.

That’s how it is for now. And, if you’ve played with ChatGPT you know that have the opportunity to give a thumbs up or thumbs down to each response. OpenAI is, of course, logging that information and using it to further tune ChatGPT.

I have come to think of these large so-called Foundation Models as the digital wilderness. We need to explore it, stake out claims, clear the land, and start building on it. Ultimately, “building on it” is going to mean linking it to a world model. We’re not there yet, alas.

Here's the last words in the article:

“This is terra incognita,” Dr. Sejnowski said. “Humans have never experienced this before.”

YES! As I said in the title of a working paper I did two and a half years ago, Here be Dragons!

There’s more at the link.

Thoughts on the implications of GPT-3, two years ago and NOW [here be dragons, we're swimming, flying and talking with them]

When GPT-3 first came out, I registered my first reactions in a comment at Marginal Revolution, which I've appended immediately below the picture of Gojochan and Sparkychan. I'm currently completing a working paper about my interaction with ChatGPT. That will end with an appendix in which I repeat my remarks from two years ago and append some new ones. I've appended those after the comment to Marginal Revolution.

* * * * *

A bit revised from a comment I made at Marginal Revolution:

Yes, GPT-3 [may] be a game changer. But to get there from here we need to rethink a lot of things. And where that's going (that is, where I think it best should go) is more than I can do in a comment.

Right now, we're doing it wrong, headed in the wrong direction. AGI, a really good one, isn't going to be what we're imagining it to be, e.g. the Star Trek computer.

Think AI as platform, not feature (Andreessen). Obvious implication, the basic computer will be an AI-as-platform. Every human will get their own as an very young child. They're grow with it; it'll grow with them. The child will care for it as with a pet. Hence we have ethical obligations to them. As the child grows, so does the pet – the pet will likely have to migrate to other physical platforms from time to time.

Machine learning was the key breakthrough. Rodney Brooks' Gengis, with its subsumption architecture, was a key development as well, for it was directed at robots moving about in the world. FWIW Brooks has teamed up with Gary Marcus and they think we need to add some old school symbolic computing into the mix. I think they're right.

Machines, however, have a hard time learning the natural world as humans do. We're born primed to deal with that world with millions of years of evolutionary history behind us. Machines, alas, are a blank slate.

The native environment for computers is, of course, the computational environment. That's where to apply machine learning. Note that writing code is one of GPT-3's skills.

So, the AGI of the future, let's call it GPT-42, will be looking in two directions, toward the world of computers and toward the human world. It will be learning in both, but in different styles and to different ends. In its interaction with other artificial computational entities GPT-42 is in its native milieu. In its interaction with us, well, we'll necessarily be in the driver's seat.

Where are we with respect to the hockey stick growth curve? For the last 3/4 quarters of a century, since the end of WWII, we've been moving horizontally, along a plateau, developing tech. GPT-3 is one signal that we've reached the toe of the next curve. But to move up the curve, as I've said, we have to rethink the whole shebang.

We're IN the Singularity. Here be dragons.

[Superintelligent computers emerging out of the FOOM is bullshit.]

* * * * *

ADDENDUM: A friend of mine, David Porush, has reminded me that Neal Stephenson has written of such a tutor in The Diamond Age: Or, A Young Lady's Illustrated Primer (1995). I then remembered that I have played the role of such a tutor in real life, The Freedoniad: A Tale of Epic Adventure in which Two BFFs Travel the Universe and End up in Dunkirk, New York.

* * * * *

To the future and beyond!

I stand by those remarks from two years ago, but I want to comment on four things: 1) AI alignment, 2) the need for symbolic computing, 3) the need for new kinds of hardware, and 4) a future world in which humans and AIs interact freely.

Considerable effort has gone into tuning ChatGPT so that it won’t say things that are offensive (e.g. racial slurs) or give out dangerous information (e.g. how to hotwire cars). These efforts have not been entirely successful. This is one aspect of what is now being called “AI alignment.” In the extreme the field of AI alignment is oriented toward the possibility – which some see as a certainty – that in the future (somewhere between, say, 30 and 130 years) rogue AIs will wage a successful battle against humankind.[1] I don’t think that fear is very creditable, but, as the rollout of ChatGPT makes abundantly clear, AIs built on deep learning are unpredictable and even, in some measure, uncontrollable.

I think the problem is inherent in deep learning technology. Its job is to fit a model to, in the case of ChatGPT, an extremely large corpus of writing, much of the internet. That corpus, in turn, is ultimately about the world. The world is vast, irregular, and messy. That messiness is amplified by the messiness inherent in the human brain/mind, which did, after all, evolve to fit that world. Any AI engine capable of capturing a significant portion of the order inherent in our collective writing about the world has no choice but to encounter and incorporate some of the disorder and clutter into its model as well.

I regard such Foundation models[2], as they have come to be called, as wilderness preserves, digital wilderness. They contain what digital humanist Ted Underwood calls the latent space of culture. He says:

The immediate value of these models is often not to mimic individual language understanding, but to represent specific cultural practices (like styles or expository templates) so they can be studied and creatively remixed. This may be disappointing for disciplines that aspire to model general intelligence. But for historians and artists, cultural specificity is not disappointing. Intelligence only starts to interest us after it mixes with time to become a biased, limited pattern of collective life. Models of culture are exactly what we need.

In his penultimate paragraph Underwood notes:

I have suggested that approaching neural models as models of culture rather than intelligence or individual language use gives us even more reason to worry. But it also gives us more reason to hope. It is not entirely clear what we plan to gain by modeling intelligence, since we already have more than seven billion intelligences on the planet. By contrast, it’s easy to see how exploring spaces of possibility implied by the human past could support a more reflective and more adventurous approach to our future. I can imagine a world where generative models of culture are used grotesquely or locked down as IP for Netflix. But I can also imagine a world where fan communities use them to remix plot tropes and gender norms, making “mass culture” a more self-conscious, various, and participatory phenomenon than the twentieth century usually allowed it to become.

These digital wildness regions thus represent opportunities for discovery and elaboration. Alignment is simply one aspect of that process.

And by alignment I mean more than aligning the AI’s values with human values; I mean aligning its conceptual structure as well. That’s where “old school” symbolic computing enters the picture, especially language. Language  – not the mere word forms available in digital corpora, but word forms plus semantics and syntactic affordances –  is one of the chief ‘tools’ through which young humans are acculturated and through which human communities maintain their beliefs and practices. The full powers of language, as treated by classical symbolic systems, will be essential for “domesticating” the digital wilderness and developing it for human use.

However, this presents technical problems, problems I cannot go into here in any detail.[4] The basic issue is that symbolic computing involves one strategy for implementing, call it cogitation, in a physical system while the neural computing underlying deep learning requires a different physical implementation. These approaches are incompatible. While one can “bolt” a symbolic system onto a neural computing system, that strikes me as no more than an interim solution. It will get us started, indeed the work has already begun.[5]

What we want, though, is for the symbolic system to arise from the neural system, organically, as it does in humans.[6] This may well call for fundamentally new physical platforms for computing, platforms based on “neuromorphic” components that are “grown,” as Geoffrey Hinton has recently remarked.[7] That technology will give us a whole new world, one where humans, AIs and robots interact freely with one another, but will have communities of their own as well. We know that dogs co-evolved with humans over tens of thousand of years. These miraculous new devices will co-evolve with us over the coming decades and centuries.

Let us end with Miranda’s words from Shakespeare’s The Tempest:

“Oh wonder!
How many goodly creatures are there here!
How beauteous mankind is! Oh brave new world,
That has such [devices] in’t.”

* * * * *

[1] The virtual center of this belief is a website called LessWrong, which has extensive discussion of this issue going back well over a decade. Here it is, https://www.lesswrong.com/.

[2] Foundation models, Wikipedia, https://en.wikipedia.org/wiki/Foundation_models.

[3] Ted Underwood, Mapping the latent spaces of culture, The Stone and the Shell, Oct. 21, 2021, https://tedunderwood.com/2021/10/21/latent-spaces-of-culture/.

[4] I discuss this issue in this blog post, Physical constraints on computing, process and memory, Part 1 [LeCun], New Savanna, July 24, 2022, https://new-savanna.blogspot.com/2022/07/physical-constraints-on-computing.html.

[5] Consult the Wikipedia entry, Neuro-symbolic AI, for some pointers, https://en.wikipedia.org/wiki/Neuro-symbolic_AI.

[6] I discuss this in a recent working paper, Relational Nets Over Attractors, A Primer: Part 1, Design for a Mind, Version 2, Working Paper, July 13, 2022, pp. 76, https://www.academia.edu/81911617/Relational_Nets_Over_Attractors_A_Primer_Part_1_Design_for_a_Mind.

[7] Tiernan Ray, We will see a completely new type of computer, says AI pioneer Geoff Hinton, ZDNET, December 1, 2022, https://www.zdnet.com/article/we-will-see-a-completely-new-type-of-computer-says-ai-pioneer-geoff-hinton-mortal-computation/#ftag=COS-05-10aaa0j.

We’ve stepped over the threshold into the Fourth Arena, but don’t recognize it

First there was the world of inanimate matter, the First Arena. Life arose from that, the Second Arena. Only 100s of thousands of years ago humans evolved from higher primates and the Third Arena, human culture, appeared. We are now on the threshold of the Fourth Arena. How do we characterize it?

Note: These thoughts are off the top of my head [thinking-out-loud]. It was all I could do to get them out. That’s enough for now. Refinement will have to wait.

* * * * *

OpenAI released GPT-3 in the summer of 2020. It was clear to me that, yes, it has that potential. I registered my response, initially in a comment over at Marginal Revolution, and then on New Savanna, First thoughts on the implications of GPT-3 [here be dragons, we're swimming and flying with them]. Having explored ChatGPT for the past two weeks, that potential emerges before me with even greater clarity. Of course, we could blow it, nothing is guaranteed. But still...

One must wonder, dream, and hope.

Foundation Models as digital wilderness

Let us start with these deep learning models trained on large bodies of data. ChatGPT has such a model at its functional core. They have been termed Foundation Models because they “can be adapted to a wide range of downstream tasks.” I have come to think of each such models as repositories of digital wilderness.

What do we do with the wilderness? We explore it, map it, in time settle it and develop it. We cultivate and domesticate it. AI safety researchers call that alignment. The millions of people who have been using ChatGPT are part of that process. We may not think of ourselves in that way, but that, in part, is how OpenAI thinks of us. Even as we pursue our own ends while interacting with ChatGPT, OpenAI is collecting those sessions and will be using them to fine-tune the system, to align it.

Yes, it would be nice to have a system “pre-aligned” before it is released to end users. But I don’t think that’s how things are going to work out. The process by which these engines are trained on large amounts of data is powerful, but it is also messy. While some alignment can be achieved by a small in-house team tweaking the system, ultimately it will have to interact with a much larger body of users. Why not think of alignment as a process concomitant with use?

What are the business implications of a technology where the end users will inevitably playing an important role in fine-tuning and evolving the technology they are using?

Robots and the physical world

One problem that has come up is that of embodiment. These Foundation mMdels, these tracts of digital wilderness, have no access to the real world. Language models are trained on texts, visual models are trained on images. Neither have the capacity to interact with the world.

In an essay he wrote shortly after he took a position as VP of AI for Halodi Robotics, Eric Jang wrote:

Reality has a surprising amount of detail, and I believe that embodied humanoids can be used to index that all that untapped detail into data. Just as web crawlers index the world of bits, humanoid robots will index the world of atoms. If embodiment does end up being a bottleneck for Foundation Models to realize their potential, then humanoid robot companies will stand to win everything.

Those robots will thus be creating more digital wilderness. But also helping to develop it, to align it.

Symbolic Systems and alignment

One issue that has come up is that of the role of the Old School technologies of symbolic systems. Are they obsolete or, on the contrary, will the remain important? While there is wide-spread sentiment that they are obsolete, and that the technology can achieve its fullest flowering simply by scaling up machine learning, I disagree, nor am I alone.

The question is how we are going to integrate symbolic technology with machine learning. One problem is that, traditionally, such systems have been developed “by hand.” The code must be crafted by people who are experts in the application domain. This is costly and time-consuming.

Off-hand I don’t what can be done. Yes, work is being done with hybrid systems where symbolic technology is grafted on to and underlying base of machine learning. I would like to see symbolic capabilities emerge from the underlying artificial neural net technology – perhaps analogous to the way in which language develops in humans, something I’ve discussed in my recent working paper, Relational Nets Over Attractors, A Primer: Part 1, Design for a Mind, Version 2.

I note, though, that I regard the development of symbolic technology as a stream in the overall process of aligning software grounded in vast plots of digital wilderness. It will thus be a gradual process distributed widely thoughout the community of users.

AI as platform, Adept

Two years ago venture capitalist Mark Andreesen talked of AI as a platform, not a feature. He said:

I think that the deeper answer is that there’s an underlying question that I think is an even bigger question about AI that reflects directly on this, which is: Is AI a feature or an architecture? Is AI a feature, we see this with pitches we get now. We get the pitch and it’s like here are the five things my product does, right, points one two three four five and the, oh yeah, number six is AI, right? It’s always number six because it’s the bullet that was added after they created the rest of the deck. Everything is gonna’ kind of have AI sprinkled on it. That’s possible.

We are more believers in a scenario where AI is a platform, an architecture. In the same sense that the mainframe was an architecture or the minicomputer is an architecture, the PC, the internet, the cloud has an architecture. We think AI is the next one of those. And if that’s the case, when there’s an architecture shift in our business, everything above the architecture gets rebuilt from scratch. Because the fundamental assumptions about what you’re building change. You’re no longer building a website or you’re no longer building a mobile app, you’re no longer building any of those things. You’re building an AI engine that is, in the ideal case, just giving you're the answer to whatever the question is. And if that’s the case then basically all applications will change. Along with that all infrastructure will change. Basically the entire industry will turn over again the same way it did with the internet, and the same way it did with mobile and cloud and so if that’s the case then it’s going to be an absolutely explosive....

I agree, and believe that Adept: Useful General Intelligence, is moving in that direction. From their blog:

In practice, we’re building a general system that helps people get things done in front of their computer: a universal collaborator for every knowledge worker. Think of it as an overlay within your computer that works hand-in-hand with you, using the same tools that you do. We all have parts of our job that energize us more than others – with Adept, you’ll be able to focus on the work you most enjoy and ask our model to take on other tasks. For example, you could ask our model to “generate our monthly compliance report” or “draw stairs between these two points in this blueprint” – all using existing software like Airtable, Photoshop, an ATS, Tableau, Twilio to get the job done together. We expect the collaborator to be a good student and highly coachable, becoming more helpful and aligned with every human interaction.

This product vision excites us not only because of how immediately useful it could be to everyone who works in front of a computer, but because we believe this is actually the most practical and safest path to general intelligence. Unlike giant models that generate language or make decisions on their own, ours are much narrower in scope–we’re an interface to existing software tools, making it easier to mitigate issues with bias. And critical to our company is how our product can be a vehicle to learn people’s preferences and integrate human feedback every step of the way.

Judging from what they’ve written, they’re not quite where Andreesen’s conception is, but they’re moving in that direction. All you have to do is put AI at the heart of the application, rather than treating it as an add-on, and you’re there.

Robot Toys, an exercise for the reader

Do some research on the state of robotic toys and companions. Start with, say, the 1990s Tamagotchi, which wasn’t a robot at all, but a little gadget one had to care for. Do a web search on “robot toys.” Think about the Tamagotchi, think about children and dolls/action figures, and think about those robot toys. Take what you find and insert it hear. Perhaps ChatGPT can help you.

Robots and Humans, living and working together

In 1995 Neil Stephenson, published The Diamond Age: Or, A Young Lady's Illustrated Primer. It centers on a young girl, Nell, who is given a precious book, A Young Lady's Illustrated Primer, which she has with her as she grows up. It serves her as both a tutor and a companion.

That’s where we’re headed. But I have no idea when we’ll get there, a century, two, three? Who knows.

At a very young age each child will be given such a book, or perhaps a robot – why not both? – which will stay with them for the rest of their life, functioning variously as a companion, tutor, and workmate, their personal robot companion (PRC).

At the moment time of their third birthday. By this time the child has plenty of experience getting around physically and is getting better with speaking. They will have seen other kids with their PRCs and no doubt have interacted with them. They’ll know that, when the time comes, they’ll be getting one too.

The robot will have to be of an appropriate size, a book too for that matter. I will have to be replaced at the appropriate age. That will require a ritual, as did the original gifting of the PRC, and/or book.

We will evolve toward a society where robots, AIs, and people will be constantly interacting with one another. There where will communities of mixed groups, others of only one kind of being. I have no idea what that will be like. But it does sound like the Fourth Arena will be deeply entrenched in that world.

Beyond AGI, super-intelligence, and the Singularity

What about AGI (artificial general intelligence)? What about it? Originally it was simply AI, and it was right about the corner. But, alas, it really wasn’t. “AGI” was coined in the first decade of the millennium to revivify those old hopes.

While I followed work in AI back in the day, I did it out of a sense of professional obligation. I didn’t think it was going anywhere. The very different AI that we’ve got now IS going somewhere. It’s time to ditch those old dreams in favor, both of current reality and what we can build in the near-term future, and of new dreams.

I feel much the same about the idea of super-intelligence. Oh, I know how the word was used. I could follow the conversations. But I don’t think there is anything there, no substantial conceptual development.

I feel the same about the idea of a Technological Singularity. AGI recursively rewriting its own code until FOOM! super-intelligence? Nah. Not going to happen. Someone termed it The Rapture for Nerds. That sounds about right.

No, something else is afoot. We’re living on the cusp of the Fourth Arena. What could be grander?

Why I’m just now seeing a quickening in The Force (in lit crit, #DH)

A short time ago I posted the following tweet:

But you know, I've been observing the scene for a LONG TIME, and I sense a disturbance in the Force, a Quickening.

— Bill Benzon (@bbenzon) September 11, 2017

As you can see, I was replying to Ted Underwood, who was in turn replying in a thread initiated by Max Kemman:

"If [historians] do not wake up soon to the new realities of big data, computer scientists will leave [them] behind" https://t.co/kpJeRaF9yy pic.twitter.com/RzqU6JvH5k

— Max Kemman (@MaxKemman) September 8, 2017

But what do I mean by a quickening in The Force?

What I mean is that, at long last, we may see fundamental changes in the academic study of literature. Well, that’s not quite right. And getting it right will not be easy. New things have been appearing for awhile, after all, and calls for change are not at all new. But it’s not just any new things and any new calls that move me.

Let me explain.