OpenAI's Codex demo [machine learning in its natural environment]

Interesting. A major weakness of these various machine learning systems is that they have, at best, limited access to the real world, and so they cannot learn in the context in which humans ordinarily learn. We're born primed to deal with that world with millions of years of evolutionary history behind us. Machines, alas, are a blank slate. The obvious place to apply machine learning, then, would be the world of computation. That, after all, is the NATURAL world for computers, their native environment. That, apparently, is what Codex has done.

As I noted last year, when giving my initial quick take on GPT-3:

Think AI as platform, not feature (Andreessen). Obvious implication, the basic computer will be an AI-as-platform. Every human will get their own as an very young child. They're grow with it; it'll grow with them. The child will care for it as with a pet. Hence we have ethical obligations to them. As the child grows, so does the pet – the pet will likely have to migrate to other physical platforms from time to time. [...]

So, the AGI of the future, let's call it GPT-42, will be looking in two directions, toward the world of computers and toward the human world. It will be learning in both, but in different styles and to different ends. In its interaction with other artificial computational entities GPT-42 is in its native milieu. In its interaction with us, well, we'll necessarily be in the driver's seat.

Where are we with respect to the hockey stick growth curve? For the last 3/4 quarters of a century, since the end of WWII, we've been moving horizontally, along a plateau, developing tech. GPT-3 is one signal that we've reached the toe of the next curve. But to move up the curve, as I've said, we have to rethink the whole shebang.

We're IN the Singularity. Here be dragons.

My first thoughts on AI-as-platform, though I didn't use that term, can be found in a working paper containing ideas dating back to January of 2006, PowerPoint Assistant: Augmenting End-User Software through Natural Language Interaction, https://www.academia.edu/14329022/PowerPoint_Assistant_Augmenting_End_User_Software_through_Natural_Language_Interaction. Here's the abstract:

This document sketches a natural language interface for end user software, such as PowerPoint. Such programs are basically worlds that exist entirely within a computer. Thus the interface is dealing with a world constructed with a finite number of primitive elements. You hand-code a basic language capability into the system, then give it the ability to ‘learn’ from its interactions with the user, and you have your basic PPA.