Beyond AGI (Artificial General Intelligence)

Tyler Cowen recently posted "Holden Karnofsky emails me on transformative AI" over at Marginal Revolution.

I made two comments. One was similar to my recent post on Rodney Brooks, but much shorter – see this Brooks post as well. I'm posting the other one here as a place-holder. I may or may not elaborate it into a more substantial post at some future time.

Just what IS transformative AI? And AGI, what's that? Or superintelligence? They're highly abstract ideas that can easily be decked out in a grand way.

I've been reading around and about in the material at and linked to that "most important century" page. I certainly haven't read it all. It's interesting stuff. But, for better or worse, I find that Kim Stanley Robinson's science fiction novel, New York 2150, presents a more tangible and believable picture of the future. I'm not saying I agree with it, much less like it, only that it feels more credible to me than a world in which, for example, completely digital people are zipping around in a big pile of compute sometime later in this century.

Now, here's a page listing mostly recent open access articles about robotics and AI that have been published in the Nature family of journals. I've not read any of them nor even the abstracts. My remarks are offered solely on the basis of these titles:

A wireless radiofrequency-powered insect-scale flapping-wing aerial vehicle.
Enhancing optical-flow-based control by learning visual appearance cues for flying robots.
Highly accurate protein structure prediction with AlphaFold.
Advancing mathematics by guiding human intuition with AI.
A mobile robotic chemist.
Neuro-inspired computing chips.
Designing neural networks through neuroevolution.
Deep learning robotic guidance for autonomous vascular access.

I'm pretty sure that none of them assert that they've achieved AGI and I'd be surprised if any of them even hinted that AGI will be popping up in mid-century. I see no reason why these kinds of things won't keep coming and coming and coming. Some will eventuate in practical technologies. Some, likely most, won't. But, what's the likelihood that the cumulative result will be transformative, without, however, producing such wonders as fully digital people?

You might want to take a look at Michael Jordan, "Artificial Intelligence — The Revolution Hasn’t Happened Yet", Medium 4/19/2018. He suggests that beyond what Jordan calls "human imitative AI" (such as digital people) we should recognize "Intelligence Augmentation" and "Intelligent Infrastructure." He concludes:

Moreover, we should embrace the fact that what we are witnessing is the creation of a new branch of engineering. The term “engineering” is often invoked in a narrow sense — in academia and beyond — with overtones of cold, affectless machinery, and negative connotations of loss of control by humans. But an engineering discipline can be what we want it to be.

In the current era, we have a real opportunity to conceive of something historically new — a human-centric engineering discipline.

I will resist giving this emerging discipline a name, but if the acronym “AI” continues to be used as placeholder nomenclature going forward, let’s be aware of the very real limitations of this placeholder. Let’s broaden our scope, tone down the hype and recognize the serious challenges ahead.