Michael Jordan: How AI Fails Us, and How Economics Can Help

Jordan argues that AI, which is mostly machine learning these days, remains dominated by the Dr. Frankenstein notion of creating an artificial human. He regards that as a mistake, and argues for a more collective approach. (Cf. this post from two years ago, Beyond "AI" – toward a new engineering discipline.)

From the YouTube page:

Artificial intelligence (AI) has focused on a paradigm in which intelligence inheres in a single agent, and in which agents should be autonomous so they can exhibit intelligence independent of human intelligence. Thus, when AI systems are deployed in social contexts, the overall design is often naive. Such a paradigm need not be dominant. In a broader framing, agents are active and cooperative, and they wish to obtain value from participation in learning-based systems. Agents may supply data and resources to the system, only if it is in their interest. Critically, intelligence inheres as much in the system as it does in individual agents. This perspective is familiar to economics researchers, and a first goal in this work is to bring economics into contact with computer science and statistics. The long-term goal is to provide a broader conceptual foundation for emerging real-world AI systems, and to upend received wisdom in the computational, economic and inferential disciplines.

Michael I. Jordan is the Pehong Chen Distinguished Professor in the departments of electrical engineering and computer science and of statistics at the University of California, Berkeley. His research interests bridge the computational, statistical, cognitive, biological and social sciences. Jordan is a member of the National Academy of Sciences, the National Academy of Engineering, and the American Academy of Arts and Sciences, and a foreign member of the Royal Society. He was a plenary lecturer at the International Congress of Mathematicians in 2018. He received the Ulf Grenander Prize from the American Mathematical Society in 2021, the IEEE John von Neumann Medal in 2020, the IJCAI Research Excellence Award in 2016, the David E. Rumelhart Prize from the Cognitive Science Society in 2015 and the ACM/AAAI Allen Newell Award in 2009.

Two slides from the presentation:

Here’s a paper where Jordan is one of the authors:

By Divya Siddarth, Daron Acemoglu, Danielle Allen, Kate Crawford, James Evans, Michael Jordan, E. Glen Weyl, How AI Fails Us, Edmond J. Safra Center for Ethics and Carr Center for Human Rights Policy, Harvard University (December 1, 2021).

Abstract:

The dominant vision of artificial intelligence imagines a future of large-scale autonomous systems outperforming humans in an increasing range of fields. This “actually existing AI” vision misconstrues intelligence as autonomous rather than social and relational. It is both unproductive and dangerous, optimizing for artificial metrics of human replication rather than for systemic augmentation, and tending to concentrate power, resources, and decision-making in an engineering elite.  Alternative visions based on participating in and augmenting human creativity and cooperation have a long history and underlie many celebrated digital technologies such as personal computers and the internet.  Researchers and funders should redirect focus from centralized autonomous general intelligence to a plurality of established and emerging approaches that extend cooperative and augmentative traditions as seen in successes such as Taiwan’s digital democracy project to collective intelligence platforms like Wikipedia. We conclude with a concrete set of recommendations and a survey of alternative traditions.

Beyond AGI (Artificial General Intelligence)

Tyler Cowen recently posted "Holden Karnofsky emails me on transformative AI" over at Marginal Revolution.

I made two comments. One was similar to my recent post on Rodney Brooks, but much shorter – see this Brooks post as well. I'm posting the other one here as a place-holder. I may or may not elaborate it into a more substantial post at some future time.

Just what IS transformative AI? And AGI, what's that? Or superintelligence? They're highly abstract ideas that can easily be decked out in a grand way.

I've been reading around and about in the material at and linked to that "most important century" page. I certainly haven't read it all. It's interesting stuff. But, for better or worse, I find that Kim Stanley Robinson's science fiction novel, New York 2150, presents a more tangible and believable picture of the future. I'm not saying I agree with it, much less like it, only that it feels more credible to me than a world in which, for example, completely digital people are zipping around in a big pile of compute sometime later in this century.

Now, here's a page listing mostly recent open access articles about robotics and AI that have been published in the Nature family of journals. I've not read any of them nor even the abstracts. My remarks are offered solely on the basis of these titles:

A wireless radiofrequency-powered insect-scale flapping-wing aerial vehicle.
Enhancing optical-flow-based control by learning visual appearance cues for flying robots.
Highly accurate protein structure prediction with AlphaFold.
Advancing mathematics by guiding human intuition with AI.
A mobile robotic chemist.
Neuro-inspired computing chips.
Designing neural networks through neuroevolution.
Deep learning robotic guidance for autonomous vascular access.

I'm pretty sure that none of them assert that they've achieved AGI and I'd be surprised if any of them even hinted that AGI will be popping up in mid-century. I see no reason why these kinds of things won't keep coming and coming and coming. Some will eventuate in practical technologies. Some, likely most, won't. But, what's the likelihood that the cumulative result will be transformative, without, however, producing such wonders as fully digital people?

You might want to take a look at Michael Jordan, "Artificial Intelligence — The Revolution Hasn’t Happened Yet", Medium 4/19/2018. He suggests that beyond what Jordan calls "human imitative AI" (such as digital people) we should recognize "Intelligence Augmentation" and "Intelligent Infrastructure." He concludes:

Moreover, we should embrace the fact that what we are witnessing is the creation of a new branch of engineering. The term “engineering” is often invoked in a narrow sense — in academia and beyond — with overtones of cold, affectless machinery, and negative connotations of loss of control by humans. But an engineering discipline can be what we want it to be.

In the current era, we have a real opportunity to conceive of something historically new — a human-centric engineering discipline.

I will resist giving this emerging discipline a name, but if the acronym “AI” continues to be used as placeholder nomenclature going forward, let’s be aware of the very real limitations of this placeholder. Let’s broaden our scope, tone down the hype and recognize the serious challenges ahead.

Beyond "AI" – toward a new engineering discipline

Another bump to the top, this time because I'm thinking about Facebook, the future of social media, and the need for new institutional actors to counter-act both for-profit social media companies and the government. AI in the form of Intelligent Infrastructure, see below, surely has a role to play here.

* * * * *

 

I'm bumping this to the top of the queue in response to remarks by Ted Underwood on Twitter and by Willard McCarty in the Humanist Discussion Group.

* * * * *

Mark Liberman at Language Log posted a link to an excellent article by Michael Jordan, "Artificial Intelligence — The Revolution Hasn’t Happened Yet", Medium 4/19/2018. Here are some passages.

Whether or not we come to understand “intelligence” any time soon, we do have a major challenge on our hands in bringing together computers and humans in ways that enhance human life. While this challenge is viewed by some as subservient to the creation of “artificial intelligence,” it can also be viewed more prosaically — but with no less reverence — as the creation of a new branch of engineering. Much like civil engineering and chemical engineering in decades past, this new discipline aims to corral the power of a few key ideas, bringing new resources and capabilities to people, and doing so safely. Whereas civil engineering and chemical engineering were built on physics and chemistry, this new engineering discipline will be built on ideas that the preceding century gave substance to — ideas such as “information,” “algorithm,” “data,” “uncertainty,” “computing,” “inference,” and “optimization.” Moreover, since much of the focus of the new discipline will be on data from and about humans, its development will require perspectives from the social sciences and humanities.

While the building blocks have begun to emerge, the principles for putting these blocks together have not yet emerged, and so the blocks are currently being put together in ad-hoc ways.

He goes on to observe that the issues involved are too often discussed under the rubric of "AI", which has meant various things at various times. The phrase was coined in the 1950s to denote the creation of computing technology possessing a human-like mind. Jordan calls this "human-imitative AI" and notes that it was largely an academic enterprise whose objective, the creation of "high-level reasoning and thought", remains elusive. In contrast:

The developments which are now being called “AI” arose mostly in the engineering fields associated with low-level pattern recognition and movement control, and in the field of statistics — the discipline focused on finding patterns in data and on making well-founded predictions, tests of hypotheses and decisions.

This work is often packaged as machine learning (ML).

Since the 1960s much progress has been made, but it has arguably not come about from the pursuit of human-imitative AI. [...] Although not visible to the general public, research and systems-building in areas such as document retrieval, text classification, fraud detection, recommendation systems, personalized search, social network analysis, planning, diagnostics and A/B testing have been a major success — these are the advances that have powered companies such as Google, Netflix, Facebook and Amazon.

One could simply agree to refer to all of this as “AI,” and indeed that is what appears to have happened. Such labeling may come as a surprise to optimization or statistics researchers, who wake up to find themselves suddenly referred to as “AI researchers.” But labeling of researchers aside, the bigger problem is that the use of this single, ill-defined acronym prevents a clear understanding of the range of intellectual and commercial issues at play.

He then goes on to coin two more terms, "Intelligence Augmentation" (IA) and "Intelligent Infrastructure" (II). In the first

...computation and data are used to create services that augment human intelligence and creativity. A search engine can be viewed as an example of IA (it augments human memory and factual knowledge), as can natural language translation (it augments the ability of a human to communicate). Computing-based generation of sounds and images serves as a palette and creativity enhancer for artists.

The second involves

...a web of computation, data and physical entities exists that makes human environments more supportive, interesting and safe. Such infrastructure is beginning to make its appearance in domains such as transportation, medicine, commerce and finance, with vast implications for individual humans and societies.

And now we get to his central question:

Is working on classical human-imitative AI the best or only way to focus on these larger challenges? Some of the most heralded recent success stories of ML have in fact been in areas associated with human-imitative AI — areas such as computer vision, speech recognition, game-playing and robotics. So perhaps we should simply await further progress in domains such as these. There are two points to make here. First, although one would not know it from reading the newspapers, success in human-imitative AI has in fact been limited — we are very far from realizing human-imitative AI aspirations. Unfortunately the thrill (and fear) of making even limited progress on human-imitative AI gives rise to levels of over-exuberance and media attention that is not present in other areas of engineering.

Second, and more importantly, success in these domains is neither sufficient nor necessary to solve important IA and II problems.

And he goes on to explore that theme.

Moreover,

...the current focus on doing AI research via the gathering of data, the deployment of “deep learning” infrastructure, and the demonstration of systems that mimic certain narrowly-defined human skills — with little in the way of emerging explanatory principles — tends to deflect attention from major open problems in classical AI. These problems include the need to bring meaning and reasoning into systems that perform natural language processing, the need to infer and represent causality, the need to develop computationally-tractable representations of uncertainty and the need to develop systems that formulate and pursue long-term goals. These are classical goals in human-imitative AI, but in the current hubbub over the “AI revolution,” it is easy to forget that they are not yet solved.

Coming to the end he makes and interesting historical observation:

It was John McCarthy (while a professor at Dartmouth, and soon to take a position at MIT) who coined the term “AI,” apparently to distinguish his budding research agenda from that of Norbert Wiener (then an older professor at MIT). Wiener had coined “cybernetics” to refer to his own vision of intelligent systems — a vision that was closely tied to operations research, statistics, pattern recognition, information theory and control theory. McCarthy, on the other hand, emphasized the ties to logic. In an interesting reversal, it is Wiener’s intellectual agenda that has come to dominate in the current era, under the banner of McCarthy’s terminology. (This state of affairs is surely, however, only temporary; the pendulum swings more in AI than in most fields.)

We need to realize that the current public dialog on AI — which focuses on a narrow subset of industry and a narrow subset of academia — risks blinding us to the challenges and opportunities that are presented by the full scope of AI, IA and II.

This scope is less about the realization of science-fiction dreams or nightmares of super-human machines, and more about the need for humans to understand and shape technology as it becomes ever more present and influential in their daily lives.

His concluding paragraphs:

Moreover, we should embrace the fact that what we are witnessing is the creation of a new branch of engineering. The term “engineering” is often invoked in a narrow sense — in academia and beyond — with overtones of cold, affectless machinery, and negative connotations of loss of control by humans. But an engineering discipline can be what we want it to be.

In the current era, we have a real opportunity to conceive of something historically new — a human-centric engineering discipline.

I will resist giving this emerging discipline a name, but if the acronym “AI” continues to be used as placeholder nomenclature going forward, let’s be aware of the very real limitations of this placeholder. Let’s broaden our scope, tone down the hype and recognize the serious challenges ahead.

Computation, Mind, and the World [bounding AI]

Some thoughts on the above topics. Think of it as an exercise in conceptual factoring. What’s being factored? The “space” of AI.

* * * * *

The debate goes on:

FYI my reply to that is here: https://t.co/eRyiUNGOK8

— Gary Marcus (@GaryMarcus) December 27, 2019

* * * * *

A week ago I did another post in my continuing effort to understand the limits of AI, AI at its best, pratfalls and all [the common sense problem is the resistance that the world presents to us]. That post ended like this:

It's as though Go and chess embody the abstract mental powers we bring to bear on the world (Chomskyian generativity? Cartesian rationality?) while the common sense problem, in effect, represents the resistance that the world presents to us. It is the world exerting its existence by daring us: "parse this, and this, and this, and...!"

What I’m suspecting is that the right mathematician should be able some how to put a boundary around this whole domain. But how? I’m certainly not the right mathematician, I’m not any kind of mathematician at all. But, hey! I’ve posed the question. I might as well ramble on about it.

Let us, for the moment, restrict ourselves to the realm of “common sense”, no specialized knowledge, just stuff that everyone knows more or less.

What we need is an abstract mathematical characterization of the world, the whole damn thing. Sounds crazy, no? Yes, definitely. But I’m not after what some physicists call a Theory of Everything. What I’m after isn’t physics at all. Forget physics, forget the ‘deep’ world. I’m interested in the surface, where we live, with our sensorimotor apparatus. Call it the phenomenal world. I’m interested in a mathematical characterization of THAT. What does the phenomenal world have to be like in order to be intelligible?

A completely chaotic world would not be intelligible, for there are no patterns to grasp. And it would be very difficult to make your way in a “smooth” world, where any given thing is very much like any number of other things such that discriminating between any two of them is difficult, though not impossible.

We living in a “lumpy” world. What do I mean by lumpy? Consider visual appearance. Cats look resemble one another to a high degree, more so than any of them resembles a dog, who in turn resemble one another as well, though my impression is that there is greater variety in the appearances of dogs than of cats (and I’m not only thinking of domestic cats, but of the wild ones too). Similarly with snakes. Now is there anything that resembles a snake as much as it resembles a cat? No? The “form space” between cats and snakes is pretty empty, as is the form space between dogs and snakes. That’s what I mean by lumpy. In a smooth world the space between cats, dogs, and snakes would be populated so that dividing the space into distinctly different kinds of creatures is all but arbitrary.

Of course this lumpy world isn’t confined to objects. Things move, they grow, act, and sense, and so forth. All these are aspects of lumpiness as well.

Given that we live in a lumpy world, what’s the structure of the lumpiness? That’s what I want to know. That’s one thing.

* * * * *

Let’s confine ourselves language systems that learn from large bodies of text. What kind of argument would I like to see made about such systems?

Those texts, of course, were generated by humans using their full mental facilities, including their sensorimotor systems, to learn about the world, which we have posited as, in some sense, being lumpy. The texts themselves are a “flattened” or “smoothed” (in the sense I’ve indicated in the previous section) representation of the world. As such, those texts have already “squeezed out” a lot of that lumpy structure. We can deal with the resulting compressed representations because we always have recourse to the world itself and so can, in effect, expand them. All those piles of text have no access to the “pointers” we use to guide the expansion. That is, the various representations we produce about the world – which, after all, are data deep learning feeds on – do not in fact capture our knowledge of the world no matter how many of those representations are consumed.

What deep learning systems gain by using every larger bodies of text is more and more resolution in their recovery of structure from the text, a smoothed representation. But they can never recover or reconstruct the information that was lost in the smoothing process. THAT’s the limitation of these new techniques.

And THAT’s what I want this hypothetical mathematician to begin investigating. We need 1) some account of the lumpy world, 2) some account of what happens in a smoothed (verbal) expression of that world, so that 3) we can argue that deep learning can never recover that lost information.

* * * * *

Note that I don’t really think that symbolic systems can produce a full expression of the world any more than natural language itself can. But the humans coding symbolic systems can take advantage of their knowledge of the world to introduce things into the system that aren’t available to deep learning systems.

What things? And how can we model that? How much common sense can human modelers introduce into the system through symbolic means?

* * * * *

What’s critical in dealing with the “lumpiness” of the world is interacting with the world through a sensorimotor system. We’re going to need robots, not just isolated AI ‘thinking machines’.

* * * * *

Let’s return to where we began. I said:

Forget physics, forget the ‘deep’ world. I’m interested in the surface, where we live, with our sensorimotor apparatus. Call it the phenomenal world. I’m interested in a mathematical characterization of THAT.

What’s the relationship between a robust characterization of sensorimotor perception, action, and cognition and THAT mathematical characterization? In particular, what’s the relationship between a robust mathematical characterization of sensorimotor perception, action, and cognition and our mathematical characterization of the phenomenal world?

And, of course, the common sense world is not all we live in, not at all. We have worlds of specialized knowledge, and, in particular, of abstract knowledge. What of them? Hays and I have argued (reference below) that to date four major techniques of constructing abstract knowledge have emerged: metaphor, metalingual definition, algorithm, and control. What of them? I note that some, though certainly hot all, of these abstract worlds already have rich mathematical characterizations.

William Benzon and David Hays, The Evolution of Cognition, Journal of Social and Biological Structures. 13(4): 297-320, 1990, https://www.academia.edu/243486/The_Evolution_of_Cognition.

We've also done a little work on how metaphor advances thought into new regions: William Benzon and David Hays, Metaphor, Recognition, and Neural Process, The American Journal of Semiotics, Vol. 5, No. 1 (1987), 59-80, https://www.academia.edu/238608/Metaphor_Recognition_and_Neural_Process.

While I’m at it, Hays and I also took a run on the brain. We reviewed a wide range of work in perceptual and cognitive psychology, neuroscience, developmental psychology and comparative psychology and neuroanatomy. This is what we came up with: William Benzon and David Hays, Principles and Development of Natural Intelligence, Journal of Social and Biological Structures, Vol. 11, No. 8, July 1988, 293-322, https://www.academia.edu/235116/Principles_and_Development_of_Natural_Intelligence.

* * * * *

Where are we? I began by suggesting that “the right mathematician should be able some how to put a boundary around this whole domain.” In particular, that mathematician would provide a mathematical characterization of the “lumpiness” of the phenomenal world. Of course I don’t know whether or not we’re there yet. It was just a suggestion.

I ended up by pointing out that there is more to the human world than the phenomenal world of common sense perception, action, and cognition. We’ve also got the worlds created through various techniques of abstraction and that some of those already have mathematical characterizations. Would a mathematical characterization of the phenomenal world then, in effect, close the space?

Who knows if that’s even a meaningful question? To say it is meaningful would be to imply that we know how to go about answering it? Do we?

If the answer to that is, yes, and it’ll take centuries, then, no, we haven’t a clue.

Is any one ready to hazard, yes, in a decade or three we’ll be there?

* * * * *

Michael Jordan has written, “Artificial Intelligence — The Revolution Hasn’t Happened Yet”, Medium 4/19/2018. For the revolution to happen, we need a more differentiated sense of the application domain: What techniques work for what applications and why? A framework like the one I've attempted to sketch out above would be useful there, wouldn't it?