Earlier this week I posted another article to 3 Quarks Daily:
The title says it all, sort of. The article centers on the fact that, as far as I can tell, many of our AI experts aren’t so expert, that is, they often don’t know what they’re talking about. People will criticize their arguments, but won’t call them out for not having the expertise they claim they have.
Actually, I don’t quite get that far in the article. That’s one topic for this post. The other is an extension of the whaling metaphor in the title. It seems to me that the mad dash for AGI is a bit like Ahab’s quest for Moby Dick, which did not end well for either of them.
What’s an AI expert expert about? NOT human language & cognition.
I chose that whaling analogy to emphasize the peculiar nature of expertise in machine learning: You don’t have to know much about human language and cognition in order to build an AI engine that mimics human cognitive behavior astonishingly well, much better than anyone would have predicted as recently as 2019, the year before GPT-3 was unveiled. Hence the analogy, and AI expert is like a whaling captain who knows all about his ship, but little about whales.
How did this come about and, more to the point, why do we let them get away with it? I didn’t actually pose the latter question, but my essay did suggest an answer to it: This feature of AI-culture has become quasi-institutionalized so that responsibility for pronouncements made by individual researchers must be apportioned between the culture and the individuals. If we question their expertise directly, rather than simply criticizing their arguments, that critique threatens our quasi-institutional understandings about the scope of that culture. Once we start down that road, what other institutional understandings will start to unravel? Let’s not go there.
But that’s a digression. I’m more interested in saying a bit more about how this situation came about.
As I pointed out in the paper, the issue can be traced back to Turing’s famous paper, “Computing Machinery and Intelligence” (1950). “That’s the paper in which he proposed the so-called Turing Test for evaluating machine accomplishment. The test explicitly rejects comparisons based on internal mechanisms, regarding them as intractably opaque and resistant to explanation, and instead focuses on external behavior.” That was (perhaps) a reasonable thing to do at the time. That test, however, was rendered useless in the late 1960s by Joseph Weizenbaum’s ELIZA, a simple computer program simulated human conversation if a very compelling way. At that time, the actual accomplishments of the discipline were not very compelling, at least to researchers outside the discipline, and over-reaching proclamations (about when computers will surpass humans) had few implications for practical action. That is no longer the case. Billions of dollars are being wagered on predictions offered by AI experts.
If we look at what actually happened – and here I’m sketching out a history I haven’t researched thoroughly, I’m just making this up out of what’s already in my mind, so beware of LLM-like confabulation – we see that early researchers did attend to human cognition. That’s most obvious in the case of chess, where research into human chess playing was undertaken, and the ubiquitous expert systems, where human experts were interviewed about their thought processes as preparation for designing the system. Around the corner, researchers in the sibling discipline of computational linguistics (originally machine translation) called on linguistics and cognitive psychology for insight into the design of these systems.
These two lines research of research came together in a large project sponsored by the Defense Department, the ARPA Speech Understanding Project. It extended over five years in the mid-1970s and involved three separate projects involving perhaps a half-dozen research organizations in universities and other research organizations. They undertook to create systems in which a computer would take spoken language questions and provide answers in written English. This was a massive research effort that recruited expertise both in computer science and engineering and in human perception and cognition. No one researcher was expert in all the disciplines involved, but the enterprise encompassed them all, and I assume that at least some researchers read all the reports produced by the project in which they took part, if not all the reports from all the projects. (As bibliographer for Computational Linguistics at the time, I scanned and prepared abstracts for them all.)
Things began to change in the 1980s, when research in connectionist neural networks re-commenced and statistical machine learning techniques began emerging. These techniques effectively separated computational expertise from domain knowledge. It was no longer so necessary to bring deep domain expertise to bear on the design of computer systems. That divergence widened into a yawning chasm with the use of GPUs in the second decade of this century. But the implicit quasi-institutional understandings that had developed back in the 1950s and 1960s remained in place.
The people who develop the computer systems are THE experts. And they certainly are experts. But as far as I can tell, the level of expertise these machine experts have in linguistics and human cognition is nothing to write home about. In that sense they are like the hapless captain of a whaling vessel who knows about his ship, but not about whales.
This is not a healthy situation.
Ahab, Moby Dick, and AI Doom
When I first came up with my title it was but a device to point out the divergence between AI researcher’s knowledge of their programs and their ignorance about language. I had no intention of elaborating it into a conceit that I’d employ at various points throughout the essay. Nor did I have in mind that I’d actually reference Melville’s Moby Dick. That just happened (near the end).
And now that it has, I wonder. Is the pursuit of (the mythical) AGI like Ahab’s pursuit of Moby Dick? Is their fear that the AGI will turn on them, is that like Ahab’s fear of and animosity toward Moby Dick? Is the underlying psychology pretty much the same despite all the obvious differences between the two passions? I don’t know. But as Edward Mendelson argued back in 1976, Moby Dick is an encyclopedic narrative, one that set out to encompass the whole of mid-19th century America. The danse macabre between Ahab and the whale is not thus a private affair between a man and an animal; it is a figure for something at the heart of America. What? Or was Melville just imagining things?
Whatever.
The high-tech industry’s dash to AI supremacy has a similar sweep. We're on a Nantucket sleigh-ride. Instead of 45 ton Moby Dick pulling a 35 ft. whaleboat, a bunch of techbros and Silicon Valley billionaires have hitched the earth to a rampaging Jupiter (318 times the mass of the earth). Look at the numbers in this tweet:
These numbers are based on quite literally nothing. This is SO DUMB. It looks rigorous because it involves numbers, but in reality it's nothing more than baseless hunches. smh. pic.twitter.com/xc5xLG0VYa
— Dr. รmile P. Torres (@xriskology) December 8, 2023
I recognize every one of those names, though I know more about some than others. I assume that the mythical everyone knows who Elon Musk is, but not many are likely to know who Zvi Mowshowitz is. I don’t either, not really. That is, I can’t recite the story of how he came to be regarded as an expert, but I do read his long posts at LessWrong. Musk thinks there’s a 20% to 30% chance of AI killing us all; Mowshowitz puts the number at 60%. The whole range is from 10% to 90%.
Those numbers are insane, Ahab-like insane. If you were running in a marathon and someone came up to you during the race and told you that there was a mere 10% chance you will die unless you exit the race, NOW, what would you do? You’d stop running, just as you would if the chances were 20%, 50%, or 90%. In what way do these people believe those numbers? To what reality are they tethered? To what extent are the tech industry’s actions guided by perceptions no more grounded in reality than the so-called hallucinations of a large language model?
* * * * *
Addendum 12.11.23: In terms of the informal game theory argument offered in the section, “Deconstructing AI Doom,” of my article, A New Counter Culture, those numbers are a Schelling point, a rallying point for a new (counter) culture. As such, what’s important about those numbers IS NOT their plausibility, though they do come draped in epistemic theatre intended to create the appearance plausibility, but their distinctiveness. Those numbers are not tethered to the reality of mainstream media, whatever that is. They’re a clear demarcation of a different way of looking at the world, one regarded as superior by its proponents.
* * * * *
Addendum: See this most interesting article by Andrew Schenker, After Melville, in The Baffler.
No comments:
Post a Comment