I thought that, when I’d posted notice of my 3 Quarks Daily review of Robert Wright’s The God Test that that was that. But then I read his interview with Liron Shapira, AI Dystopia Is Just 8 Years Away — Robert Wright, Bestselling Author of “The God Test.” Whoops! There’s that phrase again.
The phrase I’m talking about is “reverse engineer.” I believe that the term has been kicking around in psychology for several decades, but I associate it with Steven Pinker’s 1997 book, How the Mind Works (pp. 21-22:
Reverse-engineering is what the boffins at Sony do when a new product is announced by Panasonic, or vice versa. They buy one, bring it back to the lab, take a screwdriver to it, and try to figure out what all the parts are for and how they combine to make the device work. We all engage in reverse-engineering when we face an interesting new gadget. In rummaging through an antique store, we may find a contraption that is inscrutable until we figure out what it was designed to do. When we realize that it is an olive-pitter, we suddenly understand that the metal ring is designed to hold the olive, and the lever lowers an X-shaped blade through one end, pushing the pit out through the other end. The shapes and arrangements of the springs, hinges, blades, levers, and rings all make sense in a satisfying rush of insight.
With that in mind, let’s look at some remarks Wright made to Liron:
Even so, what this approach to training does is, I think, replicate specific cognitive functionality in these machines that exist in the human mind. I don’t mean it does things exactly the way the human mind does. These models have independently invented things that natural selection invented. Edge detection’s a very clear case.
Wright doesn’t use the phrase there, but that’s what he has in mind. Here Wright uses the phrase:
... but even with the current paradigm, you give it kinds of data, visual, auditory, written, whatever, and it basically reverse engineers parts of the human mind that do the transmutation of one form of data into another.
There’s the key phrase. Here’s one last passage. The phrase isn’t here, but the idea certainly is:
You just make the machine good at predicting the next sequence of letters. When you first show it the stuff, it’s pure gibberish, but the machine itself finds a way of mapping the meaning. We do give it the basic... We do say you gotta use vectors to represent the words. It didn’t invent that. But we didn’t say, “By the way, you should choose numbers to fill in the blanks in the vectors that capture this thing we call meaning.” No, it in effect discovered that meaning is a property of words. I would put it that way.
NO.
It’s one thing to talk about psychologists “reverse engineering” bits of human behavior. I have no problem with that. But that’s not what Wright is doing here. He’s talking about machine learning, about transformers, reverse engineering human language and cognition. That’s not at all a useful way of conceptualizing what’s going on. It’s anthropomorphizing. It’s also the kind of mystification that Silicon Valley has been using to hype AI.
Once again I refer you to Berk Idem’s review, where he notes:
My problem is the road he takes in the book. Wright keeps telling the story of AI as if the machines discovered things on their own, that they found the meanings of words, that they grew something like an eye, that they started to evolve, when in fact people set almost all of the machinery up on purpose, with a pretty clear idea of what they were doing and why. I never expected myself to be on the “intelligent design” side of a debate, yet here I am, for instance, arguing that LLMs did not miraculously discover meaning, they were designed to do that. Wright gives too much credit to what models discover during training and too little to the architecture, objective, and research program that produced those discoveries.
Idem is correct. You should consult his review for more details, but I’ll say a couple of words about edge detection in vision and the meaning of words.
Back in 1959 Hubel and Wiesel published their seminal work on the visual cortex of the cat, work that led to their 1981 Nobel Prize in Physiology or Medicine. Virtually all work in machine vision has been informed by that study in one way or another. Machine vision systems are engineered to be sensitive to edges.
The same is true for language. It simply is not true that transformers “in effect discovered that meaning is a property of words.” There is a long tradition within linguistics and computational linguistics of thinking about the meanings of words as a function of the contexts in which words are used, something I review in a recent working paper, The Origins of LLMs – A long tectonic subduction event finally producing a visible volcanic eruption in November 2022 (the subtitle was suggested by ChatGPT). The idea dates back to the 1950s while its computational exploitation dates back to work that Gerard Salton began in the late 1960s on information retrieval. He’s the one who came up with the idea of using vectors to represent linguistic meaning. The transformer is thus a recent elaboration of an idea that’s been around for decades, an idea that human researchers came up for conceptualizing meaning.
* * * * *
There’s much more in Wright’s interview with Liron, much of it interesting. And some of it is bothersome, but I’ve said enough on that score. The God Test is an interesting book. But be careful of what Wright attributes to the machine. You’re better off having no explanation than accepting one that’s misleading at best.
No comments:
Post a Comment