ZDNET's report on AI Debate 3 – Where we are (and where we might be going?)

Machines that think like humans: Everything to know about AGI and AI Debate 3 https://t.co/2Q9SMUiHPK by @TiernanRayTech

— ZDNET (@ZDNET) December 26, 2022

I watched two-thirds of the debate and thought it quite useful. Here's a good passage from the article:

Ferrucci gave examples of where GPT-3 fell down on common-sense reasoning tasks. Such systems, given enough data, "will start to reflect what we consider common sense." Poking around GPT-3, he said "is like exploring a new planet, it's sort of this remarkable thing," and yet "it's ultimately unsatisfying" because it's only about output, not about the reasons for that output. 

His company, said Ferrucci, is pursuing a "hybrid" approach that uses language models to generate hypotheses as output, and then performing reasoning on top of that using "causal models." Such models "can be induced" from output, but they also require humans to interact with the machine. That would lead to "structured representations" on top of language models.

After Ferrucci, Dileep George returned to give some points on common sense, and "mental simulation." He discussed how humans will imagine a scenario -- simulating -- in order to answer common-sense reasoning questions. The simulation enables a person to answer many questions about a hypothetical question. 

George hypothesized that the simulation comes from the sensorimotor system and is stored in the "perceptual+motor system."

Language, suggested George, "is something that controls the simulation." He proposed the idea of conversation as one person's attempt to "run the simulation" in someone else. 

AGI by 2030? The bet is on.

Clune proposed "three pillars" in which to push: "We need to meta-learn the architectures, we need to meta-learn the algorithms, and most important, we need to automatically generate effective learning environments and/or the data." 

Clune observed AI improves by "standing on the shoulders" of various advances, such as GPT-3. He gave the example of the OpenAI project where videos of people playing games brought a "massive speed-up" to machines playing Minecraft.

Clune suggested adding a "fourth pillar," namely, "leveraging human data."

Clune predicted there is a 30% change of achieving AGI by 2030, as defined as "capable of doing more than 50% of economically valuable human work." Clune added that the path is "within the current paradigm," meaning, no new paradigm was needed.

Clune closed with the statement, "We are not ready" for AGI. "We need to start planning now."

Marcus said "I'll take your bet" about a 30% chance of AGI in 2030.

Clune was followed by Sara Hooker, who heads up the non-profit research lab Cohere For AI, talking on the topic, "Why do some ideas succeed and others fail?" 

Hooker's presentation was based on ideas from her paper, "The Hardware Lottery," which I've already blogged about.

The discipline, said Hooker, has become locked into the hardware, and with it, bad "assumptions," such as that scaling of neural nets alone can succeed. That, she said, raises problems, such as the great expense of "memorizing the long tail" of phenomena. 

Progress, said Hooker, will mean "reducing the cost of different hardware-software-algorithm combinations."

From the ensuing conversation:

Clune kicked off the Q&A, with the question, "Wouldn't you agree that ChatGPT has fewer of those flaws than GPT-3, and GPT-3 has fewer than GPT-2 and GPT-1? If so, what solved those flaws is the same playbook, just scaled up. So, why should we now conclude that we should stop and add more structure rather than embrace the current paradigm of scaling up?"[...]

Choi replied, "ChatGPT is not larger than GPT-3 DaVinci, but better trained through a huge amount of human feedback, which comes down to human annotation." The program, said Choi, has produced rumors that "they [OpenAI] might have spent over $10 million for making such a human annotated data. So, it's maybe a case of more manual data." Choi rejected the notion ChatGPT is better.

"Your brain, your job, and your most fundamental beliefs will be challenged by AI like nothing ever before. Make sure you understand it, how it works, and where and how it is being used." – David Ferrucci 

There's much more in the article, including discussions of ethics and policy. For example, near the end Kai-Fu Lee spoke of dangers:

In past, said Lee, "I could either think of algorithms or at least imagine technological solutions" to the "externalities" created by AI programs. "But now I am stuck because I can't easily imagine a simple solution for the AI-generated misinformation as specifically targeted misinformation that's powerful commercially," said Lee. 

What if, said Lee, Amazon Google or Facebook can "target each individual and mislead […] and give answers that could potentially be very good for the commercial company because there is a fundamental misalignment of interests, because large companies want us to look at products, look at content and click and watch and become addicted [so that] the next generation of products […] be susceptible to simply the power of capitalism and greed that startup companies and VCs will fund, activities that will generate tremendous wealth, disrupt industries with technologies that are very hard to control."

That, he said, will make "the large giants," companies that control AI's "foundation models" even more powerful. 

"And the second big, big danger is that this will provide a set of technologies that will allow the non-state actors and the people who want to use AI for evil easier than ever." [...] Non-state actors, he suggested, might "lead people to thoughts" that could disrupt elections, and other terrible things --- "the beginning of what I would call "Cambridge Analytica on steroids."

Lee urged his peers to consider how to avert that "largest danger." 

Here is the debate program with links to various papers.