Is AI about to enter a period of stagnation driven by mistaken business imperatives?

First some remarks by François Chollet in a recent podcast where he suggests that OpenAI has set back AI progress by 5 to 10 years by switching into full-business mode, which requires secrecy and over-commitment to LLMs. I follow that by some remarks by Yann LeCun about how to manage AI development.

Remarks by François Chollet

Chollet is a researcher at Chollet. He had a discussion with Dwarkesh Patel in which he expressed skepticism about reaching AGI with LLMs alone. Later entrepreneur Mike Knoop joined the conversation to discuss a prize they’re offering. I’m interested in remarks Chollett made in the second half of the conversation.

Here’s the timestamps for the whole conversation:

00:00:00 – The ARC benchmark
00:11:53 – Why LLMs struggle with ARC
00:19:43 – Skill vs intelligence
00:28:38 – Do we need “AGI” to automate most jobs?
00:49:11 – Future of AI progress: deep learning + program synthesis
01:01:23 – How Mike Knoop got nerd-sniped by ARC
01:09:20 – Million $ ARC Prize
01:11:16 – Resisting benchmark saturation
01:18:51 – ARC scores on frontier vs open source models
01:27:02 – Possible solutions to ARC Prize

Note that ARC stands for Abstraction and Reasoning Corpus, a set of benchmarks for measuring AI performance.

I’m interested in Chollet’s remarks on frontier research starting at 1:06:08 (transcript here):

It's actually really sad that frontier research is no longer being published. If you look back four years ago, everything was just openly shared. All of the state-of-the-art results were published. This is no longer the case.

OpenAI single-handedly changed the game. OpenAI basically set back progress towards AGI by quite a few years, probably like 5-10 years. That’s for two reasons. One is that they caused this complete closing down of frontier research publishing.

But they also triggered this initial burst of hype around LLMs. Now LLMs have sucked the oxygen out of the room. Everyone is just doing LLMs. I see LLMs as more of an off-ramp on the path to AGI actually. All these new resources are actually going to LLMs instead of everything else they could be going to.

If you look further into the past to like 2015 or 2016, there were like a thousand times fewer people doing AI back then. Yet the rate of progress was higher because people were exploring more directions. The world felt more open-ended. You could just go and try. You could have a cool idea of a launch, try it, and get some interesting results. There was this energy. Now everyone is very much doing some variation of the same thing.

The big labs also tried their hand on ARC, but because they got bad results they didn't publish anything. People only publish positive results.

I’m sympathetic to his complaint. At the end of my article in 3 Quarks Daily, Aye Aye, Cap’n! Investing in AI is like buying shares in a whaling voyage captained by a man who knows all about ships and little about whales, I worried that the AI industry would over-commit to LLMs and fall victim to the sunk cost fallacy:

Even now other research directions, such as those being proposed by Gary Marcus and others, are not being pursued. Basic research cannot deliver results on a timetable. Some directions will pan out, others will not. There is no way to determine which is which beforehand. Without other viable avenues for exploration, the prospect of throwing good money after bad may seem even more appealing and urgent. What at the moment looks like a race for future riches, intellectual and material, may turn into a death spiral race to the bottom.

Yann LeCun on the management of AI

In this context it’s worth looking at Yann LeCun’s prescription for running a successful AI lab. As you may know, Yann LeCun is a Turing Award winner, is VP and Chief AI Scientist, and is a Professor at NYU. From his X-feed:

It is of paramount importance that the management of a research lab be composed of reputable scientists.

Their main jobs are to:

1. Identify, recruit, and retain brilliant and creative people.
   
2. Give them the environment, resources, and freedom to do their best work.
   
3. Identify promising research directions (often coming from the researchers themselves) and invest resources in them. Put the scientists in charge and get out of the way.
   
4. Be really good at detecting BS, not necessarily because scientists are dishonest, but often because they are self-deluded. It's easy to think you've invented the best thing since sliced bread. Encouraging publications and open sourcing is a way to use the research community to help distinguish good work from not-so-good work.
   
5. Inspire researchers to work on research projects that have ambitious goals. It's too easy and less risky to work on valuable improvements that are incremental.
   
6. Evaluate people in ways that don't overly focus on short-term impact and simple metrics (e.g. number of publications). Use your judgment. That's why you get paid the big bucks.
   
7. Insulate rogue-but-promising projects from the scrutiny of upper management. A watched pot never boils. Planned innovation and 6-months milestones never bring breakthroughs.

You can't do any of these cat herding jobs unless you are an experienced, talented, and reputable scientist with a research record that buys you at least some legitimacy in the eyes of the scientists in your organization.

Note 5, 6, and 7 in particular. While AI companies certainly have ambitious goals, #5, the need to recoup large capital investments in hardware and to ship product will force them to focus on specific short-term goals (#6) and curtail any work that isn’t directly aimed to shipping produce (#7).