AI's biggest illusion, that chess is a good model for intelligence in general

I have been saying in various times and places that it seems to me that AI has (implicitly) taken chess as its prototype for AI research. For one thing, we have John McCarthy's well-known article, “Chess as the Drosophila of AI” (1990). That is, however, a mistake, as I have pointed out in a recent working paper, Computation, Chess, and Language in Artificial Intelligence. Chess is well-defined, while natural language is not. As a consequence the search space for chess is simple in form, a tree, and well understood. That is not at all the case for natural language. Finally, chess is finite, very large, but finite. That is not at all the case for natural language. Consequently chess is not at all a good paradigm for intelligence in general. Intuitions thus gained from it are likely to be misleading for the general problem.

It's in that context that I offer the passage from a recent podcast by Dwarkesh Patel, Eric Jang – Building AlphaGo from scratch. Note the lede for the podcast, "AlphaGo is still the cleanest worked example of the primitives of intelligence: search, learning from experience, and self-play." Here's how Dwarkesh introduces the podcast:

Eric Jang walks through how to build AlphaGo from scratch, but with modern AI tools.

Sometimes you understand the future better by stepping backward. AlphaGo is still the cleanest worked example of the primitives of intelligence: search, learning from experience, and self-play. You have to go back to 2017 to get insight into how the more general AIs of the future might learn.

Once he explained how AlphaGo works, it gave us the context to have a discussion about how RL works in LLMs and how it could work better – naive policy gradient RL has to figure out which of the 100k+ tokens in your trajectory actually got you the right answer, while AlphaGo’s MCTS suggests a strictly better action every single move, giving you a training target that sidesteps the credit assignment problem. The way humans learn is surely closer to the second.

Note that MCTS (Monte Carlo tree search) is one of the oldest algorithm in the book. Trees are very well defined. How do you structure language as a tree? So human intelligence may well tend toward that second alternative, the one based on MCTS, but it is not at all clear how studying chess is going to help you figure out how human intelligence does it. MCTS is not available.