A historical problem solved by a human but that is beyond chatbots

Elon Danziger, ChatGPT Will Never Beat Indiana Jones, NYTimes, Dec. 22, 2025.

Across from the Florence Cathedral in Italy stands a much older church, the Baptistery of San Giovanni. It is a beloved center of religious life, where many Florentines are baptized to this day. Staid columns and lively arches hug its eight sides, half-camouflaged in patterns of green and white marble. Without the baptistery’s emulation of the architecture of ancient Rome, it’s hard to imagine Florence birthing the architectural Renaissance that changed the face of Europe. Yet for centuries, there has been no compelling solution as to who built it and when and for what reasons. Decades ago, I gave tours of the baptistery and came to revere it, and in the early 2020s I began delving into its origins.

After years of poring over historical documents and reading voraciously, I made an important discovery that was published last year: The baptistery was built not by Florentines but for Florentines — specifically, as part of a collaborative effort led by Pope Gregory VII after his election in 1073. My revelation happened just before the explosion of artificial intelligence into public consciousness, and recently I began to wonder: Could a large language model like ChatGPT, with its vast libraries of knowledge, crack the mystery faster than I did?

So as part of a personal experiment, I tried running three A.I. chatbots — ChatGPT, Claude and Gemini — through different aspects of my investigation. I wanted to see if they could spot the same clues I had found, appreciate their importance and reach the same conclusions I eventually did. But the chatbots failed. Though they were able to parse dense texts for information relevant to the baptistery’s origins, they ultimately couldn’t piece together a wholly new idea. They lacked essential qualities for making discoveries.

There are a few reasons for this. Large language models have read more text than any human could ever hope to. But when A.I. reads text, it’s merely picking up patterns. Peculiar details, outlier data and unusual perspectives that can influence thinking can get lost. Without eccentric or contrarian ideas, I never would have made my discoveries. [...]

Synthesizing so many pieces of medieval history into a new interpretation required stepping back and reconsidering their importance and how they relate to one another. A.I. may be able to optimize the process of collecting those pieces, but discovery means drawing new connections — something far beyond current A.I. capabilities, as the tests I did confirmed to me.

This is consistent with a series of posts I did on my own work: One about interpreting Jaws, one about investigating the "Xanadu" meme, and one about ring-composition in Heart of Darkness.