Toward a manifesto for the naturalistic analysis of large language models

Here's a comment I just posted in response to a post by Tyler Cowan, GPT and my own career trajectory:

I too have spent a great deal of time using ChatGPT, though to somewhat different ends. I analyze and study texts, and so I'm analyzing ChatGPT's output. That's why, for example, I had it do a Girardian reading of Jaws. I wasn't expecting or hoping that it would come up with new insights. For one thing, it hasn't seen the film, but has to work from commentary widely available on the web, and from, I assume, complete scripts that it consumed during training. I just wanted to see if it could do it at all. I discovered that, yes, it could. And so I directed it to do similar work with A.I. and with the Astro Boy stories.

In a similar fashion I've observed what it does in constructing conversations, dealing with abstract concepts (e.g. justice and charity), ungrammatical or nonsense sentences, and stories. In the case of stories I get it to work variations on one or two basic story ideas: What remains constant from one version to the next, and what changes? Of course, I ask for specific changes; I'm looking to see how ChatGPT implements them.

Why am I doing this?

• I analyze texts; I enjoy doing it. And ChatGPT's texts are complicated enough to be interesting under analysis.
• I want to understand what's going on "under the hood." Yes, we can pop the hood and take a look. People are doing that. But it's difficult; we've got a long way to go. By producing clear and rigorous descriptions of phenomena in ChatGPT's output, I can provide the model mechanics with things to look for.
• We need to develop new benchmarks that are more adequate to the capabilities of LLMs. This work contributes to that.

I'm working toward a coherent and well-defined methodology for what, for lack of a better term, I call the naturalistic investigation of LLMs. We may have created these devices; they are artificial in that sense. But they are also very complex. We don't understand them in the way we understand automobiles or "standard-issue" computer programs, programs handcrafted by programmers (with the aid of various coding tools, now including AIs). These things thus become part of the world in which we live. They are natural, and need to be understood on those terms.

More later.