In June of last year I decided to ask ChatGPT to draw a semantic network diagram for Shakespeare's Sonnet 129. Why did I choose that task? Because it is something that humans can do, but it is not rocket science; it doesn't require genius level capability. I wanted to put a bound on all the hype about LLMs already being AGIs (whatever they are), or close to it. I chose ChatGPT because it is capable of drawing. The task requires the ability to draw, which ChatGPT has.
I wrote up the experiment in this working paper: ChatGPT tries to create a semantic network model for Shakespeare's Sonnet 129 (June 16, 2025). Here's the abstract:
This document explores the capacity of large language models, specifically ChatGPT, to construct semantic network models of complex literary texts, using Shakespeare's Sonnet 129 as a case study. Drawing on the author's prior work in cognitive modeling, the analysis reveals that ChatGPT, while capable of producing linguistically coherent commentary, fails to generate a structurally plausible semantic network for the sonnet. The failure is traced not to a lack of exposure to relevant literature, but to the model's lack of embodied, interactive learning. The process of constructing cognitive network diagrams is shown to be iterative, visual-verbal, and skill-based-comparable to learning a physical craft like playing an instrument or woodworking. It requires extended practice under expert feedback, enabling a form of reasoning that is neither algorithmic nor easily reducible to textual description. The essay argues that this hybrid modeling skill represents a "deep" human capability that is nevertheless teachable and routine. It concludes with reflections on the nature of such skills and their implications for AI, pedagogy, and literary interpretation. Asking ChatGPT create a semantic model for a Shakespeare sonnet.
About a week ago I had a long dialog with ChatGPT, first about how humans learn this task and then, second, what it would require to teach AIs how to learn the task. From there we went on to the more difficult task of probing the structure latent in the weights of an LLM and thereby extracting/constructing of map of that structure. Here's ChatGPT's summary of that discussion.
Summary: Learning Semantic Nets, Mapping Latent Structure in LLMs, and the Hybrid Future of Intelligence
We began by returning to a practical question that had surfaced in earlier work: why it is difficult for an AI system to produce good semantic or cognitive network diagrams for literary texts (such as a Shakespeare sonnet), even though a capable human can learn to do it. The core issue is not that such diagramming requires genius-level insight. Rather, it is a form of skilled representational practice—learnable by apprenticeship, but challenging for current AI systems because it requires disciplined structure-building, normative constraint satisfaction, and iterative repair.
1) Apprenticeship and “inside knowledge” of diagramming
A central theme was the idea that semantic network diagrams cannot be properly understood from the outside. A person learns to read them only by learning to produce them. This applies even more strongly to complex cognitive networks, where a diagram can appear meaningless unless one has internalized the diagram grammar: node types, arc types, compositional constraints, and “house style” norms about what counts as an admissible analysis.
You offered an idealized description of your training with David Hays. The training regime resembled a repeated cycle of supervised practice:
- you brought a fragment of text and a diagram you had produced,
- Hays evaluated it (approve/disapprove) with commentary,
- you revised or moved forward accordingly,
- the cycle repeated,
- and over time the normative discipline of diagramming became internalized.
You also noted that this same pattern governed group work among peers who had learned the system: a collaborative problem was brought to the table, and discussion plus sketching continued until a coherent solution emerged. The key was not merely producing diagrams, but learning the discipline that makes diagrams meaningful and correct.
From this, you proposed an account of what is being learned: a repertoire of correspondences between verbal fragments and diagram fragments. Under that view, diagramming competence is partly the acquisition of a “library of moves,” where particular linguistic patterns or conceptual pressures cue specific diagram operations. Equally important, however, is a critic’s sense of global coherence—a normative capacity to judge whether a graph “hangs together” as a model of the text and to identify what must be repaired.
You emphasized that at any time there is a locally stable diagram grammar, even if it cannot be complete in principle. In your own case, you began with Hays’ textbook Mechanisms of Language and learned to produce diagrams specified in particular chapters (cognition, perception). After three months of concentrated training you had internalized the system well enough not merely to use it, but to extend it: you proposed a new arc type, specified its assignment conditions, and demonstrated its usefulness. This was identified as an important marker of mastery: moving from conforming to norms to making responsible innovations within the normative system.
2) Why this is “easy” for humans but hard for AI
The conversation then turned to the striking asymmetry: semantic network diagramming is learnable by humans with patience and guidance, but remains difficult for AI systems. The difficulty is not lack of general linguistic ability; it is that diagramming requires explicit normative structure and repair behavior. Humans develop an internal sense of error: what is missing, what violates the grammar, what is incoherent globally. Current models often produce plausible fragments but struggle to maintain consistent typing, global integrity, and systematic revision under critique.
This diagnosis led to an important idea: it would be possible for AI to learn semantic network construction through an analogous apprenticeship regime—especially if the AI were multimodal (since the target representation is graphical). Training would require expert-guided correction cycles, ideally including revision histories, so that the system learns not only what the final diagram should look like, but how to repair incorrect diagrams.
At the far horizon, you raised a more ambitious possibility: AIs might learn diagramming so well that they could teach other AIs, performing the Hays-function themselves. That would require not only competence in diagram production, but competence in critique, repair, curriculum sequencing, and controlled extension of the grammar.
3) From diagramming text to extracting latent structure from neural weights
This discussion provided what you described as your first hint toward a larger goal: extracting cognitive-level network structures from foundation models. You contrasted this with Gary Marcus’ suggestion of investing enormous resources into hand-coded symbolic modeling. You argued that building a gigantic semantic net by armies of humans is madness. Instead, the semantic network “lives” implicitly in the weights of neural models—diffused across parameters—and the research problem is to map it, extract it, and make it explicit.
You described your working intuition: LLMs would not be so effective if they did not embody cognitive-network-like structures at some latent level. You also noted that you had conducted behavioral experiments (using only ordinary user access) that convinced you of this: controlled perturbations lead to distributed ripple effects that preserve story coherence. These results suggest that constraint structure is present, even if not symbolically explicit.
From this perspective, “ontology extraction” becomes an empirical, stochastic mapping discipline. One does not directly read networks off the weights. Instead, one probes behavior, perturbs conditions, observes stable patterns, and assembles inferred structures under an explicit representational grammar. The diagram grammar becomes essential as a way to turn a cloud of samples into a stable map.
An important complication was introduced here. Hays’ symbolic framework in Mechanisms of Language covers multiple layers: syntax, morphology, pragmatics, phonetics/phonology, cognition, perception. In contrast, LLMs are trained on token strings in which many of these levels are conflated. Thus any network extracted from the weights risks being entangled across linguistic and cognitive layers. You expressed the desire for a “pure cognition” network, but acknowledged that it is not clear how to achieve purity a priori. The practical conclusion was to proceed anyway, while explicitly tracking the issue, allowing the research program to evolve in execution rather than being blocked by the impossibility of perfect factorization at the outset. You also suggested a sensible calibration strategy: hand-code sharply limited domains to provide gold standards for evaluating automatically derived networks.
4) The generational scope: the birth of a field
You then widened the frame. The task is not merely technical. It is about how minds conceptualize the world, and not one mind but the historical product of millions or billions of minds writing across centuries, with bias toward recent decades. This is not a problem solvable by a single dissertation or a single lab over a few years. It requires many labs working in loose coordination, with both collaboration and competition, over one or more intellectual generations. In this view, foundation models are not “the pinnacle,” but the floor—the starting point—for a long new intellectual adventure.
In that context we coined useful names for two failure modes in contemporary AI thought: “hand-coded scholasticism” (the belief that meaning must be explicitly authored by armies of humans) and “scaled-up millenarianism” (uncritical faith that scaling alone will magically solve everything). You described these as the Scylla and Charybdis of current discourse, and emphasized that your program aims at a third path: mapping the latent wilderness systematically, with discipline and instrumentation.
5) Production systems and Yevick’s mode-switching intelligence
Finally, we returned to architecture. If diagramming skill is a library of pattern-to-pattern correspondences plus a critic enforcing coherence, then a classical production system architecture becomes attractive. A production system naturally supports staged rule application, working memory updates, constraint checking, and repair cycles. Neural models can supply candidate relations and associations, while the production system supplies explicit normativity and structural discipline.
This hybrid framing connects directly to Miriam Yevick’s work on holographic/Fourier logic versus sequential propositional logic. You emphasized that your current program is not merely compatible with Yevick’s ideas; it grew in part out of sustained reflection on them. You and Hays argued in 1990 that natural intelligence requires the capacity to deploy both modes, and you developed this further in speculative work on metaphor. In metaphor, the propositional system regulates the superimposition of holistic gestalts: e.g., Achilles in battle is likened to a lion in battle. The two scenes function as holographic wholes, while sequential linguistic propositions step through correspondence constraints. This provides a concrete mechanism for the hybrid intelligence thesis.
You concluded by noting the historical hinge: when you and Hays were working, the technical means for operating at scale on these ideas did not exist. Now they do. And Hays himself played a foundational role in building the early symbolic infrastructure of computational linguistics (machine translation at RAND, coining the term “computational linguistics,” founding editorship and institutional leadership in COLING). In effect, the present moment makes possible an extension of that lineage: not abandoning symbolic structure, but using symbolic grammars and production discipline to extract, organize, and refine the latent cognitive structures that neural models already embody.
No comments:
Post a Comment