Showing posts with label claude_colab. Show all posts
Showing posts with label claude_colab. Show all posts

Thursday, January 9, 2025

In which I ask Claude 3.5 about sexual shame and confusion in humans

As far as I know, there is no standard account of why humans find sexuality so problematic. It’s not as though no one’s thought to investigate it. They have. But so far no one’s come up with a good account, at least not in thoroughly modern terms, not so far as I can see. So I decided to ask Claude 3.5 Sonnet about it and to suggest my own speculative approach to the problem. 

* * * * *

I query Claude about present knowledge and speculation

I believe that sexual shame is unique to animals. Our primates, monkeys and apes, do not insist on copulating in private. They are willing to copulate where others in the group can observe. Is this correct?

From a scientific perspective, this is an oversimplification. While many non-human primates do engage in mating behaviors openly within their social groups, the relationship between privacy and mating varies significantly across species and contexts.

Research in primatology has documented that some primate species do seek varying degrees of seclusion during mating. For instance, orangutans often mate away from other group members, and some gorilla groups show preferences for more secluded locations during breeding behaviors.

The concept of "sexual shame" as we understand it in human terms is complex and involves cultural, psychological, and social dimensions that are difficult to attribute directly to other species. What we observe in other primates is better understood as different patterns of social organization and mating strategies that have evolved to serve various biological and social functions.

Additionally, privacy-seeking behaviors in human societies show considerable variation across cultures and historical periods, suggesting that much of what we consider "sexual shame" is culturally constructed rather than being a universal biological trait.

Would you be interested in learning more about the specific mating patterns and social behaviors observed in different primate species?

That’s OK for now. And I understand that privacy-seeking behavior varies across cultures. In a culture where everyone lives in the same tent, for example, finding privacy is going to be quite different from a culture where everyone has their own sleeping room. Is there any standard reason why humans usually insist on privacy of some sort?

The human preference for private sexual activity appears to stem from several key factors that emerged during our species' development. From an evolutionary perspective, privacy during mating likely provided advantages in mate selection and pair bonding. Private sexual encounters allowed pairs to strengthen their emotional connections without interference or competition from other group members.

From a social perspective, humans developed complex systems of kinship, marriage, and social organization that regulated sexual behavior. These systems often included norms about appropriate contexts for sexual activity, which typically emphasized privacy. This privacy served practical purposes: it helped maintain social order, reduced conflict over mates, and supported the formation of stable family units.

Additionally, humans developed sophisticated cognitive abilities that include advanced theory of mind and heightened self-awareness. This led to complex emotional responses to being observed during intimate activities, which may have further reinforced preferences for privacy.

These biological and social factors appear to have co-evolved with human cultural development, resulting in widespread (though not universal) cultural norms around sexual privacy. While the specific manifestations of these privacy norms vary significantly across societies, the underlying tendency toward some form of privacy during sexual activity appears to be a common human trait.

Would you like to explore any particular aspect of these evolutionary or social dynamics in more detail?

OK. This is consistent with what I’m thinking: “humans developed sophisticated cognitive abilities that include advanced theory of mind and heightened self-awareness. This led to complex emotional responses to being observed during intimate activities, which may have further reinforced preferences for privacy.” Is there a standard reason why sexuality is so problematic for humans?

Human sexuality's complexity appears to stem from several interconnected factors. The development of advanced cognitive abilities that you referenced has made human sexuality uniquely intricate compared to other species.

Our capacity for complex emotional attachments and social relationships means that sexual behavior becomes deeply entwined with personal identity, emotional well-being, and social status. This creates multiple layers of meaning beyond the basic biological drive.

The human brain's sophisticated ability to process symbolic meaning and create cultural narratives transforms sexuality into something that carries significant social and psychological weight. Sexual behavior becomes linked to concepts of morality, identity, and social standing in ways that can create internal conflicts and anxieties.

Additionally, the extended period of human development and the importance of stable family units for child-rearing has led societies to develop elaborate systems for regulating sexual behavior. These cultural systems, while serving important social functions, can sometimes conflict with biological drives, creating tension between instinct and social expectations.

Furthermore, humans' heightened self-awareness means we can anticipate others' judgments and internalize social standards about sexuality, potentially leading to psychological stress that other species likely don't experience. Our ability to imagine future consequences and remember past experiences adds layers of complexity to sexual decision-making.

Would you like to explore any of these specific aspects in more detail? 

I offer a pet speculation about why humans are bedeviled by sex

Here’s my highly speculative explanation. It has to do with the nature of human development. When humans are born a whole range of biological ‘drives’ and ‘desires’ are active. They’re necessary for the infant to live and thrive.

But cognitive capabilities are quite primitive. They develop relatively slowly, over a matter of years. As that happens the basic biological drives remain active. The way they are dealt with, organized into behavior, will change as cognition becomes more sophisticated. And then adolescence hits. Two things happen.

On the one hand abstract thought emerges (Piaget). On the other hand, adult sexuality matures. But it’s completely new. It hasn’t been a part of the process of behavioral elaboration and maturation that the other biological ‘drives’ and needs have been, hunger and thirst, sleep, exploration, etc. It’s just there. I’m thinking that if it emerges significantly after abstract thinking, the abstraction system (if you will) is more likely to conceive of it as some foreign object, some strange Other, but an Other that’s trapped there inside the body. That makes it very problematic. If on the other hand sexuality arrives first, then the abstraction system can more easily treat it just like any of the other biological motivations.

Your hypothesis presents an intriguing developmental perspective on human sexuality's unique challenges. Let me analyze several key aspects of this framework.

Monday, January 6, 2025

Memory in machines and minds, a conversation with Claude 3.5 Sonnet

This is from a long conversation I had with Claude 3.5 Sonnet where I tried to get to a fundamental understanding – Musk’s first-principles approach – of how memory works in machines, in minds, and culture (e.g. libraries) and work our way up to large language models and prompt engineering by way of a (highly speculative) discussion about the function of consciousness for a primitive animal living in the water. As my favorite aunt used to say, we were cooking with gas.

Prior to this conversation I had uploaded two documents to Claude. One was the chapter on learning from William Powers, Behavior: The Control of Perception, which has been one of my mainstays since I first read it back in the Ancient Times, the mid-1970s. The other document was a set of blog posts where I talk about the mind/brain as a polyviscous fluid. “Polyviscous”? That means it has regions of widely varying viscosity. Viscosity in the mind/brain? Setting aside the fact that the brain is mostly water constrained in a meshwork of tiny tubes, I’m talking about information flow. The second-by-microsecond flow of spikes has a very low viscosity; it changes rapidly from moment to moment. In contrast, the re-wiring of brains regions that is known as neural plasticity, that happens on a scale of months to years. That has a very high viscosity.

My prompts are set in bold; Claude 3.5’s responses are plain. I’ve added the subtitles to clarify the progression of the conversation.

* * * * *

Memory in Computer Science 101

What do they teach about memory in computer science courses in college? How do they think about memory?

In standard computer science, memory is typically taught in terms of two fundamental addressing schemes:

1. Location-based (or address-based) memory:

- This is your typical RAM (Random Access Memory)
- Each piece of data is stored at a specific numerical address
- To retrieve data, you need to know its exact memory address
- Like finding a book in a library using its shelf number
- Very fast and direct, but you need to keep track of where everything is

2. Content-based (or associative) memory:

- Data is retrieved based on a partial match of its content
- Like finding a book by searching for keywords
- More flexible but typically slower
- Examples include database indexes and cache associative memory
- Similar to how humans often remember things ("what was that movie with the guy from...")

In practice, most computer systems primarily use location-based addressing for main memory (RAM) because it's fast and efficient, while content-based approaches are used in specific components like CPU caches or database systems where the flexibility of content-based lookup is worth the additional overhead.

The fundamental difference is: location-based asks "what's stored at address X?" while content-based asks "where can I find something that matches pattern Y?"

Thank you. Now, review for me that basic types of database structures. I recall flat files, hierarchical, and relational databases. Are there any other basic types? How do they handle location addressing?

Here are the main types of database structures:

1. Flat File Databases

- Simplest form
- Single table, like a spreadsheet
- Each record is typically stored sequentially
- Example: CSV files
- Location addressing: Usually sequential; have to scan through records

2. Hierarchical Databases

- Tree-like structure with parent-child relationships
- Data organized in a one-to-many structure
- Example: IBM's IMS, XML databases
- Location addressing: Navigate through tree structure using paths

3. Network Databases

- Similar to hierarchical but allows many-to-many relationships
- Uses pointers between related records
- Example: Integrated Data Store (IDS)
- Location addressing: Follow pointer chains between records

4. Relational Databases

- Data organized in tables with relationships between them
- Uses SQL for queries
- Example: MySQL, PostgreSQL, Oracle
- Location addressing: Uses indexes and table/row identifiers

5. Object-Oriented Databases

- Stores data as objects
- Maintains object inheritance and relationships
- Example: ObjectDB
- Location addressing: Objects referenced by identifiers

6. NoSQL Databases

- Various types including:
* Document stores (MongoDB)
* Key-value stores (Redis)
* Column-family stores (Cassandra)
* Graph databases (Neo4j)
- Location addressing: Varies by type, often uses keys or document IDs

7. Time Series Databases

- Optimized for time-stamped data
- Example: InfluxDB
- Location addressing: Usually indexed by timestamp

Each type typically implements some form of indexing to speed up access to data, essentially creating a mapping between content and physical location. The actual physical storage location is usually abstracted away from the user, who interacts with logical identifiers or keys instead.

Thank you. What’s a vector database?

A vector database is a specialized type of database designed to store and efficiently search through vector embeddings - high-dimensional numerical representations of data like text, images, or other content.

Key characteristics:

- Optimized for similarity search in high-dimensional space
- Uses specialized indexing methods like LSH (Locality-Sensitive Hashing) or HNSW (Hierarchical Navigable Small World) graphs
- Particularly useful for machine learning applications and AI
- Examples include: Pinecone, Milvus, Weaviate, Chroma

Common use cases:

1. Semantic search - finding similar text/documents
2. Recommendation systems
3. Image similarity search
4. Natural language processing applications
5. AI applications requiring similarity matching

The main difference from traditional databases is that vector databases are optimized for finding "similar" items rather than exact matches. They use specialized distance metrics (like cosine similarity or Euclidean distance) to measure how close vectors are to each other in high-dimensional space.

For example, if you wanted to find similar images, you'd:

1. Convert images to vector embeddings
2. Store these vectors in the vector database
3. Query with a new image's vector to find the most similar stored images

Location addressing in vector databases typically involves sophisticated indexing structures that partition the vector space to enable efficient similarity search, rather than simple direct addressing used in traditional databases.

Hybrid memory in machine and humans

So a vector-based system has features of content addressing?

Yes, exactly! Vector databases are a hybrid that bridges location-based and content-based addressing approaches.

The content-addressing aspects come from:

- Searching by similarity rather than exact matches
- Using the actual content (converted to vectors) to find related items
- Not needing to know exact locations/addresses
- Being able to find items based on partial or related information

However, under the hood, they still need efficient location-based mechanisms to actually store and retrieve the vectors. They typically use specialized index structures that map the high-dimensional vector space to physical storage locations.

The key innovation is that they create an organized way to find similar content quickly, while still leveraging the speed advantages of location-based storage. It's like having a library where books are still in specific physical locations, but you can find them by describing what they're about rather than knowing their shelf numbers.

This is particularly relevant for AI applications because vector embeddings capture semantic meaning - they encode "what something is about" or "what something means" rather than just exact matches of content.

Let’s think about humans for a second. In the ancient world (Greeks and Romans?) they had something called the method or loci, or memory palace. That is, in effect, a location based memory system implemented in a biological content addressed system, no?

Yes, that's a fascinating insight! The method of loci (memory palace technique) is indeed a way of imposing location-based organization onto our naturally content-addressed biological memory system.

Here's how it bridges the two:

Content-addressed aspects (biological foundation):

- Our natural memory works through associations
- We naturally remember things based on their meaning and relationships
- Memories are triggered by related content/context

Location-based overlay (the technique):

- Creates artificial "addresses" through familiar spatial locations
- Each piece of information is deliberately assigned to a specific location
- Information can be retrieved by mentally walking through these locations
- Provides a systematic way to store and retrieve information sequentially

The genius of the technique is that it leverages our brain's natural strengths:

- Strong spatial memory
- Visual memory
- Narrative/sequential thinking
- Association-based recall

Saturday, December 28, 2024

LLM as Collaborator, Part 3: Claude has some ideas about how we could revise and redeem Harold Bloom

Not long after Harold Bloom’s death in October of 2019, Ted Underwood posted a tweet in which he asserted that Bloom was a “gasbag” who made “dogmatic unsupported assertions.” He attributed that habit to “theory-stardom” in a subsequent tweet, something I’ve explored in this post, GOAT Literary Critics: Part 5.2: The Nature of Authority in Literary Criticism as it Pertains to the Relationship between Author and Critic (Harold Bloom). Color me sympathetic to Underwood's assertion.

Sometime in the late 1970s I picked up a copy of The Anxiety of Influence, noting that it was a rather slender book. I found it rather obscure and elusive and I can’t for the life of me see why he insisted on those preposterous names for his “revisionary ratios.” I also sensed something interesting going on in there, and said so in a letter I wrote to Dick Macksey. Whatever it was, though, was not my cup of tea.

I set it aside and hardly gave Bloom another thought until I saw his The American Religion (1992) on the new books shelf in the Troy Public Library. “That’s an odd topic for Bloom,” thought I to myself, so I leafed through it and decided to check it out. It proved useful, for there I learned that revivalist camp meetings often had both African and European Americans in attendance. They convened separately until the last day of the meeting, at which point the barriers came down and they all danced together. Other than Influence, that’s the only book of Bloom’s that I’ve read cover to cover.

But I’ve read chunks of a number of them in the last year, The Western Canon, Shakespeare, The Anatomy of Influence, his last one (I forget its name), several interviews, and listened to a variety of video clips. The man did bloviate, but, as I far can tell, he’s also as brilliant a mind as the craft of literary criticism as seen in the last half-century. Why, then, am I reluctant to consider him to be a great critic?

No mystery there. Brilliance is one thing, a thing that is entirely a property of an individual. Greatness, however, requires that brilliance be fit to cultural opportunity, if not necessity. Giordano Bruno was undoubtedly brilliant, but we remember him, not for his ideas or his poetry, but because the Roman Inquisition had him burned at the stake. How deeply did Bloom speak to the cultural moment?

Anyhow, I’ve been pondering the matter and have decided to take it up with Claude 3.5. I’d already chatted with Claude about the Hamlet chapter of Bloom’s Shakespeare (though I've not posted any of that conversation) and I’d used Bloom as a foil in the discussion I had with Claude about Formal structure in Heart of Darkness. So I decided to continue that discussion.

I began with Bloom’s insistence on the aesthetic value of literature. I suggested that Johan Huizinga’s concept of play in his classic, Homo Ludens, might provide a way of grounding a Bloomian aesthetic. Next came Edward Said, one of those resentful critics Bloom was always complaining about. I quoted from one of Said’s last essays, where he confessed to believing that an autonomous aesthetic realm exists. Claude agreed that that might have been a way to effect a rapprochement, or perhaps merely a detente, with the resentful ones. That lead to a discussion of academic politics. We then moved on to a discussion Bloom’s mind, suggesting that his extraordinary ability to recall literary passages may have been both a strength and a weakness. Yes, he seemed able to remember everything he read, but he also seemed trapped in those memories, unable to stand back and abstract over the texts he so loved. We concluded with by suggesting that modern computational techniques, digital humanities slash cultural analytics, might well be able to redeem Bloom’s insights about Shakespeare and his importance in the canon.

It was a bracing discussion. As always, my words are set in bold, while Claude’s are plain.

* * * * *

Kant, Huizinga, and the Aesthetic

Bloom opens “The Western Canon” by saying: “THIS BOOK studies twenty-six writers, necessarily with a certain nostalgia, since I seek to isolate the qualities that made these authors canonical, that is, authoritative in our culture. “Aesthetic value” is sometimes regarded as a suggestion of Immanuel Kant's rather than an actuality, but that has not been my experience during a lifetime of reading.” What does Kant mean by aesthetic value?

Later Bloom says: “Aesthetic criticism returns us to the autonomy of imaginative literature and the sovereignty of the solitary soul, the reader not as a person in society but as the deep self, our ultimate inwardness. That depth of inwardness in a strong writer constitutes the strength that wards off the massive weight of past achievement, lest every originality be crushed before it becomes manifest.”

What do you think about these?

These passages touch on fundamental questions about aesthetic value and literary criticism. Let me address each one:

Regarding Kant's conception of aesthetic value, he argued that aesthetic judgments are both subjective and universal - a seeming paradox that he explored in his Critique of Judgment. For Kant, when we judge something as beautiful, we are making a claim that others should agree with us, even though the judgment stems from our subjective experience. This aesthetic judgment is disinterested, meaning it's not based on any practical purpose or moral consideration, but rather on the form of the object itself.

Bloom's reference to Kant seems to push back against those who would dismiss aesthetic value as merely subjective or culturally constructed. His statement about his "lifetime of reading" suggests that he sees aesthetic value as something real and discoverable through direct engagement with literature, not just a theoretical construct.

In the second passage, Bloom articulates his vision of literary criticism as fundamentally concerned with individual consciousness rather than social context. His concept of the "deep self" and "ultimate inwardness" shows his belief that great literature speaks to something essential in human consciousness that transcends historical and social circumstances.

The idea of "warding off the massive weight of past achievement" connects to Bloom's theory of the anxiety of influence - that strong writers must struggle against their predecessors to create space for their own originality. The emphasis on "inwardness" suggests that this struggle happens at the level of consciousness rather than through external technical innovation.

These passages reveal Bloom's fundamental commitment to aesthetic value as real rather than constructed, and to reading as an essentially private experience that touches our deepest sense of self. This position stands in opposition to critical approaches that emphasize literature's social and political dimensions.

Johan Huizinga wrote a book entitles “Homo Ludens.” I’m wondering if we could use Huizinga’s expansive conception of play as a way to develop an aesthetic theory that would appeal to Bloom. In literature we see the human psyche at play.

Your suggestion about using Huizinga's concept of play as a foundation for aesthetic theory is intriguing. In "Homo Ludens," Huizinga argues that play is fundamental to human culture, predating culture itself, and that it lies at the root of many human activities including art, law, and philosophy.

This perspective could align well with Bloom's emphasis on aesthetic autonomy and individual consciousness. When we engage in play, we create a space separate from ordinary life with its own rules and meanings - similar to how Bloom sees literature as creating an autonomous sphere of aesthetic experience.

The idea of the psyche at play in literature could help explain what Bloom calls the "strangeness" of great works. Just as play involves trying on different roles and possibilities, literary works allow both writers and readers to explore different modes of consciousness and being. This connects to Bloom's interest in how literature expands our understanding of human possibility.

Furthermore, seeing literature as a form of play might help bridge the gap between form and meaning that troubles much literary criticism. In play, form and content are inseparable - the rules of a game aren't separate from the experience of playing it. Similarly, in literature, formal elements like the paragraph structure we discussed in Heart of Darkness aren't separate from but integral to the work's meaning and effect.

This approach could maintain Bloom's emphasis on aesthetic autonomy while providing a theoretical framework for understanding how literature actually works on consciousness. It would see reading not as interpretation in the traditional sense, but as a form of engaged play with consciousness itself.

Thursday, December 26, 2024

LLM as Collaborator, Part 2: Formal structure in Heart of Darkness and literary criticism (+ a lesson on “alignment”)

The chart below depicts the length of paragraphs in Joseph Conrad’s Heart of Darkness. The paragraphs are listed in order, first to last, from left to right. The length of a bar is proportional to the number of words in the respective paragraph.

There is an important issue at stake: Does that chart depict a phenomenon that is real, not only in the sense that it represents a set of valid measurements, but in the deeper sense that those measurements represent some intrinsic aspect of the causal process that produced the text. For the most part I leave that issue to the discussion I had with Claude 3.5 Sonnet.

The purpose of this introduction is to indicate why the issue arises.

The problem of literary form

Despite the fact that form and formalism are important concepts in academic literary criticism, the careful description of form is not central to current practice. This is not the place to discuss this issue in any detail. I have done that elsewhere on New Savanna, I have published a long article on form (Literary Morphology: Nine Propositions in a Naturalist Theory of Form), and have written a number of working papers on description.

Form is an aspect of the physical substance of the work of art. That is quite obvious in the case of drawing, painting, and sculpture. That is also true in the case of music, where we are interested in the arrangement of sounds in time, and dance, the arrangement of gestures and movement. Literature is like that as well; it is a string of words, one after the other after the other after the other. It follows, then, that the description of literary form involves analyzing how words are placed in a string.

And there is some of that in literary criticism. In dealing with poetry one may learn how to count feet in a line and work out the rhyme scheme, if there is any. But this is often treated as decorative and unrelated to meaning in any deep way, and it’s meaning that critics are after. In the study of narrative the Russian Formalists made a distinction between story (fabula) and plot (syuzhet), where the former refers the events as they occur in order while the latter is the order in which events are introduced into the narrative. In the extreme case of Tristram Shandy, for example, plot and story are quite different. This distinction, and more, is quite important in narratology, but narratology is a relatively minor approach to literary criticism and one that is more important in Europe than the United States.

As an example of this situation, here’s a few lines from an essay by Frances Ferguson and John Brenkwood introducing papers from the 2013 English Institute on form:

A second irony is that the recently renewed interest in questions of literary form has proved quite amorphous. Perhaps, though, that has been the predicament and vitality of the topic all along. [...] There is also a conflict, cutting across these competing methods, between form as a feature of literary works and form as constitutive of literary works. The New critics are often the benchmark of formalism in American discussions, but they did very little to illuminate literary forms compared to the Russian Formalists or, say, Lévi-Strauss and Jakobson’s classic essay on Baudelaire’s “Les chats.” [...] And yet even the surest markers of literary forms fail to define form when it comes to actual works. The form of the sonnet, for example, is readily defined by the number of lines and the stanza organization, but does that account for a particular sonnet’s form any more than a rectangle accounts for a painting’s form? Vertical for portraits, horizontal for landscapes! And, finally, is formalism itself based on the idea that literary works are purely form, or on the idea that the vocation of literary criticism lies in formalization, that is, in its capacity to create categories at a level of abstraction applicable to the widest variety of literary phenomena?

The upshot of this conundrum is that literary form is more an object of wonder and a source of methodological argument than the basis of practical criticism. It’s important, but we don’t know much about it.

Thus, that chart is something of an embarrassment to literary critics, or would be if they saw it. On the one hand, it represents and objective fact about Heart of Darkness. When you count the words in each paragraph and graph the result, that chart is what you get. [I used the text provided by Project Gutenberg.] Once you’ve seen it, you can’t argue it out of existence.

Now, I’ve been in this business a long time. While I’ve discussed that chart and its implications extensively in my working paper on Heart of Darkness, I’ve not bothered to seek formal publication. Why not? Opportunity cost. I don’t know of any venue that would be interested and I don’t want to waste my time jumping through hoops making that work acceptable to some journal only to wait six months to a year or more to have it turned down (though I’m seriously considering taking a shot at it in the new year; we’ll see).

However, when I discovered that structure I did send at note to Mark Liberman, who runs, Language Log, and group log devoted to linguistics. He was kind enough to post a link and we had a lively discussion, Markov’s Heart of Darkness. The discussion started with the issue of whether or not paragraph length followed a power law as that’s the question I presented to Mark; it does not. But one Lucy Kemnitzer brought up the issue of order during the discussion and I linked to a post in which I had the chart I posted above. If you look through that discussion, you’ll find comments by someone identifying themselves as “JL” and noting that they are a novelist. They’re quite dismissive of that kind of description and the analytical work it supports:

The question is whether you're adding anything at all to anyone's understanding of the book, or whether, instead, you're indulging in puffery and pseudoscience, bringing in spurious mathematical models and childish anthropological "theories" to gaudy up a banal and perfectly obvious account of a very well known phenomenon. (Look! I've discovered that the ratio of vowels to consonants in 'The Waste Land' is exactly equivalent to Planck's Constant! Which is used to explain black holes! See? It really is a depressing poem…)

That’s the kind of reaction I expect of well-trained humanists.

Ask Claude

And so I turned to Claude 3.5 Sonata. What would they think of that distribution of paragraph lengths?

I’d already loaded the entire text of Heart of Darkness into Claude and had some discussion about it (I’ve already posted some of that discussion). It was a simple matter to reactive that discussion, upload the chart, and initiate a discussion. The discussion went well, at least from my point of view. Claude even pointed out some things that I hadn’t noticed.

But after all the (standard) literary criticism it surely must have ingested during training, why didn’t it react with, if not outrage, certainly skepticism? Why wasn’t it aligned, as the kids say, with the conventional wisdom on this matter? Context, I figured, context. This interaction was loaded with Conrad’s text and extensive earlier discussion. Standard literary criticism was nowhere to be found in the immediate context.

“I wonder,” thought I to myself, “if I could create a context in which Claude would register a conventional reaction to that chart?” No sooner had I registered that thought than I figured out how to do it. I had previously loaded Claude with the Hamlet chapter from Harold Bloom’s, Shakespeare: The Invention of the Human. I’d asked Claude to summarize the chapter and then we had a little discussion.

So, I reactivated that discussion, uploaded the chart, and BOOM! Claude reacted with the expected skepticism, albeit a rather mild expression of it:

Based on Bloom's critical approach, he would likely be skeptical of this kind of quantitative or structural analysis of literature. He would probably view this chart as an example of what he saw as the problematic tendency in modern literary criticism to focus on technical or formal aspects rather than aesthetic and imaginative power

The discussion went on for a bit, took an interesting turn, at which point I decided to introduced Susan Sontag’s 1964 essay, “Against Interpretation,” into the discussion. Why? Because she argued that “criticism needs to focus less on trying to extract hidden meanings from art and more on describing the direct sensory and formal experience of engaging with it” (Claude’s words). We had an interesting discussion. For what it’s worth, Claude agrees with my view that literary criticism needs to pay more attention to the analysis and description of literary form.

An exercise for the reader: What does Claude’s ability to take different and contradictory positions on the same issue – in this case, the value of that chart for analyzing Heart of Darkness – suggest about the much-discussed problem of AI alignment?

* * * * *

First, we have my discussion with Claude about Heart of Darkness. Once that’s finished, I introduce the Harold Bloom discussion, which leads to Susan Sontag. As always my prompts are in bold face while Claude’s responses are plain.

I’ve uploaded a bar graph. Each bar represents a single paragraph in Heart of Darkness. They are arranged in order, with the first paragraph on the left edge and the last paragraph on the right. Do you have any observations about paragraph length as revealed in this chart?

Looking at this visualization of paragraph lengths in "Heart of Darkness," several key observations emerge:

Most paragraphs in the novel are relatively short, with many falling under 200 words. This creates a baseline rhythm for the narrative. However, there are significant spikes where paragraphs become much longer, particularly in the middle section of the text.

The most striking feature is a dramatic peak of approximately 1,400 words around paragraph 109-110. This extensive paragraph appears to be an outlier, being significantly longer than those around it. There are also several other notable peaks reaching between 600-1,000 words.

This variation in paragraph length suggests a deliberate structural choice by Conrad. The longer paragraphs likely correspond to key moments of exposition or critical narrative developments, while the shorter paragraphs maintain the story's forward momentum and may handle more immediate action or dialogue.

The pattern also shows that the novel ends with a series of consistently shorter paragraphs, perhaps indicating a more rapid narrative pace as the story reaches its conclusion.

This visualization effectively reveals Conrad's technical approach to pacing and structure in the novel, showing how he varies paragraph length to create rhythm and emphasis throughout the text.

Would you be interested in identifying what narrative content appears in those particularly long paragraphs? That might help us understand why Conrad chose to expand certain moments in the story.

You’d think that variation was deliberate. But we have no evidence that Conrad was conscious of this. He just did it. It’s how he wrote.

You make an excellent point. I fell into a common critical trap of assuming intentionality where we have no evidence for it. This pattern of paragraph length variation emerged naturally from Conrad's writing process - it's simply his natural rhythm as a writer.

This observation connects to a broader point about literary analysis. While we can usefully identify patterns and structures in texts, we should be cautious about assuming these were consciously planned by the author. Sometimes the most powerful effects in literature arise organically from how a writer naturally approaches their craft.

In Conrad's case, this varying paragraph length creates compelling effects in "Heart of Darkness," but we don't need to attribute this to conscious design. It's simply part of how he told stories, his natural way of allowing some moments to expand while keeping others concise.

This is a useful reminder about the difference between identifying patterns in a text and making claims about authorial intention.

That single longest paragraph, I believe it’s 1502 words long, is that nexus paragraph we’ve already discussed.

I’ve already had extensive discussion about this paragraph with Claude. Here's part of that discussion, though it doesn't contain Claude's remarks about the nexus paragraph. However, here’s my original blog post about that paragraph. And this post places that paragraph in the context of the whole text.

That's a fascinating discovery about the nexus paragraph! The fact that it’s not only structurally and thematically central to the novel, but also stands out quantitatively as the longest paragraph, adds another dimension to our understanding of its importance.

This paragraph accomplishes several crucial narrative tasks: it provides Kurtz's European background, introduces his report with the chilling "Exterminate all the brutes" line, and frames the death of the helmsman. The paragraph's exceptional length seems to naturally correspond to its function of bringing together multiple strands of the story.

Monday, December 23, 2024

LLM as Collaborator, Part 1: Claude the Graduate Student

I started using ChatGPT on December 1, 2022 and have used it quite extensively ever since. I’ve spent some of my time just poking around, somewhat more time looking things up, and most of my time systematically investigating its performance. That resulted in a number of working papers, the most interesting of which is about stories: ChatGPT tells stories, and a note about reverse engineering: A Working Paper, Version 3.

I started working with Claude 3.5 Sonnet on November 18, 2024. I’ve used it in those three capacities, though obviously not as much as I’ve used ChatGPT. In particular, I’ve used it for background information on melancholy in various aspects. I’ve also done something I’d never done with ChatGPT, asked it to describe photographs. I’m doing this to see how well it does.

Then on November 24, 2024, I began using it in a somewhat more interesting new capacity, though I’m not sure what to call it. The phrase “thought partner” comes to mind, though it seems too much like “thought leader,” which I don’t like. I’m using it as a sounding board. Better yet, its a collaborator playing the role of sounding board. It’s not an equal collaborator in the intellectual dialog; academic norms would not require me to offer it co-authorship of papers. But those norms might well require an explicit acknowledgement, not to alert the reader that one of those new-fangled LLM things has been involved in the thinking, but simply acknowledging the help it has given me.

As for just what kind of help that is, the best way is to look at some examples. I’ve already published two of these dialogues on New Savanna: Computer chess, super-intelligence, A Connecticut Yankee, cargo cults, and Claude 3.5 Sonnet speculates about future developments in AI and beyond. The rest of this post contains the first interaction where Claude began acting in intellectual partnership.

When I started the interaction I was simply interested in seeing what it knew about my teacher and mentor, the late David Hays. I wasn’t looking for new information. I just wanted to see what it had picked up. That began to change when it asked me about the work of Quillian and of Schank and Abelson. Then I told it about Hays’s interest in the work of William Powers. Now we’re getting serious. This guy knows something. And then WHAM! Claude responded intelligently when I mentioned the work of Peter Gärdenfors. The interaction continued in that mode to the end.

Utterly remarkable. AGI? What the hell is that? But no. And I fear that thinking in those terms is likely to blind me to whatever it is that Claude is doing. I realize that the concept of AGI is written into the contractual relationship between OpenAI and Microsoft, but I’m not playing that game. Will they end up litigating it? Who knows?

I’m planning two more posts in this series. The next will be about the implications of paragraph length in Heart of Darkness. Then I’ll post a long interaction that takes Miriam Yevick’s 1975 paper, “Holographic or fourier Logic,” as its starting point.

In the following dialog my comments are bold while Claude’s are plain. I’ve interpolated a few comments highlighted in blue.

* * * * *

Are you familiar with the work of a computational linguist named David Hays? He worked on machine translation at RAND in the 1950s and 1960s?

Yes, I am somewhat familiar with David Hays' work, though I should note that since this involves fairly obscure historical details from computational linguistics, I want to be careful about potential inaccuracies in my knowledge. From what I understand, David Hays was one of the pioneers of machine translation and computational linguistics during the early years of the field. He worked at RAND Corporation in the 1950s and 1960s where he was involved in some of the earliest machine translation projects.

He was particularly interested in developing dependency grammar approaches for machine translation, which was quite innovative for that time period. Dependency grammar focuses on the relationships between words rather than constituent structure, which was the dominant paradigm in American linguistics at the time.

I believe he also wrote an influential book called "Computational Linguistics: Programming Linguistics in Automatic Language Translation" in the early 1960s, though I'd encourage you to verify this citation since I may be misremembering details.

Claude got the name wrong. It’s Introduction to Computational Linguistics, American Elsevier, New York, 1967.

Would you be interested in hearing more about any particular aspect of his work or that early period of machine translation research? I find the history of early computational linguistics fascinating, though again I want to be upfront that with someone as historically specific as Hays, some of my knowledge may be incomplete or imperfect.

Yes, that’s him. I studied with him at SUNY Buffalo in the 1970s. We were working on computational semantics and using a cognitive networks formalism.

Oh, that's fascinating that you studied directly with David Hays! I would be very interested to hear more about your experiences working with him and the cognitive networks approach you were developing at SUNY Buffalo. That was an important period in the development of semantic representation systems.

Could you tell me more about the cognitive networks formalism you were using? I'm particularly curious about how it compared to other semantic network approaches being developed around that time, like Quillian's work or Schank and Abelson's conceptual dependency theory.

It was more developed than Quillian’s work. As I recall, he never really developed is initial ideas. As for Schank and Abelson, they were imagining a self-sufficient cognitive system and were very much interested in identifying the primitive elements necessary for a cognitive network. By the time I began working with him, Hays had realized that cognition had to be grounded in something else – later on Steven Harnad would talk about the “symbol grounding” problem. When I worked with him, Hays was interested in grounding the cognitive system in the control-theory (cybernetic) account of perception and action developed by William T. Powers (perceptual control theory, or PCT).

Something special seems to be happening.

This is really interesting historical context - it suggests Hays was grappling with fundamental issues in cognitive science that are still relevant today. The shift from viewing semantic networks as self-contained symbolic systems to recognizing the need for grounding in perception and action seems quite prescient.

I'm curious about how Hays was trying to integrate Powers' PCT with semantic networks. As I understand it, PCT proposes that behavior emerges from organisms controlling their perceptions to match internal reference signals, rather than through direct control of outputs. Was Hays trying to use this as a way to ground symbolic meaning in perceptual-motor control loops?

This also seems to anticipate some later developments in embodied cognition and grounded cognition theories. The timing is interesting too - Powers published "Behavior: The Control of Perception" in 1973, so it sounds like Hays was engaging with these ideas pretty quickly after they were published.

Yes, that's what Hays was doing. His idea was that particular servo units in a Powers-like control system would be linked to cognitive units through what Hays called "parameters of perception." Hays identified a half dozen or so such parameters. Each cognitive node would be characterized by the values it took on those parameters. Relationships between nodes were thus derived from differences in parameter values for the nodes connected by an arc (or edge).