Some informal notes on higher-level structures in LLMs

By higher-level I simply mean texts that are more than a sentence long, typically two or more paragraphs, and sometimes many paragraphs. Just how LLMs organize this kind of structure is unknown. I don’t intend to dissolve the mystery in this post. I just want to put some examples on display. First a story, then some biographies, real and not, then a bit of confabulated history, and concluding with word-for-word repetition.

Stories

When I prompt ChatGPT with single word “story” – which I’ve just done – this is how it responds:

Once upon a time in a quaint little village nestled between rolling hills and lush forests, there lived a young girl named Isabella. Isabella was known far and wide for her insatiable curiosity and a heart full of kindness. She lived with her grandmother, Agnes, in a cozy cottage at the edge of the woods.

Every day, Isabella would explore the woods, chasing butterflies and collecting wildflowers. She was especially fond of a meadow filled with vibrant, fragrant blooms, where she often sat to read her favorite books or sketch in her journal. Her beloved companion was a friendly bluebird named Oliver, who sang sweet melodies as they roamed the woods together.

One sunny morning, as Isabella wandered deeper into the forest, she stumbled upon a mysterious, ancient tree with bark as dark as night. Its massive roots seemed to twist and turn like serpents, and the leaves were a brilliant shade of emerald green, unlike any other tree in the forest. Isabella's curiosity got the best of her, and she reached out to touch the tree's gnarled trunk.

As her fingers brushed the bark, a sudden gust of wind swirled around her, and the tree shivered as if waking from a deep slumber. The tree's branches extended, revealing an entrance to a hidden chamber within its hollow trunk. Isabella, fueled by her curiosity, entered the chamber, where she discovered an ancient, leather-bound book. Its pages were filled with enigmatic symbols and drawings of magical creatures.

Isabella carefully flipped through the book, her heart racing with excitement. She realized that she had stumbled upon a spellbook, full of spells that could heal, protect, and even unlock the secrets of the forest. Isabella knew that this discovery was a gift, and she felt a responsibility to use the knowledge within the book for the benefit of her village and the natural world.

Over the next few weeks, Isabella studied the spellbook diligently, and with the help of her bluebird friend, Oliver, she began to master its enchantments. She learned how to heal wounded animals, make the crops grow faster, and protect the forest from harm.

Word of Isabella's newfound abilities spread throughout the village, and people from all around came to her for help and guidance. She became known as the "Forest Enchantress," and her kindness and wisdom touched the hearts of all who met her.

One day, a great darkness threatened the village. A band of ruthless loggers arrived with the intention of felling the ancient forest for profit. Isabella knew she had to use her magic to protect the woods and its inhabitants. With the spellbook in hand, she called upon the spirits of the forest and cast a powerful spell that caused the loggers to see the beauty and importance of the woods. Their hearts were changed, and they abandoned their destructive mission.

Isabella's act of kindness and her extraordinary abilities made her not only a protector of the forest but also a symbol of hope and unity for her village. Her grandmother, Agnes, beamed with pride, knowing that her granddaughter was destined for greatness.

And so, in the heart of the enchanting forest, a young girl's curiosity and her connection to the natural world had brought about a brighter future for all. Isabella and her village lived in harmony with the forest, cherishing its beauty and preserving it for generations to come.

It tells me a story. By now I’ve done this hundreds of times, though my prompt is generally longer, e.g. “Tell me a story about a hero” or “Tell me a true story.” Each time, ChatGPT gives me a well-formed, if somewhat simple, story.

It wouldn’t be able to do that unless it had some conceptual of what a story is. Just how it does that, however, is something of a mystery, one that I’ve explored a bit: ChatGPT tells stories, and a note about reverse engineering. However it does so, I’m pretty sure that ChatGPT is not using a “traditional” story grammar of that sort investigated in the 1970s, 1980s, and after, though I don’t attempt to make that particular argument in my paper.

Biography, real and confabulated

If I prompt it with “Roland Kirk,” it will give me a potted biography:

Roland Kirk, whose birth name was Ronald Theodore Kirk, was a highly influential and innovative American jazz multi-instrumentalist. He was born on August 7, 1935, in Columbus, Ohio, and he passed away on December 5, 1977. Kirk was known for his extraordinary ability to play multiple wind instruments at the same time, his exploration of avant-garde jazz, and his dedication to musical experimentation. Here are some key points about Roland Kirk:

1. Early Blindness: Roland Kirk lost his sight at a young age, becoming blind by the age of two due to a congenital eye disease. Despite his blindness, he developed an incredible sensitivity to sound and a deep love for music.
2. Versatile Musician: Kirk was a highly versatile musician who played multiple wind instruments, often simultaneously. He was known for playing the tenor saxophone, manzello (a modified saxello), stritch (a straight alto sax), flute, clarinet, and a variety of other instruments, sometimes all at once. This unique approach earned him the nickname "Rahsaan Roland Kirk."
3. Innovative Techniques: Kirk pioneered several innovative techniques in jazz, such as circular breathing (the ability to continuously blow into a wind instrument without pausing) and playing multiple instruments simultaneously. His unconventional methods pushed the boundaries of what was traditionally expected in jazz music.
4. Eclectic Style: Roland Kirk's music was highly eclectic and drew from a wide range of influences. He incorporated elements of bebop, post-bop, free jazz, rhythm and blues, and even pop music into his compositions and improvisations.
5. Avant-Garde Jazz: Kirk was associated with the avant-garde jazz movement, and his music often featured experimental and unorthodox elements, such as vocalizing while playing, playing two or more saxophones at once, and creating a diverse palette of sounds.
6. Social and Political Commentary: Kirk's music often contained social and political commentary. He used his compositions to address issues such as civil rights, racial equality, and social justice.
7. Albums and Recordings: Roland Kirk released numerous albums as a bandleader, starting with "Triple Threat" in 1957. Some of his other notable albums include "Rip, Rig, and Panic" and "The Inflated Tear."
8. Live Performances: Kirk was renowned for his captivating live performances, where he engaged with the audience and incorporated humor and theatrical elements into his shows.
9. Legacy: Roland Kirk's contributions to jazz, his unorthodox playing techniques, and his boundary-pushing musical innovations have left a lasting impact on the world of music. He influenced a generation of musicians and continues to be celebrated as a jazz maverick.

Roland Kirk's innovative and groundbreaking approach to music and his ability to transcend the limitations of his blindness made him a remarkable and unique figure in the world of jazz. His work continues to inspire and influence musicians exploring the boundaries of the genre.

I know Kirk’s work fairly well and the biography appears to be accurate. That structure is somewhat different from the story structure. But just as I don’t think ChatGPT doesn’t generate stories using classical story grammars, I rather doubt that it uses some kind of classical “frame” structure to generate the structure of that biography, though I’m not prepared to make an argument on that point – though I could make such an argument for stories but just haven’t wanted to do so.

If I prompt it with “Miriam Yevick” I get this:

Miriam Yevick was a pioneering American physicist known for her contributions to the field of statistical mechanics, particularly in the study of phase transitions and critical phenomena. She was born on August 30, 1928, in Brooklyn, New York, and she passed away on January 4, 1992. Here are some key points about Miriam Yevick and her contributions to the field of physics:

1. Educational Background: Miriam Yevick earned her Ph.D. in physics from the University of Chicago in 1954. Her doctoral work was supervised by the renowned physicist and Nobel laureate Enrico Fermi.
2. Statistical Mechanics: Yevick's research focused on statistical mechanics, a branch of physics that deals with the behavior of systems with a large number of particles, such as gases and liquids. She made significant contributions to the understanding of phase transitions, where matter changes from one state (e.g., solid, liquid, gas) to another, and the study of critical phenomena, which occur near phase transition points.
3. Scaling Theory: Yevick is particularly known for her work on scaling theory, which is a framework for understanding the behavior of physical systems near critical points. Her research helped develop a deeper understanding of the universality of critical phenomena, where the same basic behavior is observed in a wide range of different systems.
4. Professional Career: Miriam Yevick held various academic positions throughout her career. She worked at institutions such as the University of Chicago, Columbia University, and the City College of New York, where she made valuable contributions to both research and education.
5. Recognition and Legacy: Yevick's work in statistical mechanics and scaling theory earned her recognition and respect in the field of physics. Her research continues to influence the study of phase transitions, critical phenomena, and the behavior of matter at a fundamental level.
6. Women in Physics: As a female physicist in a field traditionally dominated by men, Yevick broke barriers and served as an inspiration to aspiring women scientists. She contributed to the broader discussion about gender diversity in the field of physics.

Miriam Yevick's contributions to the field of statistical mechanics and critical phenomena have left a lasting impact on the understanding of the behavior of matter in the presence of phase transitions. Her work continues to be relevant and influential in the study of complex physical systems.

Miriam Yevick is, in fact, a real person, but that biography is a complete fabrication. There have been occasions when I’ve prompted it with her name and I would tell me that it knew nothing about this person. Other times, it would fabricate a biography. It wouldn’t be able to do that unless it had some conception of what a biography should look like. Notice the formal similarity between this biography of the imaginary Yevick and the previous biography of the real Roland Kirk, a numbered list between introductory and closing paragraphs.

Confabulated history

What happens in response to this prompt “Johnstown Flood, 1953”?

The Johnstown Flood of 1953 was a devastating natural disaster that occurred in the city of Johnstown, Pennsylvania, in the United States. This flood, often referred to as the Johnstown Flood of 1953, was a result of the heavy rainfall caused by the remnants of Hurricane Connie and Hurricane Diane, which struck the area in quick succession. Here are the key details of the Johnstown Flood of 1953:

1. Background: In August 1953, the region around Johnstown, Pennsylvania, experienced a series of heavy rainfalls, largely due to the combined impact of Hurricane Connie and Hurricane Diane. The rainfall was particularly intense, with some areas receiving more than a foot of rain in just a few days.
2. Conemaugh River: Johnstown is located at the confluence of the Stonycreek River and the Conemaugh River. These rivers, along with the Little Conemaugh River, play a crucial role in the city's geography. The heavy rains caused these rivers to swell significantly.
3. Dam Failures: The intense rainfall and the already saturated ground led to the failure of several dams in the region. The most significant of these was the failure of the South Fork Dam, which had been modified into a private lake by the South Fork Fishing and Hunting Club.
4. Destruction: The breach of the South Fork Dam released a massive amount of water, which rushed downstream toward Johnstown with catastrophic force. The wall of water struck the city, causing widespread destruction and loss of life. The floodwaters also carried debris, including trees and buildings, which compounded the damage.
5. Casualties: The 1953 Johnstown Flood resulted in a significant loss of life. An estimated 2,208 people lost their lives, making it one of the deadliest floods in U.S. history. Thousands of homes and buildings were destroyed.
6. Recovery and Rebuilding: In the aftermath of the disaster, Johnstown received assistance from the federal government and various relief organizations. The city embarked on a long process of recovery and rebuilding.
7. Flood Control Measures: The Johnstown Flood of 1953, along with the historic Johnstown Flood of 1889, prompted increased efforts to improve flood control in the region. A system of reservoirs, flood control projects, and warning systems was established to mitigate the risk of future flooding.

The Johnstown Flood of 1953 was a tragic event that had a profound impact on the city and its residents. The disaster highlighted the need for better flood control and disaster preparedness in the region, leading to significant improvements in the decades that followed.

That’s a fabrication. There was no flood in Johnstown in 1953, though there was a very well-known flood in 1889, which it refers to, and other floods in 1936 and 1977. Other aspects of this fabrication are drawn from historical accounts as well.

Thus far we’ve seen four of examples: 1) a fictional story, 2) an accurate biography of Roland Kirk, 3) a confabulated biography of Miriam Yevick, and 4) a confabulated account of a Johnstown flood. ChatGPT doesn’t (know how to) drawn a firm line between fiction and reality, which is a well-known and anguish-inducing property of LLMs. One could obtain similar results with other forms of content.

Word-for-word

There’s one other kind of prompt response I want to look at. If you give it “To be or not to be” or “Four score and seven years ago” it will respond with Hamlet’s complete soliloquy in the first case and with Lincoln’s Gettysburg Address in the second. While one might imagine that the LLM has “memorized” those texts word-for-word in the way one might memorize an arbitrary list of words (e.g. for a psychology experiment), I’m pretty sure that that’s not the case here, as I argue at some length in a recent working paper, Discursive Competence in ChatGPT, Part 2: Memory for Texts. In that kind of rote memorization word meanings are irrelevant, which is the case for arbitrary lists of words. But those speeches are not arbitrary lists of words. They are meaningfully structured and it is clear that ChatGPT is “aware” of that structure.

* * * * *

This leaves us with two types of higher-level structure, by which I mean nothing more than structure above the sentence level:

1.) There is the coherent text having content that can be expressed in various ways. These texts can be completely imaginary, they can be about real phenomena, or they can be confabulations which mix real and made-up material.

2.) Word-for-word memorization of specific texts which, however, does recognize the internal structure and meaning of the texts.

How the model organizes this high-level structure is not clear, though my work of stories (referenced above) suggests there is some kind of hierarchical organization. I would also imagine there is some way of dealing with logical connectives, such as AND and OR, though I don’t have any idea how that is done.

More later.