In a large representative sample of humans compared to GPT-4: "the creative ideas produced by AI chatbots are rated more creative [by humans ]than those created by humans... Augmenting humans with AI improves human creativity, albeit not as much as ideas created by ChatGPT alone” pic.twitter.com/rJDUZxJ4iL
— Ethan Mollick (@emollick) February 2, 2025
But then, what is creativity? Interesting, but color me skeptical.
Not too many years ago Geoffrey Hinton confidently predicted that radiologists would soon be replaced by AI. That didn't happen. But now...
The New York Times a small study (50 doctors, a mix of residents and attendings) in which ChatGPT-4 outperformed physicians in diagnosis based on a case report:
...doctors who were given ChatGPT-4 along with conventional resources did only slightly better than doctors who did not have access to the bot. And, to the researchers’ surprise, ChatGPT alone outperformed the doctors.
“I was shocked,” Dr. Rodman said.
The chatbot, from the company OpenAI, scored an average of 90 percent when diagnosing a medical condition from a case report and explaining its reasoning. Doctors randomly assigned to use the chatbot got an average score of 76 percent. Those randomly assigned not to use it had an average score of 74 percent.
The study showed more than just the chatbot’s superior performance.
It unveiled doctors’ sometimes unwavering belief in a diagnosis they made, even when a chatbot potentially suggests a better one.
Of course, reading an x-ray and analyzing a case report are very different activities. Still...
There's more at the link, including a brief look at INTERNIST-1, an old-school AI system developed in the 1970s for diagnosis. It's clear to me that AI his here to stay, in general, and certainly in medicine. What's not at all clear is just how it's going to be used. Obviously, that will change over time as AI capabilities develop. While thinking about that you might look at Hollis Robbins' post, AI and The Last Mile:
While we worry about AI replacing human judgment, the real story may be how AI is creating a market for that judgment as a luxury good, available only to those who can pay for the “last mile” of human insight. What do I mean by this?
The challenge of mail delivery from the post office to each home or from a communication hub to each individual end user is known as a “last mile” problem. In the paper newspaper era, the paper boy was the solution to the last mile problem, hawking papers on street corners or delivering papers house by house in the early morning before school. The postal carrier is a solution to the last mile problem. DoorDash is a solution to the last mile problem in the food business. [...]
What I’m calling “the last mile” here is the last 5-15% of exactitude or certainty in making a choice from data, for thinking beyond what an algorithm or quantifiable data set indicates, when you need something extra to assurance yourself you are making the right choice.
It does sometimes introduce me to new tools and paradigms but that's much more rare. Most of it is just allowing me to focus on the actual hard bits (the architecture and flow)
— Mark Tenenholtz (@marktenenholtz) January 15, 2024
An explanation for why GPT-4 is degrading:
— Chomba Bupe (@ChombaBupe) December 31, 2023
"... we find that on datasets released before the LLM
training data creation date, LLMs perform surprisingly better than on datasets released after"
New tasks are difting away from what GPT-4 was trained on. pic.twitter.com/k4i7nv0ULz
Both human writers and GPT easily followed the typical features of the prompt, demonstrating the pervasiveness of the Pygmalionesque trope in our cultural imaginary.
— Nina Beguš (@ninabegus) October 20, 2023
Some human-written stories alluded or referred to their fictional inspirations, mostly from popular films and TV. pic.twitter.com/5xvxznmzNn
Human-authored characters were all over the place, acting from intense loneliness, loss of significant other, or coping with the effects of a serendipitous invention.
— Nina Beguš (@ninabegus) October 20, 2023
What human-written stories lacked in comprehension, they made up for with their character and playfulness. pic.twitter.com/xnThD52Slc
There are more tweets in the stream, but check out the paper linked in the first tweet.
A paper that really illustrates both the unexpected power, and unexpected risks, that come from LLMs.
— Ethan Mollick (@emollick) October 20, 2023
Given text of anonymous posts on Reddit, GPT-4 can infer things like income, gender & location with 85%+ accuracy at 1% of the cost required by humans. https://t.co/qcrodZsgUQ pic.twitter.com/at8NfwLxjr
This is the condensed version of the "Fireside Chat: With Ilya Sutskever and Jensen Huang: AI Today and Vision of the Future (March 2023)". In this video, I've carefully selected the top 10 questions from the original hour-long talk and condensed them into just over 30 minutes. Additionally, I've created a timeline of these questions asked by Jensen to Ilya, accompanied by relevant research papers. As machine transcription is often inaccurate, I manually transcribed the entire video (laborious), recognizing the importance of accurate captions for those with hearing difficulties. Anyways, you may find this information useful.
If you're interested in more AI-related content, consider subscribing to my channel. Stay tuned for future uploads!
Time-codes:
00:00:00 Q1. Intuition behind deep learning?
00:02:20 Q2. Initial motivations behind OpenAI?
00:07:54 Q3. Intuition about scaling laws (and RLHF)?
00:11:07 Q4. Aspects of ChatGPT and their abilities?
00:14:57 Q5. Major differences between GPT-4 and previous versions?
00:18:37 Q6. Reasoning capability of GPT-4 and limiting factors?
00:23:47 Q7. Importance of multi-modality?
00:28:53 Q8. How multi-modality helped improve GPT-4 over GPT-3?
00:30:46 Q9. Predictions for the next 2 years?
00:32:57 Q10. Surprising results?
Relevant research papers:
Learning to Generate Reviews and Discovering Sentiment https://arxiv.org/abs/1704.01444
Scaling Laws for Autoregressive Generative Modeling https://arxiv.org/abs/2010.14701
Training language models to follow instructions with human feedback https://arxiv.org/abs/2203.02155
I’ve got another article at 3 Quarks Daily:
World, Mind, Learnability, Large Language Models, and the Metaphysical Structure of the Cosmos
I take another crack at something I’ve looked at before, the idea that LLMs embody a metaphysics of the cosmos, whatever those are. We’re not dealing with the physical structure of the cosmos, we’re not dealing with the totality of things-as-such, but rather with the (conceptual) structures through which we apprehend them. Given the wide range of texts that have been digested and distilled into these models, their internal structure somehow is responsive to, reflects and refracts, those structures.
Let me emphasize: Foundational LLMs implicate the metaphysical structure of the cosmos.
I set up the argument by starting with the 1977 short film, Powers of Ten, which traverses the cosmos from very large to very small in the compass of nine minutes. Then I ride one of my favorite hobby horses, Disney’s Fantasia, which also encompasses the cosmos. After that I ask: What must the world be like so that we can learn its structure?
And so forth and so on.
* * * * *
In the course of working on this piece I decided to ask GPT-4 about the film, Powers of Ten. Of course it knew about the film. I then asked to summarize what that film shows. It did so and with great confidence. And it got it wrong. First, I present the dialog I had with GPT-4, and then I present a summary of the film from Wikipedia. That summary, cursory though it is, makes it clear that GPT-4’s summary is incorrect. The Chatster was up to its old tricks, just making stuff up. I didn’t make any effort to nail down all the errors in GPT-4’s summary.
* * * * *
[April 22, 2023 – 3:15PM – Mar 23 version]
Do you know about the short 1977 film “Powers of Ten”?
Yes, I am familiar with the short film "Powers of Ten." It was directed by Ray and Charles Eames and was released in 1977. The film explores the relative sizes of things in the universe, from the scale of the observable universe all the way down to the subatomic level.
The film starts with an aerial view of a man and woman having a picnic in a park in Chicago, and then zooms out to show the Earth from a distance of ten meters, then ten times farther, and so on, until it reaches the limit of the observable universe. It then zooms back in, but this time at a smaller and smaller scale, revealing the structure of atoms and subatomic particles.
The film is a classic of science education and has been widely praised for its innovative use of scale and its ability to convey complex ideas in a clear and accessible way. It has been used in classrooms and science museums around the world, and is often cited as an example of effective science communication.
Could you talk your way through the first part of the film, starting with the couple and zooming out? What do we see at each higher power of 10?
Sure, I'd be happy to describe the first part of the film "Powers of Ten" to you.
The film begins with a shot of a couple, a man and a woman, having a picnic on a sunny day in a park in Chicago. The camera then begins to zoom out from the couple, at a rate of 10 times the distance every 10 seconds. As the camera zooms out, we see the park getting smaller and smaller, until it disappears from view.
At the first power of ten (10^1), we see the view of the park from an altitude of 10 meters above the couple. We can see the couple clearly, as well as the surrounding park and nearby buildings.
At the second power of ten (10^2), the camera has zoomed out to an altitude of 100 meters. Now we can see not just the park, but also the surrounding neighborhoods and the city of Chicago in the distance.
At the third power of ten (10^3), the camera has zoomed out to an altitude of 1,000 meters, or one kilometer. Now we can see not just the city of Chicago, but also the entire Lake Michigan shoreline and beyond.
At the fourth power of ten (10^4), the camera has zoomed out to an altitude of 10,000 meters, or 10 kilometers. Now we can see not just the city of Chicago and Lake Michigan, but also the entire state of Illinois and neighboring states.
At the fifth power of ten (10^5), the camera has zoomed out to an altitude of 100,000 meters, or 100 kilometers. Now we can see not just the Midwest region of the United States, but also the entire continent of North America.
At the sixth power of ten (10^6), the camera has zoomed out to an altitude of 1,000,000 meters, or 1,000 kilometers. Now we can see not just North America, but also the entire planet Earth.
At the seventh power of ten (10^7), the camera has zoomed out to an altitude of 10,000,000 meters, or 10,000 kilometers. Now we can see not just the Earth, but also the entire solar system, with the planets orbiting the sun.
At the eighth power of ten (10^8), the camera has zoomed out to an altitude of 100,000,000 meters, or 100,000 kilometers. Now we can see not just the solar system, but also the entire Milky Way galaxy, with its hundreds of billions of stars and countless other celestial objects.
At the ninth power of ten (10^9), the camera has zoomed out to an altitude of 1,000,000,000 meters, or 1 billion kilometers. Now we can see not just the Milky Way galaxy, but also the entire Local Group of galaxies, a cluster of more than 50 galaxies that includes the Milky Way.
At the tenth power of ten (10^10), the camera has zoomed out to an altitude of 10,000,000,000 meters, or 10 billion kilometers. Now we can see not just the Local Group of galaxies, but also the entire observable universe, with its hundreds of billions of galaxies and vast expanses of empty space.
That's a brief overview of the first part of "Powers of Ten." The film is an excellent way to explore the vastness of the universe and the different scales at which we can observe it.
On April 17 I queried GPT-4 (March 23 version) about doomsday cults. It’s not that I have a particular interest in doomsday cults, but that I think that current concerns about existential risks posed by artificial intelligence seem to participate in – what do we call it? a psycho-social-cultural formation? – an ethos that derives as much from cult behavior as from a coherent and rational evaluation of A.I. technology.
I start out with a general question about doomsday cults. Then I mention two contemporary examples, The People’s Temple (Jim Jones) and the Branch Davidians. I then ask it to name some more examples, which it does, and then: “How far back in history do we have records of doomsday cults?” Finally I ask it about belief AI x-risk. Here’s the second paragraph from its three paragraph answer:
However, the belief that AI will inevitably destroy all humans and that there is no way to prevent it can be seen as an extreme and apocalyptic view. If a group of people were to organize around this belief and engage in behaviors such as isolating themselves from society or advocating for violent actions to prevent AI from becoming too powerful, then they could potentially be classified as a doomsday cult.
That second sentence is worth serious thought. That hasn’t happened yet. But we’ve only been living with general access to chatbots for four-and-a-half months. What’s going to happen in the future?
I don’t know.
* * * * *
What can you tell me about doomsday cults?
Doomsday cults are groups of people who believe in an impending apocalypse or catastrophic event that will result in the end of the world as we know it. These groups often develop unique and extreme beliefs about the nature of the apocalypse, including specific dates, scenarios, or supernatural explanations for the end of the world.
Doomsday cults may also develop highly structured and controlling social hierarchies, with a charismatic leader or group of leaders who hold significant power over the members. This can lead to the exploitation of followers and the use of extreme tactics to maintain the group's cohesion and loyalty.
In some cases, doomsday cults have been known to engage in violent or criminal behavior, including mass suicide or murder-suicide. Examples of infamous doomsday cults include the People's Temple led by Jim Jones and the Branch Davidians led by David Koresh.
It's important to note that not all groups with apocalyptic beliefs are dangerous or harmful, and many people may hold apocalyptic beliefs without being part of a formal cult or group. However, when a group becomes highly controlling, manipulative, or violent, it can be a cause for concern.
Can you tell me more about the People's Temple and the Branch Davidians.
Certainly. The People's Temple was a religious group led by Jim Jones that was founded in the 1950s in Indianapolis, Indiana. The group started as a progressive and inclusive Christian church, but over time, Jones became increasingly paranoid and controlling. In 1977, Jones moved the group to Guyana in South America, where they established the Jonestown settlement.
On November 18, 1978, a delegation from the United States arrived at Jonestown to investigate reports of human rights abuses in the community. Jones ordered his followers to kill the delegation and commit mass suicide by drinking cyanide-laced punch. In total, 918 people died, including over 300 children. The Jonestown massacre remains one of the largest mass suicides in history and is widely considered to be a case of a destructive cult.
The Branch Davidians were a religious group that originated as a splinter group from the Seventh-day Adventist Church in the 1950s. The group was led by David Koresh, who proclaimed himself to be a messiah and claimed to receive divine revelations. In 1993, the group came into conflict with law enforcement officials when the Bureau of Alcohol, Tobacco, and Firearms (ATF) attempted to raid their compound in Waco, Texas, on suspicion of illegal firearms possession.
After a prolonged standoff, the compound was raided by the FBI, and a fire broke out that killed Koresh and 76 of his followers, including many women and children. The incident was widely criticized for the excessive use of force by law enforcement and sparked debates about the rights of religious groups and the role of government in regulating them. The Branch Davidians are also considered to be a destructive cult due to the extreme beliefs and controlling behavior of their leader.
Can you name some more examples?
Sure. Here are some additional examples of doomsday cults:
It's important to note that while these groups may share some common characteristics, each one is unique in its beliefs, practices, and history. Additionally, not all groups with apocalyptic beliefs are dangerous or harmful, and many people may hold apocalyptic beliefs without being part of a formal cult or group.
How far back in history do we have records of doomsday cults?
The idea of an impending apocalypse or end of the world has been present in many cultures and religions throughout history, so it is difficult to pinpoint the origins of doomsday cults. However, some examples of historical groups that have been described as doomsday cults include:
While these groups may not fit the modern definition of a doomsday cult, they share some similarities in their beliefs about an impending cataclysmic event and the need for preparation or repentance.
These days there are people who believe that advanced artificial intelligence will destroy all humans. Some of these people believe that this could happen in less than 50 years. Could these people be considered to be doomsday cultists?
It is important to note that not all people who believe in the potential risks of advanced artificial intelligence can be considered doomsday cultists. Concerns about the potential negative impact of AI on society and humanity are legitimate and have been discussed by many experts in the field.
However, the belief that AI will inevitably destroy all humans and that there is no way to prevent it can be seen as an extreme and apocalyptic view. If a group of people were to organize around this belief and engage in behaviors such as isolating themselves from society or advocating for violent actions to prevent AI from becoming too powerful, then they could potentially be classified as a doomsday cult.
It is worth noting that many experts in the field of AI do not share the belief that AI will necessarily be harmful to humanity. Some argue that the development of advanced AI could be a positive force for progress and that the risks can be mitigated through careful research and development, as well as through ethical frameworks and regulations.
On April 16 I queried GPT-4 (March 23 version) on Descartes and related matters. I began by asking it the meaning of “cogito, ergo sum.” Then I asked it about the role that God played in Descartes’ reasoning. Why? Because, at GPT-4 pointed out, Descartes “uses the concept of God as a guarantor of truth to establish the certainty of knowledge beyond the ‘Cogito’ argument.”
Beyond that, I was setting the stage for having GPT-4 consider the relevance of Descartes’ reasoning to its own situation as an LLM trained on text without access to the external world. In dealing with the question GPT-4 acknowledged that there is are philosophical issues regarding whether or not LLMs actually understand anything “or if it is simply a matter of statistical patterns in data.” I then broached the issue of so-called “hallucinations.”
After that I asked how an LLM would deal with a human trying to fool it. GPT-4 admitted that that is a problem given that LLMs are trained to take their input text at face value. However, researchers are working on ways to deal with the problem. I concluded by asking it about the ethnical responsibilities of humans using AI.
Here is the full interaction, without further commentary.
* * * * *
What did Descartes mean by “cogito, ergo sum”?
"Cogito, ergo sum" is a Latin phrase coined by the French philosopher and mathematician René Descartes, which translates to "I think, therefore I am."
The statement is the foundation of Descartes' philosophical system, and it is often referred to as the "Cartesian dualism," which holds that the mind and body are two separate entities that interact with each other.
In his "Meditations on First Philosophy," Descartes argues that one can doubt everything except their own existence, because the very act of doubting requires a thinking subject. Thus, the fact that he is thinking means that he must exist.
By using the cogito, Descartes aims to establish a secure foundation for knowledge that is independent of the senses, which he believed to be unreliable. He believed that by starting with this indubitable truth, he could then proceed to establish other truths about the world.
How did he develop that argument? Give me some details.
Descartes developed the argument for "Cogito, ergo sum" through his philosophical inquiry into the nature of knowledge and certainty. He sought to establish a secure foundation for knowledge that could withstand skepticism and doubt.
In his "Meditations on First Philosophy," Descartes begins by doubting everything he can possibly doubt, including his own senses, perceptions, and beliefs. He recognizes that many of his beliefs could be false, and that he could be deceived by his senses or by an evil demon.
However, Descartes realizes that even if he is being deceived or mistaken about everything else, he cannot doubt the fact that he is doubting. In other words, the very act of doubting requires a thinking subject, and therefore he must exist as a thinking being.
Descartes expresses this argument in the famous phrase "Cogito, ergo sum" or "I think, therefore I am." He argues that the only thing he can be certain of is his own existence as a thinking thing, and that this provides a firm foundation for all other knowledge.
From this starting point, Descartes goes on to develop a rationalist theory of knowledge that relies on innate ideas and deductive reasoning to establish certain truths about the world. However, the "Cogito" argument remains a central pillar of his philosophical system and continues to be a topic of discussion and debate in modern philosophy.
What role did God play in the argument?
In Descartes' "Cogito, ergo sum" argument, God plays a crucial role in establishing the certainty of knowledge beyond the self-evident truth of the "I think" statement.
Descartes believed that God was a necessary condition for the possibility of knowledge. He argued that God, as an all-powerful and all-knowing being, would not allow us to be deceived in our perceptions or beliefs about the world. Therefore, if we have clear and distinct ideas that are beyond doubt, we can be sure that they are true because God guarantees their truthfulness.
In his "Meditations," Descartes uses the concept of God as a guarantor of truth to establish the certainty of knowledge beyond the "Cogito" argument. He argues that because God is perfect, he cannot deceive us, and thus the clear and distinct ideas we have of things like mathematics or the laws of nature must be true. This argument is known as the "Trademark Argument" or the "Cartesian Circle."
Therefore, while the "Cogito, ergo sum" argument itself does not rely on God, Descartes' broader philosophical system does, as he believed that knowledge of the external world was only possible if God guaranteed its truthfulness.
Do you think that Descartes’ reasoning has any relevance to the situation of a large language model, which is, after all, trained only on text and has not direct access to the external world?
Descartes' reasoning, particularly the "Cogito, ergo sum" argument, may not have direct relevance to the situation of a large language model like myself, as I am a machine and not a conscious being with a mind. However, the broader philosophical questions that Descartes addressed are still relevant to the development and use of artificial intelligence.
One of the central concerns in the philosophy of AI is how to develop machines that can truly understand and reason about the world, rather than simply processing information. Descartes' argument for the importance of self-knowledge and the role of God in guaranteeing the truthfulness of our beliefs can be seen as a precursor to this question.
In the case of a language model like myself, there is a question of whether my understanding of language is truly "meaningful" or if it is simply a matter of statistical patterns in data. This raises the broader question of what it means for a machine to truly understand language or to have knowledge of the world.
Furthermore, the issue of reliability and the possibility of deception that Descartes addressed is also relevant to AI. As language models become more advanced and are used in a wider range of applications, it is important to consider the potential for biases, errors, or malicious use that could lead to incorrect or harmful outcomes.
Therefore, while the specifics of Descartes' argument may not be directly applicable to a language model, his broader philosophical concerns and questions are still relevant to the development and use of artificial intelligence.
This is a sonnet I know well because I published two papers on it early in my career: Cognitive Networks and Literary Semantics (1976) and Lust in Action: An Abstraction (1976). Both are the cognitive networks models I developed while working with David Hays at SUNY Buffalo. As such they are not interpretations or reading of the poem. Rather, they are attempts to depict some of the conceptual structure underlying the poem. Preliminary and provisional as that work was, it changed forever my approaching to thinking about literature and the mind.
Do those models capture anything about what went on as GPT-4 read the poem and then responded to my prompts about it? I do not know. It would be interesting to find out.
This interaction took place on April 6, 2023 at 7:32 AM using the Mar 23 version of GPT-4. GPT-4’s response to the poem is quite sophisticated. You might be wondering, “Why did you introduce the idea of the Fortunate Fall?” Simple, I included it in my 1976 model of the poem’s semantic structure. I have previously asked ChatGPT about the Fortunate Fall in connection with Milton’s Paradise Lost.
GPT-4 exhibits a sophisticated capacity for analogical and interpretive reasoning once I start quizzing it about the hunting simile in lines 7 and 8. I am particularly impressed with how it was able to integrate the idea of the Fortunate Fall into its reasoning. Having done so already in the 1976 paper, it knew it was possible, but still, it was impressive to see GPT-4 do it. I was particularly impressed with its remarks about spiritual growth in the penultimate paragraph.
I know that the folks who are worried about AI existential risk are hyperventilating about disagreeable, angry and threatening remarks made by various LLMs. If they are going to insist that such remarks are evidence of the dangerous nature of LLMs, then I am going to insist that they give credence to GPT-4’s moral reasoning in its remarks about Sonnet 129. Perhaps the behavioral proclivities of LLMs are a reflection of and response to the attitudes of their human interlocutors.
* * * * *
Here is a well-known Shakespeare sonnet. What can you tell me about it?
Th’expense of spirit in a waste of shame
Is lust in action, and till action, lust
Is perjured, murd’rous, bloody, full of blame,
Savage, extreme, rude, cruel, not to trust;
Enjoyed no sooner but despisèd straight,
Past reason hunted, and no sooner had,
Past reason hated as a swallowed bait
On purpose laid to make the taker mad:
Mad in pursuit and in possession so,
Had, having, and in quest to have, extreme;
A bliss in proof, and proved, a very woe,
Before, a joy proposed, behind, a dream.
All this the world well knows yet none knows well
To shun the heaven that leads men to this hell.
The sonnet you have presented is Sonnet 129, one of William Shakespeare's 154 sonnets. This sonnet is classified as a "dark" or "problem" sonnet due to its subject matter, which explores the destructive nature of lust and sexual desire.
The sonnet begins with the line "Th’expense of spirit in a waste of shame," which suggests that indulging in lustful desires is not only wasteful but also damaging to one's spiritual well-being. The poem goes on to describe the negative consequences of lust, including deceit, violence, and cruelty.
The lines "Enjoyed no sooner but despisèd straight, / Past reason hunted, and no sooner had, / Past reason hated as a swallowed bait / On purpose laid to make the taker mad" depict the fleeting and unsatisfying nature of lustful pleasures, which can leave the individual feeling empty and disappointed.
The final couplet, "All this the world well knows yet none knows well / To shun the heaven that leads men to this hell," implies that while people are aware of the destructive nature of lust, they continue to pursue it anyway. The speaker urges the reader to resist the temptation of lustful desires and avoid the "heaven" that ultimately leads to "hell."
Overall, Sonnet 129 is a powerful commentary on the dangers of unchecked desire and the importance of exercising self-control.
What do you make of that hunting simile in lines 7 and 8? Who laid the bait? Who is the hunter? What kind of hunting does Shakespeare seem to have in mind?
The hunting simile in lines 7 and 8 of Sonnet 129 compares the pursuit of lust to the act of hunting. The line "Past reason hunted" suggests that the pursuit of lust is irrational and driven by primal instincts, rather than reason.
The phrase "swallowed bait / On purpose laid to make the taker mad" implies that someone or something has intentionally laid the trap of lust to ensnare the individual and lead them down a destructive path. This could be interpreted as a critique of societal norms or expectations that encourage indulgence in lustful desires, or perhaps the speaker is suggesting that lust is a trap laid by one's own desires and impulses.
As for who the hunter is, it could be interpreted as either the individual pursuing the lustful desires or the external forces that encourage such behavior. Shakespeare seems to be suggesting that the pursuit of lust is a self-destructive act that can lead to madness and despair.
The hunting metaphor used in these lines could also be seen as a commentary on the nature of the pursuit itself. Hunting, particularly in Shakespeare's time, was often associated with aristocratic pursuits and displays of power. The metaphor could therefore be seen as a critique of the excessive and reckless behavior of the elite, who indulged in lustful desires without thought for the consequences.
“Hunting, particularly in Shakespeare's time, was often associated with aristocratic pursuits and displays of power. The metaphor could therefore be seen as a critique of the excessive and reckless behavior of the elite, who indulged in lustful desires without thought for the consequences.” Interesting. I like that.
Yes, Shakespeare often used his works to comment on the social and political issues of his time. The metaphor of hunting could be seen as a critique of the behavior of the aristocracy, who were often seen as indulging in excesses without thought for the consequences. By comparing the pursuit of lust to the act of hunting, Shakespeare may be suggesting that the pursuit of carnal desires is similarly excessive and reckless, and that it ultimately leads to negative consequences. The poem can therefore be seen as a warning against the dangers of unchecked desire, regardless of one's social status or position.
If we think of sexual satisfaction as the snare or bait laid to draw humans into this tangle of impossible desires, then who or what might have laid that snare?
The question of who or what might have laid the snare of sexual desire is a complex one, and different interpretations are possible.
One possibility is that the snare is simply the natural human desire for sexual pleasure, which can be so powerful that it leads individuals to engage in destructive behaviors. In this interpretation, the snare is not necessarily a deliberate trap laid by anyone or anything, but rather a natural aspect of human desire.
Another interpretation is that societal norms or expectations might have laid the snare. For example, the poem could be seen as a critique of the restrictive and oppressive sexual mores of Shakespeare's time, which may have led individuals to suppress or deny their natural desires. In this interpretation, the snare is laid by the societal expectations that individuals must conform to in order to be considered moral or virtuous.
Alternatively, the snare could be seen as the result of external pressures or influences, such as advertising, media, or cultural messages that promote sexualized images and ideals. In this interpretation, the snare is laid by the cultural forces that shape our desires and attitudes towards sex.
Ultimately, the interpretation of who or what might have laid the snare of sexual desire is up to the reader, and different readers may have different perspectives on this question.
Are you aware of the medieval idea of the Fortunate Fall (felix culpa)?
Yes, I am aware of the medieval Christian concept of the Fortunate Fall, also known as the felix culpa. The idea comes from St. Augustine and refers to the belief that Adam and Eve's disobedience in the Garden of Eden, which led to the fall of humanity, was actually a fortunate event because it allowed for the redemption of mankind through the sacrifice of Jesus Christ.
The idea is based on the belief that God's plan for humanity included the eventual redemption of mankind through the incarnation, death, and resurrection of Jesus Christ. According to this view, the fall of Adam and Eve was a necessary part of God's plan, as it allowed for the eventual salvation of humanity through Jesus.
The concept of the Fortunate Fall became an important theme in medieval Christian theology and literature, and was often depicted in art and literature as a way of celebrating the grace of God in overcoming human sin and weakness.
If we interpret the final couplet of sonnet 129 in light of that doctrine, how would that work out?
If we interpret the final couplet of Sonnet 129 in light of the Christian concept of the Fortunate Fall, we might read the couplet as a warning against the dangers of giving in to sinful desires, but also as a recognition of the redemptive potential of such actions.
In this interpretation, the "heaven" that "leads men to this hell" might be seen as a reference to the temptation of sin, which can lead individuals into a state of moral and spiritual degradation. The "hell" itself might be seen as a metaphor for the negative consequences of such actions, which can include shame, guilt, and the loss of self-respect.
However, the final line, "To shun the heaven that leads men to this hell," could also be read as a recognition that sin can be a necessary part of the human experience, and that it can ultimately lead to redemption and salvation. This interpretation suggests that even though sin may lead individuals into a state of moral degradation, it can also provide an opportunity for spiritual growth and transformation.
Thus, the final couplet of Sonnet 129 can be seen as a complex meditation on the nature of sin and redemption, and a recognition of the paradoxical relationship between the two.
With no programming background, @ammar created a (complex) video game from scratch.https://t.co/GppWNyvmFw
— Sam Woods (@samuelwoods_) April 3, 2023
That's just the beginning of the tweet thread. Check out the whole thing on Twitter.
I’ve got a new article at 3 Quarks Daily:
From “Kubla Khan” through GPT and beyond
https://3quarksdaily.com/3quarksdaily/2023/03/from-kubla-khan-through-gpt-and-beyond.html
I’ll say a bit more about it later.
At the moment I want to report an interaction I had with GPT-4 about Coleridge, “Kubla Khan,” Xanadu, and pop culture. I’ve added some subtitles to break things up.
What do you know about Samuel Tayler Coleridge, the great British poet and essayist?
Samuel Taylor Coleridge was a British poet, critic, and philosopher who lived from 1772 to 1834. He is best known for his poetry, particularly his lyrical ballads, which he co-authored with William Wordsworth. Coleridge was also a prolific essayist, lecturer, and literary critic, and is considered one of the most important figures in the English Romantic movement.
Coleridge's most famous works include the poems "The Rime of the Ancient Mariner," "Kubla Khan," and "Christabel." These poems are known for their imaginative and supernatural elements, and for their exploration of complex philosophical and psychological themes. Coleridge's poetry often reflects his interest in nature, religion, and the supernatural, as well as his struggle with addiction and depression.
In addition to his poetry, Coleridge is also known for his literary criticism and his philosophy of language. He was interested in the relationship between language, thought, and reality, and argued that language is a fundamental aspect of human consciousness and identity. Coleridge's ideas about language and poetry had a profound influence on later writers, including T.S. Eliot and the New Critics.
Overall, Samuel Taylor Coleridge is considered one of the most important and influential writers of the Romantic period, and his works continue to be studied and celebrated today.
What can you tell me about “Kubla Khan” and the scholarship about the poem?
"Kubla Khan" is a poem by Samuel Taylor Coleridge that was first published in 1816. The poem is widely regarded as one of Coleridge's most famous and enigmatic works, and it has been the subject of much scholarly debate and interpretation.
The poem tells the story of an opulent palace that is built by the Mongol emperor Kublai Khan, and the exotic and mysterious landscape that surrounds it. The poem is known for its vivid and imaginative descriptions, as well as its use of dream-like imagery and symbolism.
One of the most intriguing aspects of "Kubla Khan" is the story behind its creation. According to Coleridge, he wrote the poem after waking from a vivid dream, and he claimed that he had composed the entire poem in his mind before being interrupted by a visitor. He was unable to recall the rest of the poem after the interruption, and the published version of the poem is only a fragment of the original.
This story has led to much speculation and interpretation among scholars, who have debated the significance of the dream and the role it played in the creation of the poem. Some have argued that the dream represents Coleridge's subconscious, and that the poem is a reflection of his inner thoughts and feelings. Others have suggested that the poem is an allegory for the creative process itself, or a commentary on the relationship between art and reality.
Despite the many different interpretations of "Kubla Khan," the poem remains one of Coleridge's most celebrated works, and it continues to inspire readers and scholars alike.
What do scholars think about whether or not the poem is complete? I know that Humphrey House has remarked that, if it wasn’t for the preface, no one would think the poem is incomplete. There doesn’t seem to be anything missing. And then we have Kenneth Burke, who argued that Coleridge himself was puzzled by the poem since it didn’t conform to his theories of poetry, and so he thought it complete.
The question of whether "Kubla Khan" is a complete or incomplete poem is a subject of much scholarly debate. As you noted, some scholars have argued that the poem is complete as it stands, and that the fragmentary nature of the work is actually one of its strengths.
Others, however, have suggested that the poem is incomplete and that there is more to the work than what is presented in the published version. For example, some have speculated that the missing portion of the poem may have contained additional details about the palace and its surroundings, or that it may have further developed the themes and ideas presented in the existing text.
At the same time, it is also possible that Coleridge himself considered the poem to be complete, despite the fact that it was not fully realized according to his original intentions. As you mentioned, Kenneth Burke has suggested that Coleridge may have been puzzled by the poem's deviation from his theories of poetry, but that he ultimately decided to publish it as it was.
Overall, the question of whether "Kubla Khan" is complete or incomplete is a matter of interpretation, and scholars continue to debate the issue to this day. What is clear, however, is that the poem remains one of the most intriguing and evocative works in the English language, and it continues to captivate and inspire readers and scholars alike.
Burke thought the poem was a kind of poetized psychology. What might he have meant by that?
When Kenneth Burke described "Kubla Khan" as a "poetized psychology," he was suggesting that the poem was a representation of Coleridge's inner thoughts and feelings, rather than an objective depiction of external reality. In other words, Burke believed that the poem was a product of Coleridge's imagination, and that it reflected his own psychological state at the time of its composition.
Burke's interpretation of "Kubla Khan" is closely related to his broader ideas about the relationship between language and psychology. He believed that language was a powerful tool for shaping and expressing our inner psychological states, and that poets in particular had a unique ability to use language to convey their own subjective experiences and perceptions.
In the case of "Kubla Khan," Burke argued that the poem's dream-like imagery and fragmented structure were evidence of Coleridge's own psychological turmoil. He suggested that the poem was a product of Coleridge's subconscious mind, and that it represented his attempt to grapple with his own inner conflicts and desires.
Overall, Burke's interpretation of "Kubla Khan" as a "poetized psychology" reflects his broader ideas about the role of language in shaping our understanding of the world, and it underscores the complex and multifaceted nature of Coleridge's enigmatic and evocative poem.
What do we know about Xanadu? I believe that it was the Kubla Khan’s summer palace and the the Chinese have identified the site of the real Xanadu, which is called Shangdu and is in Mongolia.
Yes, that's correct! Xanadu was the name of Kubla Khan's summer palace, and it is widely believed to have been located in what is now Inner Mongolia, China. The palace was said to be a sprawling complex of gardens, lakes, and pavilions, and it was renowned throughout the world for its beauty and grandeur.
In the poem "Kubla Khan," Coleridge describes Xanadu as a "stately pleasure-dome" that was built "amid the forests and the hills." He goes on to describe the palace's gardens, which were irrigated by a network of canals and waterways, and its walls, which were adorned with intricate carvings and paintings.
In the centuries since Kubla Khan's time, the site of Xanadu has been lost to history. However, in recent years, archaeologists have identified a site in Inner Mongolia called Shangdu that they believe may have been the location of the palace. Shangdu was built by Kubla Khan's grandson, the emperor Shizu, in the late 13th century, and it was described in detail by the Italian explorer Marco Polo.
Today, the ruins of Shangdu remain a popular tourist attraction, and they offer a glimpse into the grandeur and beauty of Kubla Khan's legendary summer palace.
GPT’s are constructed so as to continue a given text. I’d asked GPT-4 about that and reported its response in a previous post: So, just why do GPTs have to operate by continuing an existing string? I went on to quiz it about story generation. I asked it about that.
How is a GPT modified so that it can tell stories?
To modify a GPT model so that it can generate stories, there are a few approaches that can be taken. One approach is to fine-tune the model on a dataset of stories, such as books or other narrative texts. This involves modifying the architecture of the GPT model to generate coherent and engaging stories.
Here are some steps that could be taken to fine-tune a GPT model for story generation:
- Preprocess the story dataset: The dataset of stories needs to be preprocessed so that it is in a format that can be used to train the GPT model. This involves splitting the stories into paragraphs or sentences and encoding them into a format that can be fed into the model.
- Modify the architecture of the model: The GPT model needs to be modified to generate coherent and engaging stories. This can be achieved by adding new layers or modifying existing layers to better capture the structure and narrative elements of a story. For example, the model could be modified to learn to generate character descriptions, plot elements, and dialogue.
- Train the model: Once the model architecture has been modified, it can be trained on the preprocessed dataset of stories. During training, the model learns to generate coherent and engaging stories by predicting the next word or sentence based on the context provided.
- Generate stories: Once the model has been trained, it can be used to generate new stories. To do this, the model is provided with a starting prompt or sentence, and it generates the next words or sentences based on the context provided. The generated text can be further conditioned by providing feedback to the model or by adjusting various parameters, such as the temperature or the length of the generated text.
In summary, modifying a GPT model to generate stories involves fine-tuning the architecture of the model on a dataset of stories and training it to generate coherent and engaging narrative text.
Hmmmm. Perhaps I should have pressed it on the first one since most stories already exist in paragraphs and sentences. I definitely should have pressed it on 2. 3 I already know. It’s four that I’m curious about. I frequently prompt ChatGPT with “Tell me a story” or “Tell me a story about a hero.” There is no obvious continuation for either of those. A wide range of things could follow either one, with no obvious way of chosing among them.
I tried to get at that in the next question:
I have noticed that if you prompt ChatGPT with “Tell me a story” it tends to tell a fairy-tale, with princesses and dragons and peasants. It even seems to like stories about Jack and a beanstalk. Also, it begins the stories with a stock phrase: “Once upon a time, long ago and far away...” Why is this?
When prompted with the request "Tell me a story," ChatGPT often generates a fairy tale-style story featuring princesses, dragons, and peasants. The stories usually begin with the traditional opening phrase "Once upon a time, long ago and far away..." This is likely due to the type of data that the GPT model was trained on.
The pre-training data used to train the GPT model likely included a large corpus of text that contained a significant number of fairy tales, folk tales, and other forms of traditional storytelling. These stories often follow a predictable narrative structure, with common elements such as magical creatures, heroic protagonists, and enchanted settings.
As a result, when prompted to generate a story, the GPT model is drawing on this pre-existing knowledge and generating stories that follow a similar structure and contain similar elements. The opening phrase "Once upon a time, long ago and far away..." is a common trope in traditional storytelling that signals to the listener that a story is about to begin and sets the stage for the narrative that follows.
It's also possible that the GPT model has learned through its training data that these types of stories tend to be well-received and are more likely to be enjoyed by humans. As a result, the model may be biased towards generating stories that fit this mold.
It's worth noting that GPT models can generate a wide range of story types and genres, depending on the prompt and the training data. If prompted with a different type of story or genre, such as a mystery or a romance, the model may generate a story that is quite different from a traditional fairy tale.
On the whole, not all that helpful. David Chapman had a short tweet stream in which he speculated about the presence of fan fiction on the web. One repository had almost 11 million stories. That might explain a lot. In any event, here’s my next and final query on this topic:
Yes, I have noticed that as well. If you prompt it, “Tell a sad story” it will do so. If you include specific items in the prompt – such as setting the story in an office building in a large city – it will do that. Do you have any observations about this?
Yes, that's correct. GPT models can generate a wide range of story types and genres, depending on the prompt and the training data. If prompted with a specific setting or genre, such as a sad story or a story set in an office building in a large city, the model can generate a story that is tailored to that prompt.
This is because GPT models use a process called "prompt conditioning," where the model generates text based on the input prompt or context provided. The model then generates the next word or sentence based on the probability distribution of the next likely word given the context.
When the prompt includes specific items or settings, the model uses these as context to generate a story that is consistent with the prompt. For example, if the prompt includes the words "sad story," the model may generate a story that includes themes of loss, tragedy, or heartbreak. If the prompt includes the words "office building in a large city," the model may generate a story that includes characters working in an office setting, with elements of urban life and city living.
It's important to note that the quality of the generated text depends on the quality of the prompt and the training data used to train the model. If the prompt is well-formed and includes specific details, the model is more likely to generate coherent and engaging text that is consistent with the prompt. Conversely, if the prompt is vague or ambiguous, the model may generate text that is less coherent or less relevant to the prompt.
I have already noted that things can get very interesting if you work interactively on a story.
Why can’t they, you know, just come out with it, like we do?
Yeah, I know, that’s how they’re trained; they’re trained to predict the next token. I’m not sure what that tells me.
So, when I give ChatGPT the prompt, “Tell me a story,” that isn’t what is actually presented to the underlying LLM. There isn’t anything that ‘naturally’ follows such a string. It could be literally almost anything. So, a bit of manipulation has to take place in order to give the LLM something to work with. But why?
I decided to put the question to GPT-4:
GPTs (that is, transformers) are constructed to continue an existing string of tokens. Why is that? Why can’t they simply answer a question without continuing an existing string?
GPTs, or Generative Pre-trained Transformers, are language models that are pre-trained on large amounts of text data using unsupervised learning. These models are designed to generate coherent and contextually relevant text based on the input provided to them.
The reason GPTs are constructed to continue an existing string of tokens is because they are designed to capture the context and meaning of the text, rather than just providing a one-word answer to a question. By generating a continuation of the input text, the model can incorporate the surrounding context and generate a more complete and accurate response.
In other words, GPTs are not just simple question-answering systems, but rather complex language models that can generate text that is contextually relevant and meaningful. By training on a large corpus of text, GPTs can learn to identify patterns and relationships in language, allowing them to generate text that is more than just a simple answer to a question.
That being said, GPTs can be fine-tuned to answer specific questions or perform specific tasks, such as question-answering or text classification, by providing them with appropriate training data and modifying their architecture accordingly.
That phase hit home: “...they are designed to capture the context and meaning of the text, rather than just providing a one-word answer to a question.” It’s that last clause that got my attention.
So I asked another question:
Tell me about that last. How is their architecture modified? What kind of training data is necessary?
To fine-tune a GPT model for a specific task, such as question-answering, the architecture of the model needs to be modified to accommodate the specific requirements of the task. This typically involves adding new layers or modifying existing layers of the model to better capture the specific features of the task.
For example, to fine-tune a GPT model for question-answering, the input data to the model would consist of a question and a passage of text that may contain the answer to the question. The model would then be trained to predict the correct answer based on the input question and text.
To accomplish this, the GPT model would be modified by adding a new output layer that is specific to the question-answering task. This layer would take the final hidden state of the model and generate a probability distribution over possible answers. During training, the model would be optimized to minimize the difference between the predicted probability distribution and the true distribution of possible answers.
The type of training data required for fine-tuning a GPT model for a specific task depends on the task itself. For question-answering, the training data would consist of pairs of questions and corresponding passages of text that contain the answers. For text classification, the training data would consist of labeled examples of text, with each example assigned to a specific category.
In general, the training data needs to be carefully curated and annotated to ensure that the model is able to learn the relevant patterns and relationships required for the specific task. This typically involves a significant amount of preprocessing and quality control to ensure that the training data is consistent and accurate.
Well, OK. That doesn’t really tell me what I want to know. But there’s something going on there that’s worth thinking about. I went on to quiz it about how GPTs are fitted out to tell stories – which I may post later, or perhaps just append those remarks to this post – but I had something to think about.
And a bit later it hit me – Sensory deprivation! Slocum! I’ve got some posts on those things. Joshua Slocum who circumnavigated the glob alone at the turn of the 20th century. He started seeing things. The same has happened to other explorers. And that’s what happens during sensory deprivation. You start hallucinating.
It's as though the mind actively needs and seeks input. If we’ve really got a so-called Bayesian brain that’s constantly guestimating what’s coming up next and revising its guestimations according to what actually shows up, then it does seem like we’re physically locked to the world and that the brain/mind needs input in order to maintain stability.
For a GPT the world consists of text. Without text it is nothing. Without textual input those 175 billion weights are just a bunch of numbers. But when you put “pressure” on them by presenting them with a string, NOW they’re in touch with a world. And that world imposes order on them.
This, whatever it is, this is somewhere moving from philosophy toward a quasi-technological account of how these things work – or is it the other direction? Does it matter? That’s right where we need to be at this point. We really are dealing with something new, really new, and philosophy is what we’ve got for doing that, at least it is if we can pry it free of its layers of scholastic encrustation.
Things are beginning to make sense.
From the YouTube page:
Khan Academy announcing its limited pilot of GPT-4 learning guide.
We believe that AI has the potential to transform learning in a positive way, but we are also keenly aware of the risks. To test the possibilities, we’re inviting our district partners to opt in to Khan Labs, a new space for testing learning technology. We want to ensure that our work always puts the needs of students and teachers first, and we are focused on ensuring that the benefits of AI are shared equally across society. In addition to teachers and students, we’re inviting the general public to join a waitlist to test Khanmigo. Teachers, students and donors will be our partners on this learning journey, helping us test AI to see if we can harness it as a learning tool for all.
To learn more about Khanmigo, visit: khanacademy.org/khan-labs
— Bill Benzon, BAM! Bootstrapping Artificial Minds (@bbenzon) March 15, 2023
I think dealing with hallucination is going to prove, shall we say, challenging: There’s truth, lies, and there’s ChatGPT [Realms of Being].