Saturday, March 25, 2023
Robots and elder-care
Jason Horowitz, Who Will Take Care of Italy’s Older People? Robots, Maybe. NYTimes, Mar. 25. 2023.
CARPI, Italy — The older woman asked to hear a story.
“An excellent choice,” answered the small robot, reclined like a nonchalant professor atop the classroom’s desk, instructing her to listen closely. She leaned in, her wizened forehead almost touching the smooth plastic head.
“Once upon a time,” the robot began a brief tale, and when it finished asked her what job the protagonist had.
“Shepherd,” Bona Poli, 85, responded meekly. The robot didn’t hear so well. She rose out of her chair and raised her voice. “Shep-herd!” she shouted.
“Fantastic,” the robot said, gesticulating awkwardly. “You have a memory like a steel cage.”
The scene may have the dystopian “what could go wrong?” undertones of science fiction at a moment when both the promise and perils of artificial intelligence are coming into sharper focus. But for the exhausted caregivers at a recent meeting in Carpi, a handsome town in Italy’s most innovative region for elder care, it pointed to a welcome, not-too-distant future when humanoids might help shrinking families share the burden of keeping the Western world’s oldest population stimulated, active and healthy.
Italy's elders:
Robots are already interacting with the old in Japan and have been used in nursing homes in the United States. But in Italy, the prototype is the latest attempt to recreate an echo of the traditional family structure that kept aging Italians at home.
The Italy of popular imagination, where multigenerational families crowd around the table on Sunday and live happily under one roof, is being buffeted by major demographic headwinds.
Low birthrates and the flight of many young adults for economic opportunities abroad has depleted the ranks of potential caregivers. Those left burdened with the care are often women, taking them out of the work force, providing a drag on the economy and, experts say, further shrinking birthrates.
Yet home care remains central to the notion of aging in a country where nursing homes exist but Italians vastly prefer finding ways to keep their old with them.
For decades, Italy avoided a serious reform of its long-term care sector by filling the gap with cheap, and often off-the-books, live-in workers, many from post-Soviet Eastern Europe — and especially Ukraine.
“That’s the long-term care pillar of this country,” said Giovanni Lamura, the director of Italy’s leading socio-economic research center on aging. “Without that, the whole system would collapse.”
There's more at the link.
Friday, March 24, 2023
Rodney Brooks sounds a cautionary note about GPTs
Brooks just made a post specifically directed at the type surrounding transformers. First a bit of historical perspective:
A few such instances of AI technologies that have induced gross overestimates of how soon we would get to AGI, in roughly chronological order, that I personally remember include:
John McCarthy’s estimate that the computers of the 1960’s were powerful enough to support AGI, Minsky and Michie and Nilsson each believing that search algorithms were the key to intelligence, neural networks (volume 3, perceptrons) [[I wasn’t around for the first two volumes; McCulloch and Pitts in 1943, Minsky in 1953]], first order logic, resolution theorem proving, MacHack (chess 1), fuzzy logic, STRIPS, knowledge-based systems (and revolutionizing medicine), neural networks (volume 4, back propagation), the primal sketch, self driving cars (Dickmanns, 1987), reinforcement learning (rounds 2 and 3), SOAR, qualitative reasoning, support vector machines, self driving cars (Kanade et al, 1997), Deep Blue (chess 2), self driving cars (Thrun, 2007), Bayesian inference, Watson (Jeopardy, and revolutionizing medicine), neural networks (volume 5, deep learning), Alpha GO, reinforcement learning (round 4), generative images, and now large language models. All have heralded the imminence of human level intelligence in machines. All were hyped up to the limit, but mostly in the days when very few people were even aware of AI, so very few people remember the levels of hype. I’m old. I do remember all these, but have probably forgotten quite a few…
None of these things have lived up to that early hype. As Amara predicted at first they were overrated. But at the same time, almost every one of these things have had long lasting impact on our world, just not in the particular form that people first imagined. As we twirled them around and prodded them, and experimented with them, and failed, and retried, we remade them in ways different from how they were first imagined, and they ended up having bigger longer term impacts, but in ways not first considered.
How does this apply to GPT world?
Then a caveat:
Back in 2010 Tim O’Reilly tweeted out “If you’re not paying for the product then you’re the product being sold.”, in reference to things like search engines and apps on telephones.
I think that GPTs will give rise to a new aphorism (where the last word might vary over an array of synonymous variations):
If you are interacting with the output of a GPT system and didn’t explicitly decide to use a GPT then you’re the product being hoodwinked.
I am not saying everything about GPTs is bad. I am saying that, especially given the explicit warnings from Open AI, that you need to be aware that you are using an unreliable system.
He goes on to say:
When no person is in the loop to filter, tweak, or manage the flow of information GPTs will be completely bad. That will be good for people who want to manipulate others without having revealed that the vast amount of persuasive evidence they are seeing has all been made up by a GPT. It will be bad for the people being manipulated.
And it will be bad if you try to connect a robot to GPT. GPTs have no understanding of the words they use, no way to connect those words, those symbols, to the real world. A robot needs to be connected to the real world and its commands need to be coherent with the real world. Classically it is known as the “symbol grounding problem”. GPT+robot is only ungrounded symbols. [...]
My argument here is that GPTs might be useful, and well enough boxed, when there is an active person in the loop, but dangerous when the person in the loop doesn’t know they are supposed to be in the loop. [This will be the case for all young children.] Their intelligence, applied with strong intellect, is a key component of making any GPT be successful.
At last, his specific predictions:
Here I make some predictions for things that will happen with GPT types of systems, and sometimes coupled with stable diffusion image generation. These predictions cover the time between now and 2030. Some of them are about direct uses of GPTs and some are about the second and third order effects they will drive.
- After years of Wikipedia being derided as not a referable authority, and not being allowed to be used as a source in serious work, it will become the standard rock solid authority on just about everything. This is because it has built a human powered approach to verifying factual knowledge in a world of high frequency human generated noise.
- Any GPT-based application that can be relied upon will have to be super-boxed in, and so the power of its “creativity” will be severely limited.
- GPT-based applications that are used for creativity will continue to have horrible edge cases that sometimes rear their ugly heads when least expected, and furthermore, the things that they create will often arguably be stealing the artistic output of unacknowledged humans.
- There will be no viable robotics applications that harness the serious power of GPTs in any meaningful way.
- It is going to be easier to build from scratch software stacks that look a lot like existing software stacks.
- There will be much confusion about whether code infringes on copyright, and so there will be a growth in companies that are used to certify that no unlicensed code appears in software builds.
- There will be surprising things built with GPTs, both good and bad, that no-one has yet talked about, or even conceived.
- There will be incredible amounts of misinformation deliberately created in campaigns for all sorts of arenas from political to criminal, and reliance on expertise will become more discredited, since the noise will drown out any signal at all.
- There will be new categories of pornography.
So, just why do GPTs have to operate by continuing an existing string?
Why can’t they, you know, just come out with it, like we do?
Yeah, I know, that’s how they’re trained; they’re trained to predict the next token. I’m not sure what that tells me.
So, when I give ChatGPT the prompt, “Tell me a story,” that isn’t what is actually presented to the underlying LLM. There isn’t anything that ‘naturally’ follows such a string. It could be literally almost anything. So, a bit of manipulation has to take place in order to give the LLM something to work with. But why?
I decided to put the question to GPT-4:
GPTs (that is, transformers) are constructed to continue an existing string of tokens. Why is that? Why can’t they simply answer a question without continuing an existing string?
GPTs, or Generative Pre-trained Transformers, are language models that are pre-trained on large amounts of text data using unsupervised learning. These models are designed to generate coherent and contextually relevant text based on the input provided to them.
The reason GPTs are constructed to continue an existing string of tokens is because they are designed to capture the context and meaning of the text, rather than just providing a one-word answer to a question. By generating a continuation of the input text, the model can incorporate the surrounding context and generate a more complete and accurate response.
In other words, GPTs are not just simple question-answering systems, but rather complex language models that can generate text that is contextually relevant and meaningful. By training on a large corpus of text, GPTs can learn to identify patterns and relationships in language, allowing them to generate text that is more than just a simple answer to a question.
That being said, GPTs can be fine-tuned to answer specific questions or perform specific tasks, such as question-answering or text classification, by providing them with appropriate training data and modifying their architecture accordingly.
That phase hit home: “...they are designed to capture the context and meaning of the text, rather than just providing a one-word answer to a question.” It’s that last clause that got my attention.
So I asked another question:
Tell me about that last. How is their architecture modified? What kind of training data is necessary?
To fine-tune a GPT model for a specific task, such as question-answering, the architecture of the model needs to be modified to accommodate the specific requirements of the task. This typically involves adding new layers or modifying existing layers of the model to better capture the specific features of the task.
For example, to fine-tune a GPT model for question-answering, the input data to the model would consist of a question and a passage of text that may contain the answer to the question. The model would then be trained to predict the correct answer based on the input question and text.
To accomplish this, the GPT model would be modified by adding a new output layer that is specific to the question-answering task. This layer would take the final hidden state of the model and generate a probability distribution over possible answers. During training, the model would be optimized to minimize the difference between the predicted probability distribution and the true distribution of possible answers.
The type of training data required for fine-tuning a GPT model for a specific task depends on the task itself. For question-answering, the training data would consist of pairs of questions and corresponding passages of text that contain the answers. For text classification, the training data would consist of labeled examples of text, with each example assigned to a specific category.
In general, the training data needs to be carefully curated and annotated to ensure that the model is able to learn the relevant patterns and relationships required for the specific task. This typically involves a significant amount of preprocessing and quality control to ensure that the training data is consistent and accurate.
Well, OK. That doesn’t really tell me what I want to know. But there’s something going on there that’s worth thinking about. I went on to quiz it about how GPTs are fitted out to tell stories – which I may post later, or perhaps just append those remarks to this post – but I had something to think about.
And a bit later it hit me – Sensory deprivation! Slocum! I’ve got some posts on those things. Joshua Slocum who circumnavigated the glob alone at the turn of the 20th century. He started seeing things. The same has happened to other explorers. And that’s what happens during sensory deprivation. You start hallucinating.
It's as though the mind actively needs and seeks input. If we’ve really got a so-called Bayesian brain that’s constantly guestimating what’s coming up next and revising its guestimations according to what actually shows up, then it does seem like we’re physically locked to the world and that the brain/mind needs input in order to maintain stability.
For a GPT the world consists of text. Without text it is nothing. Without textual input those 175 billion weights are just a bunch of numbers. But when you put “pressure” on them by presenting them with a string, NOW they’re in touch with a world. And that world imposes order on them.
This, whatever it is, this is somewhere moving from philosophy toward a quasi-technological account of how these things work – or is it the other direction? Does it matter? That’s right where we need to be at this point. We really are dealing with something new, really new, and philosophy is what we’ve got for doing that, at least it is if we can pry it free of its layers of scholastic encrustation.
Things are beginning to make sense.
Manga and AI in Japan
LOL! "At the same time, it turned out that Mr. Jerry Chi, the head of the Japanese branch office Stability AI Japan, is a fan of "Love Hina", so I would like to make an appointment and explore various things in the future."@CitizenSE @LeanneOgasawara @fschodt https://t.co/mxiwNPFWBP
— Bill Benzon, BAM! Bootstrapping Artificial Minds (@bbenzon) March 24, 2023
Thursday, March 23, 2023
Why is Sweden so successful in music? Scenes, it nurtures scenes.
Henrik Karlsson, Scene creation engines and apprenticeships, Escaping Flatland, Mar. 21, 2023.
After introductory material based on a story about specific musicians, Johan Schuster, Max Martin, and others, Karlston gets down to thinking:
But the most important factor behind Sweden’s outsized success seems to be that Sweden by accident created unusually good conditions for musical scene creation.
What is a scene? It is a group of people who are producing work in public but aimed at each other. The metal bands in Karlshamn, where Schuster grew up, were a scene. They performed on stage — but the audience was mainly their friends who played in other bands. If they were anything like the other local scenes I’ve seen, they challenged and supported each other to be bolder, more ambitious, better. A scene, to borrow a phrase from Visakan Veerasamy, “is a group of people who unblock each other at an accelerating rate”.
Almost invariably, when you notice someone doing bold original work, there is a scene behind them. Renaissance Florence was a scene for scholars and painters, flowering in Leonardo DaVinci. The coffeehouses of Elizabethan London were a scene for playwrights, flowering in Shakespeare.
But scenes are hard to get off the ground. They need resources and infrastructure that enable collective and open-ended tinkering. This is rarely achieved at scale.
But it was exactly this kind of infrastructure that evolved in Sweden during the mid-twentieth century. It was not done intentionally. Rather, it was an accidental side-effect of two political projects that were subverted by the grassroots: the Swedish public music education program and the study circle movement.
On the one hand:
In the 1940s, Swedish conservatives and church leaders were afraid that music imports from the US were leading to bad morals. They introduced an extensive and affordable after-school program for music. Kids could get tutoring 1-on-1 or in small groups for free once a week. They could borrow instruments, too. Later, fees were introduced, but around 30% of kids still receive free tutoring, and the fees only run up to about $100 per semester. About every third Swedish kid participates.
On the other:
In 1902, Oscar Olsson, a secondary school teacher, was working in the temperance movement, which aimed to steer the culture away from alcohol consumption and toward service of the community. To further this aim, Olsson started what he called study circles. It was basically a book club. A group of people would meet and discuss a book, and through these meetings they would develop themselves, forge social bonds, and inspire each other to raise their aspirations.
The temperance movement successfully lobbied the Riksdag, Sweden’s parliament, to get funding to purchase books. This funding was channeled through various grassroots organizations to their members on the condition that the books were made available to the general public afterward.
This funding led to explosive growth in self-organized study groups. By the 1970s, ten percent of the population were active members. But the study circles rapidly drifted away from Olsson’s original intentions. Instead of being a tool to further the agenda of political movements, it became an almost permissionless infrastructure for learning. [...] In a sort of exhausted truce, the control was ceded to the learners. Instead of ideological book clubs, people formed a myriad of learning communities: knitting clubs, discussion groups, and . . . bands.
This is turn let to the formation of lots of learning centers organized around different topics and activities, including music. And so:
The infrastructure provided by the music schools and the learning centers created fertile breeding grounds for scenes. This is where Max Martin, Johan Schuster, and nearly every other musician mentioned in this piece, spent their formative years.
And out of this breeding ground, another institution grew: the tradition for established songwriters to find young talent and nurture it through apprenticeships.
There’s much more at the link.
Adam Savage on Intuition [+ my intuitions about symbolic AI]
From the YouTube page:
Adam shares his absolute favorite magic book growing up: Magic with Science by Walter B. Gibson. Picking up this vintage copy is giving Adam memories of the countless times he pored over this book and how its demonstration of practical science experiments informed his approach and aesthetic style as a science communicator. Every illustration is clearcut and charming, and Adam is so happy to be reunited with this book!
Savage talks about reading about how things work in general, but in particular how magic tricks work as described and illustrated in this book. He puts a lot of stress on those illustrations.
And he also talks a lot about intuition (and how it is different from explicit knowledge). You get intuition, not from reading things, but from trying things out. Here he seems to be mostly about building things from ‘stuff’ and about doing those magic tricks. Intuition gives you a feel for things without, however, being (quite) able to explain what’s going on. You just know that this or that will work, or not.
I agree with this, and think a lot about intuition. I’m mostly interested in intuitions about literary works, and about thinking about the mind and so forth. In particular, it does seem to me that if you’ve done a lot of work with symbolic accounts of human thought, as I’ve done with cognitive networks, you have intuitions about language and mind that you can’t get from working on large language models (LLMs), such as GPTs. As far as I’m concerned, with advocates of deep learning discount the importance of symbolic thought, they (almost literally) don’t know what they’re talking about. Not only are they unfamiliar with the theories and models, but they lack the all-important intuitions.
More later.
* * * * *
Revised a bit from a note to Steve Pinker:
You’ve spent a lot of time thinking about language mechanisms in detail, so have I, though a somewhat different set of mechanisms. But I don’t think Mr. X has nor, for that matter, have most of the people involved in machine learning. Machine learning is about mechanisms to construct some kind of model over a huge database. But how that model actually works, that’s obscure. That is to say, the mechanisms that actually enact the cognitive labor are opaque. The people who build the models thus do not, cannot, have intuitions about them. In a sense, they’re not real. By extension, the whole world of cognitive science and GOFAI is not real. It is past. The fact that didn’t work very well is what’s salient. Therefore, the reasoning seems to go, those ideas have no value.
And THAT’s a problem. Every time Mr. X or someone else would talk about machines surpassing Einstein, Planck, etc. I’d wince. I couldn’t figure out why. At first I thought it might be implied disrespect but I decided that wasn’t it. Rather, it’s a trivialization of human accomplishment in the face of the dissonance between their confidence and the fact that they’ve haven’t got a clue about what would be involved beyond LOTS AND LOTS OF COMPUTE.
There’s no there there.
Wednesday, March 22, 2023
Hokusai "Great Wave" print sells for $2.8 million
Christie’s sold Katsushika Hokusai’s “Under the Well of the Great Wave off Kanagawa” for $2.8 million–a record high for the iconic woodblock print https://t.co/ff59UosOaI
— The Wall Street Journal (@WSJ) March 22, 2023
Report from Academia.edu • I’m Back! • Mar 22, 2023
Here’s the stats from 7:30 this morning:
I’m definitely down from the peak of March 13, but I seem to be running a bit higher than things were in February. Can I keep it up? Who knows.
But here’s the big news:
I’m now in the 99.9th percentile (top 0.1%) of Academia users. I had been running at the 99.5th percentile (top 0.5%). When you consider the nature of this distribution, that’s a significant jump.
Last March is the first time I got this high in the rankings, and that was an unexplained fluke. I had 875 views on March 28. I have no idea what caused that, but I was unable to sustain it. Perhaps things will go better this time. We’ll see.
How to make a fart sound using Ableton • [Winnebago Trickster Cycle] • {ChatGPT is allowed to tell fart jokes!}
I know, it's ridiculous. But it's also very human. Here, for example, is an excerpt from episode 23 in the Winnebago Trickster cycle as collected by Paul Radin:
As he went wandering around aimlessly he suddenly heard someone speaking. He listened very carefully and it seemed to say, 'He who chews me will defecate; he will defecate!' That was what it was saying. 'Well, why is this person talking in this manner?' said Trickster. So he walked in the direction from which he had heard the speaking and again he heard, quite near him, someone saying: 'He who chews me, he will defecate; he will defecate!' This is what was said. 'Well, why does this person talk in such fashion ?' said Trickster. Then he walked to the other side. So he continued walking along. Then right at his very side, a voice seemed to say, 'He who chews me, he will defecate; he will defecate!' 'Well, I wonder who it is who is speaking. I know very well that if I chew it, I will not defecate.' But he kept looking around for the speaker and finally discovered, much to his astonishment, that it was a bulb on a bush. The bulb it was that was speaking. So he seized it, put it in his mouth, chewed it, and then swallowed it. He did just this and then went on.
'Well, where is the bulb gone that talked so much? Why, indeed, should I defecate? When I feel like defecating, then I shall defecate, no sooner. How could such an object make me defecate!' Thus spoke Trickster. Even as he spoke, however, he began to break wind. 'Well this, I suppose, is what it meant. Yet the bulb said I would defecate, and I am merely expelling gas. In any case I am a great man even if I do expel a little gas!' Thus he spoke. As he was talking he again broke wind. This time it was really quite strong. 'Well, what a foolish one I am. This is why I am called Foolish One, Trickster.' Now he began to break wind again and again. 'So this is why the bulb spoke as it did, I suppose.' Once more he broke wind. This time it was very loud and his rectum began to smart. 'Well, it surely is a great thing!' Then he broke wind again, this time with so much force, that he was propelled forward. 'Well, well, it may even make me give another push, but it won't make me defecate,' so he exclaimed defiantly. The next time he broke wind, the hind part of his body was raised up by the force of the explosion and he landed on his knees and hands. 'Well, go ahead and do it again ! Go ahead and do it again!' Then, again, he broke wind. This time the force of the expulsion sent him far up in the air and he landed on the ground, on his stomach. The next time he broke wind, he had to hang on to a log, so high was he thrown. . However, he raised himself up and, after a while, landed on the ground, the log on top of him. He was almost killed by the fall. The next time he broke wind, he had to hold on to a tree that stood near by.
And so it goes.
Fart jokes and fart sounds are a staple of comedy of all sorts. And, I suppose, is what this educational video is. It is comedy in the form of a deadpan tutorial on how to synthesize a fart sound with Ableton Live, a standard audio workstation for Macintosh and Windows computers. I laughed out loud a half-dozen or more times while listening to this. On the one hand we have the evolving fart sound, getting ever more convincing, in contrast to the high-tech means of creating it.
Enjoy.
From the YouTube page:
49 views Mar 19, 2023 #fartsounds #ableton #sounddesign
I couldn't find an Ableton-based tutorial on this very important topic, so I decided to make my own.
0:00 Intro
0:38 Basic Oscillator
1:55 VCA Envelope and Filter
3:17 What's causing the sound?
4:35 Filter and Wavetable Envelopes
7:25 HPF and Effects
10:16 Modwheel Mapping
11:02 Final Sound
This is from the good folks at OpenAI:
Tell a fart joke.
Sure, here's a classic one:
Why did the bicycle fall over?
Because it was two-tired! (toot-toot)
See, even ChatGPT is allowed to tell a fart joke. I wonder if they're a cultural universal?
Are fart jokes found in all cultures?
Flatulence is a universal bodily function, and humor is a universal aspect of human culture. Therefore, it's not surprising that references to flatulence and jokes about it are found in many different cultures around the world.
Fart jokes and references to flatulence have been documented in ancient Greek and Roman literature, as well as in the literature of medieval Europe, including works like Chaucer's Canterbury Tales. In Japanese culture, there is a tradition of "he-gassen," or "fart battles," where artists would depict humorous scenes of people farting on each other. In many African cultures, flatulence is considered a sign of good health and is often celebrated with laughter.
That being said, the prevalence of fart jokes and references to flatulence in a particular culture can vary. Some cultures may find such humor to be more acceptable or even encouraged, while others may consider it taboo or inappropriate. Ultimately, the presence and acceptance of fart jokes in a culture is influenced by many factors, including social norms, values, and beliefs.
I rest my case.
* * * * *
META: I didn't plan this post. I just happaned. First I saw the video show up at YouTube. Why not? I thought, it could be interesting. And it was, and funny. I kept laughing. Should I make a post of it? I thought. But I can't just post it, straight out. So I started to write a little commentary, something about farts and humor. Then I thought about the Winnebago Trickster cycle, which I've written about in one of my cultural evolution papers. Then I wondered whether or not OpenAI allowed ChatGPT to tell a fart joke. I had to find out. Penultimately, I asked it whether or not fart jokes were found in all cultures.
Finally, this little pendant.
Tuesday, March 21, 2023
When Phil Woods got the nod from Bird
I understand. In a way, I got the nod from Frank Foster at a workshop in Buffalo.
"Humans are not special, scale is" [brain evolution]
This is the main driving assumption of my research and it is still holding up after 10 years: Humans are not special, scale is.
— Tim Dettmers (@Tim_Dettmers) March 19, 2023
The other main fact (sparsity): Humans are not special, but primates are. Only primates and birds have neurons not proportional to their body size https://t.co/5UPkDblxNH
There's the abstract for the underlying paper:
Objectives
Two decades ago, Rilling and Seligman, hereafter abbreviated to RAS Study, suggested modern humans had relatively larger temporal lobes for brain size compared to other anthropoids. Despite many subsequent studies drawing conclusions about the evolutionary implications for the emergence of unique cerebral specializations in Homo sapiens, no re-assessment has occurred using updated methodologies.
Methods
We reassessed the association between right temporal lobe volume (TLV) and right hemisphere volume (HV) in the anthropoid brain. In a sample compiled de novo by us, T1-weighted in vivo Magnetic Resonance Imaging (MRI) scans of 11 extant anthropoid species were calculated by-voxel from the MRI and the raw data from RAS Study directly compared to our sample. Phylogenetic Generalized Least-Squares (PGLS) regression and trait-mapping using Blomberg's K (kappa) tested the correlation between HV and TLV accounting for anthropoid phylogeny, while bootstrapped PGLS regressions tested difference in slopes and intercepts between monkey and ape subsamples.
Results
PGLS regressions indicated statistically significant correlations (r2 < 0.99; p ≤ 0.0001) between TLV and HV with moderate influence from phylogeny (K ≤ 0.42). Bootstrapped PGLS regression did not show statistically significant differences in slopes between monkeys and apes but did for intercepts. In our sample, human TLV was not larger than expected for anthropoids.
Discussion
Updated imaging, increased sample size and advanced statistical analyses did not find statistically significant results that modern humans possessed a disproportionately large temporal lobe volume compared to the general anthropoid trend. This has important implications for human and non-human primate brain evolution.
Interactive story-telling with ChatGPT [What does ChatGPT know, and when?]
I’ve asked ChatGPT to tell many stories. Most of the time I’ve given it a simple prompt – often as simple as “Tell me a story” – but I’ve also asked it to produce a new story by revising an old one. This time I decided to try something different. What would happen if I gave it the first paragraph and asked it continue from there?
It's an obvious and simple idea, but I didn’t arrive at it in an obvious and simple way. Ever since I finished my working paper, ChatGPT tells stories, and a note about reverse engineering, I’ve been thinking about just how ChatGPT evolves a story. It’s clear that, unless otherwise specified, ChatGPT will tell a story with a happy ending. How is that encoded in those weights? Does ChatGPT “know” that the ending will be happy when it begins to tell the story?
Now, if I ask it to tell a sad story, it will do so. For example:
Tell me a sad story.
Once upon a time, there was a young boy named Jack who lived in a small village at the base of a mountain. Jack had always been fascinated by the mountain, and would often spend his days exploring its rocky slopes and lush forests. One day, while on one of his adventures, Jack met a wise old man who lived in a cabin at the top of the mountain. The old man told Jack that the mountain was sacred, and that it held a great power that could only be unlocked by someone pure of heart.
Jack was determined to unlock the mountain's secrets, and so he spent many months training his body and mind, becoming stronger and more focused with each passing day. Finally, he felt ready to attempt the climb to the summit. The journey was long and difficult, but Jack never gave up. He battled through fierce storms, climbed sheer cliffs, and crossed treacherous ice fields, all the while driven by his desire to unlock the mountain's secrets.
At last, Jack reached the summit, and there he found a beautiful cave. Inside the cave, he saw a glowing crystal that radiated an otherworldly light. As he reached out to touch the crystal, he felt a powerful energy flowing through his body, and he knew that he had unlocked the mountain's secrets.
But as he emerged from the cave, he saw that his village was on fire. His family and friends were gone, killed by raiders who had attacked while he was away. Jack's heart was filled with sadness and regret, for he realized that his quest for the mountain's secrets had cost him everything he held dear. He died alone, on top of the mountain, with nothing but the memories of his loved ones and the knowledge that he had unlocked the mountain's secrets, but at too high a cost.
Notice that that story doesn’t become sad until the fourth and final paragraph. Was it only then that it “knew” it would give the story a sad ending? How was that knowledge encoded?
If you know Shakespeare, you know that at some point all of the comedies almost collapse into disaster. And Romeo and Juliet reads like a comedy much of the way through. It doesn’t take much to tip a story one way or the other – you might want to look at my comparison between The Winter’s Tale and the story on which Shakespeare based it, Robert Greene’s Pandosto.
In a lot of the stories I’ve been working with the protagonist will be introduced in the first paragraph and the antagonist in the second paragraph. Does ChatGPT “know” the antagonist from the beginning or does it only “choose” one after it has specified the protagonist? If the later, how does the choice of a protagonist influence the choice of an antagonist? On that last question, I know from one experiment that if the protagonist is a giant chocolate milkshake, the antagonist will be chosen from the same universe.
So those are the sorts of questions I had on my mind when I decided fully specify an opening paragraph and see how ChatGPT would respond. Here’s my first attempt:
I am going to give you an opening paragraph. You continue the story from there.
If was a cold day in the middle of winter. Snow blanketed the ground. John had left his house at 5:30 in the morning and began to hike through the woods to his brother’s compound. He got there at just as the sun was rising. The front door had been ripped off its hinges and a wisp of smoke came out of the door.
John's heart raced as he cautiously stepped inside, his hand instinctively reaching for the hunting knife at his belt. The sight before him made his blood run cold. Furniture was overturned, glass shattered on the ground, and there was a trail of blood leading to the kitchen. He followed the trail, gripping his knife tighter, and found his brother lying on the floor, bleeding profusely from a wound on his leg. "What happened?" John asked, as he quickly tore off a piece of his own shirt to use as a makeshift bandage. His brother looked up at him with fear in his eyes and whispered, "They came...they took her."
And? It did what I asked, it continued the story. But it didn’t finish it. So...
Could you finish it.
John's mind raced as he tried to make sense of his brother's words. Who had taken who? And why? He knew he needed to act fast if he was going to save his brother and whoever had been taken. "Who took her? Who did this to you?" John asked urgently, helping his brother to sit up. "They were wearing masks, I couldn't see their faces," his brother replied weakly. "But they said something about a ransom and demanded money." John's heart sank as he realized what was happening. His brother had been targeted for his wealth, and now someone had taken his loved one hostage. Without wasting any more time, John grabbed his backpack and set off on a mission to rescue his brother's beloved and bring the criminals to justice.
Continue.
John quickly made his way out of the compound and back into the woods. He had to find any clues that could lead him to the kidnappers. The snow on the ground made it easier to track them, but the trail was already getting faint. He followed it as fast as he could, using all his wilderness skills to keep himself hidden and avoid detection. As he went deeper into the woods, he heard voices up ahead. He moved closer, being careful not to make a sound, and saw a group of men huddled around a small cabin. One of them was holding a woman, who was tied up and blindfolded.
John's heart raced as he realized he had found them. He knew he had to act fast before they could harm the woman or escape. He assessed the situation and formulated a plan. He moved around to the back of the cabin, where he saw a window that was slightly open. He quietly slipped inside and found himself in a small room. There was a table with a few chairs, and a door that led to another room. He heard footsteps coming from the other side of the door, so he quickly hid under the table, waiting for his moment to strike.
As the footsteps got closer, John held his breath, his heart pounding in his chest. The door opened, and two of the kidnappers walked in, talking to each other in hushed tones. John could see their shadows moving across the floor. He waited for the right moment, then sprang into action, grabbing the legs of the first kidnapper and knocking him to the ground. The second kidnapper tried to reach for his gun, but John was too quick for him. He disarmed him and knocked him out with a swift blow to the head.
John quickly untied the woman and removed her blindfold. She was shaken, but unharmed. "Are you okay?" he asked urgently. She nodded, tears streaming down her face. "Thank you," she whispered.
But John knew they weren't out of danger yet. They had to get out of there before the other kidnappers came looking for their missing comrades. He grabbed the woman's hand and led her outside, where they quickly made their way back to his brother's compound. Once they arrived, John contacted the authorities and gave them all the information he had about the kidnappers. With John's help, they were able to track down and arrest the criminals, bringing them to justice for their heinous crime.
In the end, John's bravery and quick thinking saved his brother's beloved and brought the kidnappers to justice. He knew that the dangers of the wilderness could be treacherous, but he also knew that he had the skills to face any challenge that came his way.
It took a bit of work, but ChatGPT managed to finish the story. And it’s somewhat more interesting than the fairy tales it had been giving me.
I decided to try it again. Here’s the resulting interaction, without comment:
I am going to give you an opening paragraph. You continue the story from there.
If was a cold day in the middle of winter. Jane could hear the traffic down on the street below as she awoke. She got out of bed, took a shower and got dressed. She put on her best suit because she would be meeting a new client at the office. She smiled at the doorman as she left her apartment building and went to the coffee shop around the corner where she had a cup of coffee and a bagel. When she got to the office, she learned that the new client was waiting for her. “Hello, Mr. Belknap, how are you this morning?” “I’m fine, he said. Let’s discuss the merger with Smith and Baskins.”
Jane shook Mr. Belknap's hand and led him to her office. As they walked, she couldn't help but feel a sense of unease. There was something about Mr. Belknap that made her feel uneasy, but she couldn't quite put her finger on it. Once they were in her office, they sat down and Jane began discussing the details of the merger. But every time she looked up at Mr. Belknap, he seemed to be staring at her in a strange way. She tried to ignore it and focus on the meeting, but she couldn't shake the feeling that something was off. Finally, she couldn't take it anymore. "Mr. Belknap, is everything okay? You seem to be staring at me." Mr. Belknap's expression changed suddenly, and he leaned forward in his chair. "Jane, I'm sorry, but I have to tell you something. You're in danger."
That's good. Continue.
Monday, March 20, 2023
Software of the future: Everyone codes!
it seems likely to me that all computer users will soon have ability to 1) develop small software tools from scratch, 2) describe *changes* they'd like made to existing software they're already using. what will this mean for software ecosystems?? 2/
— Geoffrey Litt (@geoffreylitt) March 19, 2023
you again: "most computer users aren't nearly as good as programmers / designers at turning fuzzy ideas into real tools!!"
— Geoffrey Litt (@geoffreylitt) March 19, 2023
yeah also fine, the LLM will help them develop the spec, people will improve, and yeah, maybe most people won't get great but doesn't matter 🙂 4/
There's much more in the thread. See also my updated PowerPoint Assistant 2023: Augmenting End-User Software through Natural Language Interaction.The status quo context: software is mostly hostile to customization today at the *architectural* level. Except for small plugin + extension surfaces, general assumption is that software is centrally developed and one-size-fits-all. 6/
— Geoffrey Litt (@geoffreylitt) March 19, 2023
Language, LLMs, and culture: Out of Plato's cave
Jon Evans, Language is out Latent Space, Gradient Ascendant, March 14, 2023.
An analogy for how LLMs work:
Another analogy, as two combined can be more illuminating than one: consider snooker, the pool-like game won by sinking balls of varying value in the best possible order. Imagine a snooker table the size of Central Park, occupied by thousands of pockets and millions of numbered balls (the numbers 1 through 32000, repeated.) Now imagine that the rules of snooker — i.e. which balls are most profitable to sink — change after every shot, depending on where the cue ball is, which balls have previously been sunk, the phase of the moon, etc.
Call that “Jungle Snooker”, borrowing from Eric Jang's idea of Jungle Basketball. The numbers on the balls represent word embeddings; ‘which balls to aim to sink in which order,’ the patterns in latent space. All we have really taught modern LLMs is how to be extremely (stochastically) good at Jungle Snooker, which doesn’t feel that different, qualitatively, from teaching them how to be extremely good at Go or chess. Now, the results, when converted into words, are phenomenal, often eerie —
— but LLMs still don’t “know” that their numbers represent words. In fact they never see words per se; we actually break language into tokens, word fragments basically like phonemes, number those tokens, and feed those numbers in as inputs.
Language as the latent space of culture:
Our latent space, known as language, implicitly encodes an enormous amount of knowledge about the world: concepts, relationships, interactions, constraints. LLM embeddings in turn implicitly include a distilled version of that knowledge. A reason LLMs are so unreasonably effective is that language itself is a machine for understanding, one which, it turns out, includes undocumented and previously unused capabilities — a “capability overhang.”
You might also look at Ted Underwood's paper, Mapping the latent spaces of culture.
Out of the cave:
Invert Plato's cave, and imagine yourself as a puppet master trying to reach out to chained prisoners to whom you can only communicate with shadows. Similarly, right now all we have are machines that we can teach to play Jungle Snooker.
But if we do ever build a machine capable of genuine understanding -- setting aside the question of whether we want to, and noting that people in the field generally think it's “when” not “if” -- it seems likely that language will be our most effective shacklebreaker, just as it was for us. This in turn means today's LLMs are likely to be the crucially important first step down that path.
The question is whether there is any iterative path from Jungle Snooker to Plato's Cave to emergence. Some people think we'll just scale there, and as machines get better at Jungle Snooker, they will naturally develop a facility for abstracting complexity into heuristics, which will breed agency and curiosity and a kind of awareness — or at least behavior indistinguishable from awareness — in the same way that embeddings and latent space spontaneously emerge when you teach LLMs.
Others (including me) suspect that whole new fundamental architectures and/or training techniques will be required. But either way, it seems very likely that language will be key, and that modern LLMs, though they'll seem almost comically crude in even five years, are a historically important technology. Language is our latent space, and that's what gives it its unreasonable power.
Yes, we'll need a whole new architecture. There's more at the link.
Steve Pinker on AI Doom [My take: a Rube Goldberg confabulation of Brobdingnagian proportions]
Richard Hanania has posted an interesting conversation with Steve Pinker, Pinker on Alignment and Intelligence as a "Magical Potion", March 17, 2023. Here a long excerpt from Pinker on AI Doom:
There’s a recurring fallacy in AI-existential-threat speculations to treat intelligence as a kind of magical pixie dust, a miracle elixir that, if a system only had enough of it, would grant it omniscience and omnipotence and the ability to instantly accomplish any outcome we can imagine. This is in contrast to what intelligence really is: a gadget that can compute particular outputs that are useful in particular worlds.
That’s an interesting speculation about resistance to IQ denial as a source of support for the concept of superintelligence. I suspect it’s not historically accurate – the superintelligence proponents I’ve seen don’t bring up or refute the Gouldian arguments against IQ, but just seem to operate under the folk theory that IQ is a measure of a magical potion that you can have in various amounts. I may be wrong, but I can’t recall any mentions of the psychometrics or behavioral genetics of intelligence in these discussions.
I think there are many things wrong with the argument that we should worry about AI creating a virus that kills us all.
First, why would an AI have the goal of killing us all (assuming we’re not talking about a James Bond villain who designs an AI with that in mind)? Why not the goal of building a life-size model of the Eiffel Tower out of popsicle sticks? There’s nothing inherent in being smart that turns a system into a genocidal maniac – the goals of a system are independent of the means to achieve a goal, which is what intelligence is. The confusion arises because intelligence and dominance happen to be bundled together in many members of Homo sapiens, but that’s because we’re products of natural selection, an inherently competitive process. (As I note in Enlightenment Now, “There is no law of complex systems that says that intelligent agents must turn into ruthless conquistadors. Indeed, we know of one highly advanced form of intelligence that evolved without this defect. They’re called women.”) An engineered system would pursue whatever goal it’s given.
Sometimes you see the assumption that any engineer would naturally program the generic goal of self-preservation, or self-aggrandizement, at all costs into an AI. No, only an idiot would do that. This is crude anthropomorphization, perhaps Freudian projection.
Second, and relatedly, these scenarios assume that an AI would be given a single goal and programmed to pursue it monomaniacally. But this is not Artificial Intelligence: it’s Artificial Stupidity. No product of engineering (or for that matter natural selection) pursues a single goal. It’s like worrying that since the purpose of a car is to get somewhere quickly, we should worry about autonomous vehicles that rocket in a straight line at 120 MPH, mowing down trees and pedestrians, without brakes or steering. I’ll quote myself again: “The ability to choose an action that best satisfies conflicting goals is not an add-on to intelligence that engineers might slap themselves in the forehead for forgetting to install; it *is* intelligence.” And “Of course, one can always imagine a Doomsday Computer that is malevolent, universally empowered, always on, and tamperproof. The way to deal with this threat is straightforward: don’t build one.”
The third fallacy is one that I mentioned in the excerpt you reposted: that sheer rational cogitation is sufficient to solve any problem. In reality intelligence is limited by knowledge of the world, which is an exponential space of possibilities governed by countless chaotic and random processes. Knowledge of the world is expensive and time-consuming to attain incrementally. Me again: “Unlike Laplace’s demon, the mythical being that knows the location and momentum of every particle in the universe and feeds them into equations for physical laws to calculate the state of everything at any time in the future, a real-life knower has to acquire information about the messy world of objects and people by engaging with it one domain at a time. Understanding does not obey Moore’s Law: knowledge is acquired by formulating explanations and testing them against reality, not by running an algorithm faster and faster. Devouring the information on the Internet will not confer omniscience either: big data is still finite data, and the universe of knowledge is infinite.”
Even the Bond-villain scenario is too facile. As Kevin Kelley noted in “The Myth of the Lone Villain,” in real life we don’t see solitary evil geniuses who wreak mass havoc, because it takes a team to do anything impressive, which multiplies the risk of detection and defection, and it inevitably faces a massively larger coalition of smarter people working to prevent the havoc from happening. And as Kelley and Hanson point out, no technology accomplishes something awesome the first time it’s turned on; there are always bugs and crashes, which would tip off the white hats. This doesn’t guarantee that there won’t be a successful solitary sociopathic AI-virus-designer-designer, but it’s not terribly likely.
Many of the scenarios pile up more layers of Artificial Stupidity, such as assuming that human flesh is a good source of material for paperclips, or even that annihilating humans is plausible means to the end of self-preservation.
The AI-existential-threat discussions are unmoored from evolutionary biology, cognitive psychology, real AI, sociology, the history of technology and other sources of knowledge outside the theater of the imagination. I think this points to a meta-problem. The AI-ET community shares a bad epistemic habit (not to mention membership) with parts of the Rationality and EA communities, at least since they jumped the shark from preventing malaria in the developing world to seeding the galaxy with supercomputers hosting trillions of consciousnesses from uploaded connectomes. They start with a couple of assumptions, and lay out a chain of abstract reasoning, throwing in one dubious assumption after another, till they end up way beyond the land of experience or plausibility. The whole deduction exponentiates our ignorance with each link in the chain of hypotheticals, and depends on blowing off the countless messy and unanticipatable nuisances of the human and physical world. It’s an occupational hazard of belonging to a “community” that distinguishes itself by raw brainpower. OK, enough for today – hope you find some of it interesting.
Sunday, March 19, 2023
Khan Academy has a GPT-4 Learning Guide
From the YouTube page:
Khan Academy announcing its limited pilot of GPT-4 learning guide.
We believe that AI has the potential to transform learning in a positive way, but we are also keenly aware of the risks. To test the possibilities, we’re inviting our district partners to opt in to Khan Labs, a new space for testing learning technology. We want to ensure that our work always puts the needs of students and teachers first, and we are focused on ensuring that the benefits of AI are shared equally across society. In addition to teachers and students, we’re inviting the general public to join a waitlist to test Khanmigo. Teachers, students and donors will be our partners on this learning journey, helping us test AI to see if we can harness it as a learning tool for all.
To learn more about Khanmigo, visit: khanacademy.org/khan-labs
What’s up with Animal House [Media Notes 87]
I never saw it in theaters, though I certainly heard about it. Animal House came out in 1978, just as I was finishing graduate school and about to take my first (and only) academic gig. From the Wikipedia entry:
Of the younger lead actors, only the 28-year-old Belushi was an established star, but even he had not yet appeared in a film, having gained fame as an original cast member of Saturday Night Live, which was in its third season in autumn 1977. Several of the actors who were cast as college students, including Hulce, Karen Allen, and Kevin Bacon, were just beginning their film careers. Matheson, also cast as a student, was already a seasoned actor, having appeared in movies and television since the age of 13.
Filming took place in Oregon from October to December 1977. Following its initial release on July 28, 1978, Animal House received generally mixed reviews from critics, but Time and Roger Ebert proclaimed it one of the year's best. Filmed for only $3 million, it garnered an estimated gross of more than $141 million in the form of theatrical rentals and home video, not including merchandising, making it the highest grossing comedy film of its time.
The film, along with 1977's The Kentucky Fried Movie, also directed by Landis, was largely responsible for defining and launching the gross out film genre, which became one of Hollywood's staples Animal House is now regarded as one of the best comedy films of all time.
In 2001, the United States Library of Congress deemed National Lampoon's Animal House "culturally, historically, or aesthetically significant" and selected it for preservation in the National Film Registry. It was No. 1 on Bravo's "100 Funniest Movies". It was No. 36 on AFI's "100 Years... 100 Laughs" list of the 100 best American comedies. In 2008, Empire magazine selected it as No. 279 of "The 500 Greatest Movies of All Time".
That’s a lot of praise.
I don’t get it. Did I laugh? Yes, here and there, and at the end. But I found the people rather uniformly dull or unpleasant.
Perhaps I’m missing something. I note that the Marx Brothers generally played unpleasant characters in their films, such as Duck Soup, which I recently watched. But they were generally funny, except for Zeppo, who played a straight man. I didn’t find the characters in Animal House very funny. Belushi was weird and outrageous, but I liked him better in The Blues Brothers, which I liked a lot. Perhaps I just don’t like gross out films. Or is it fraternities and fraternity culture I don’t like? Johns Hopkins had fraternities when I was there, but it wasn’t a big fraternity school. I didn’t pledge, but a good friend of mine had.
Perhaps I’m just missing something. And perhaps all those who seem to think the film is up there with sliced bread are missing something. Who knows?
For sure, we’re all missing something.
Top Posts at New Savanna 19 Mar 23 [& notes about why]
Every once in a while I make a post about what’s going on here at New Savanna, though I tend to include action at my Academia page under that tag as well. Back in August of 2019 I took a look at top posts. Back in August of 2013 I took a look at the all-time most popular posts, as of that time. I gave counts for the top 35 posts.
It's time for another look, though I’m not going to survey the top 35. I have something different in mind.
This shows the action from the beginning, in April 2010, to the present:
Notice that big spike there around 2017. I have no idea what’s going on there. Here’s a short post about it.
This is Blogger’s list of the top ten all-time posts:
The counts are off, moreover, when you take a closer look, those really aren’t the top ten. I’ll look at the in a minute.
But let’s take a look at the list as it stands. The top post is about Ninja Scroll. That’s one of the earliest posts I ever did and it has been the top post as long as I’ve been checking. Five more of the top posts are about animation, either Disney or Miyazaki. While I’ve done many posts featuring photos of irises, that particular post has only one photo. It’s a nice photo, but I do not know why it has proven so popular. I assume the popularity of Joe Rogan post is a function of Rogan’s popularity plus Bruce Lee and Chuck Norris (it’s about a well-known fight scene). The Hilary Hahn post is nice, and I believe benefited from being listed over at Marginal Revolution. Finally, we have that old post about what’s popular here. Why’s it so popular?
This table gives more accurate counts for those posts, along with direct links to them. Thus, whereas the Bogger states had Ninja scroll at 138K, the true figure seems to be 142K. Going to the bottom, we have 3.08K for Hilary Hahn vs. 3.59K. But look at the disparity for Pigs in Spirited Away, 3.67K vs. 14.2K. Something’s messed up. I have no idea what, but I decided to do some looking around.
|
Top Ten Posts at New Savanna Listed in order according to Blogger’s order and not by “true” count March 19, 2023 |
|
|
142,091 |
|
|
94,000 |
|
|
Horikoshi’s Wife: Affective Binding and Grief in The Wind Rises |
54,877 |
|
10,082 |
|
|
9,435 |
|
|
6,338 |
|
|
8,236 |
|
|
What’s Popular and Why (April 2011) |
5,961 |
|
14,212 |
|
|
3,590 |
|
Since animation is by far the post popular topic, I looked at animation posts. I’ve created tables for Miyazaki posts, Disney posts, and other animation.
|
Most Popular Miyazaki Posts March 19, 2023 |
|
|
Horikoshi’s Wife: Affective Binding and Grief in The Wind Rises |
54,877 |
|
14,212 |
|
|
9,326 |
|
|
6,193 |
|
|
6,876 |
|
|
6,193 |
|
|
5,626 |
|
|
Most Popular Disney Posts March 19, 2023 |
|
|
94,000 |
|
|
9,435 |
|
|
8,236 |
|
|
6,682 |
|
|
5,040 |
|
|
4,373 |
|
|
Other Popular Animation Posts March 19, 2023 |
|
|
142,091 |
|
|
7,256 |
|
|
4,745 |
|
|
4,140 |
|
|
3,739 |
|
|
2,304 |
|
I’m particularly pleased the The Hottest Man in Siam has done so well. It’s about a Walter Lantz cartoon, “The Greatest Man in Siam,” which features an amazing dance sequence, two people doing the Lindy to a really hot trumpet solo. The post is mostly screen shots of that dance sequence. It’s hard to find online. The last time I looked I couldn’t find it. But maybe someone’s put it up again.
Subscribe to:
Posts (Atom)