Conceptual coherence for concrete categories in humans and LLMs

Siddharth Suresh, Kushin Mukherjee, Xizheng Yu, Wei-Chun Huang, Lisa Padua, and Timothy T. Rogers, Conceptual structure coheres in human cognition but not in large language models, arXiv:2304.02754v2 [cs.AI] 10 Nov 2023.

Abstract: Neural network models of language have long been used as a tool for developing hypotheses about conceptual representation in the mind and brain. For many years, such use involved extracting vector-space representations of words and using distances among these to predict or understand human behavior in various semantic tasks. Contemporary large language models (LLMs), however, make it possible to interrogate the latent structure of conceptual representations using experimental methods nearly identical to those commonly used with human participants. The current work utilizes three common techniques borrowed from cognitive psychology to estimate and compare the structure of concepts in humans and a suite of LLMs. In humans, we show that conceptual structure is robust to differences in culture, language, and method of estimation. Structures estimated from LLM behavior, while individually fairly consistent with those estimated from human behavior, vary much more depending upon the particular task used to generate responses– across tasks, estimates of conceptual structure from the very same model cohere less with one another than do human structure estimates. These results highlight an important difference between contemporary LLMs and human cognition, with implications for understanding some fundamental limitations of contemporary machine language.

What the abstract doesn’t tell you is that the categories under investigation are for concrete objects and not abstract: “The items were drawn from two broad categories– tools and reptiles/amphibians–selected because they span the living/nonliving divide and also possess internal conceptual structure.” Why does this matter? Because the meaning of concrete items is grounded in sensorimotor schemas whie the meaning of abstract is not.

In their conclusion, the authors point out:

Together these results suggest an important difference between human cognition and current LLM models. Neuro-computational models of human semantic memory suggest that behavior across many different tasks is undergirded by a common conceptual "core" that is relatively insulated from variations arising from different contexts or tasks (Rogers et al., 2004; Jackson et al., 2021). In contrast, representations of word meanings in large language models depend essentially upon the broader linguistic context. Indeed, in transformer architectures like GPT- 3, each word vector is computed as a weighted average of vectors from surrounding text, so it is unclear whether any word possesses meaning outside or independent of context.

For humans, the sensorimotor grounding of concrete concepts provides that conceptual core, which is necessarily lacking for LLMs, which do not have access to the physical world. Context is all they’ve got, and so their sense of meanings for words will necessarily be drawn to the context. The authors acknowledge this point at the end:

Finally, Human semantic knowledge is the product of several sources of information including visual, tactile, and auditory properties of the concept. While LLMs can implicitly acquire knowledge about these modalities via the corpora they are trained on, they are nevertheless bereft of much of the knowledge that humans are exposed to that might help them organize concepts into a more coherent structure. In this view, difference in the degree in conceptual coherence between LLMs and humans should not be surprising.

Surprise!

When their Rogue AI creatures start coming after us, the rich digerati have plans to bug out

David Bosworth, No Exit: The Uncivil Folly of Libertarian Flight, The Hedgehog Review, Fall 2023.

One of the scandalous revelations of the COVID pandemic was just how many of America’s superrich—our digerati, venture capitalists, corporate monopolists, hedge fund managers—had long been planning to abandon their fellow citizens should a dire national crisis arise. While poorly paid EMTs and other frontline health workers were risking their lives caring for the desperately ill, wealthy Americans who had amassed their fortunes during our tech-driven Gilded Age were fueling their private jets and stocking their remote shelters in unabashed displays of their proudly vaunted libertarian creed.

Such plans of escape took two forms: one leading to the private hideaway, the other to the utopian settlement. In the case of the former, the billionaires themselves initiated the project, commissioning fail-safe residences in places such as New Zealand, which, remote from global turmoil and offering government policies hospitable to foreign wealth, has become a favorite destination for today’s digerati deserters. [...]

Concerns about natural disasters or possible wars are not the only motivation for the most radical of America’s libertarians to secure hideaways or acquire citizenship in foreign lands. For this group of exiteers, paying taxes, submitting to government regulations, or simply living in a society made uncivil in part by the very technologies that have fed their wealth are burdens or dangers to be avoided. According to their credo of freedom über alles, to be a “man without a country” is not a punishment but a privileged exemption from the manacles of communal obligation. Rather than a role defined by duties as well as rights, citizenship for them is just one more consumer option in a competitive global marketplace.

Confounding the image of the survivalist as a loser driven by economic failure or religious fanaticism, other high achievers are prepping for the ultimate disaster in the United States. They have purchased remote properties in Idaho, Montana, and on islands off the coast of Washington State, stocking their retreats with food, fuel, and armaments, and investing in cryptocurrencies to insure their wealth against the imminent collapse of a financial order that has heretofore served them so well. Some preparations are more personal and particular: Reddit CEO Steve Huffman had Lasik surgery in 2015, not for appearances’ sake but out of fear that when our society collapsed, as he expected, he would be unable to buy contact lenses.

And then we have the utopians

Despite the contradictions inherent in the desire to create a utopian community populated by radical individualists, several American-led attempts have been made to establish just such a place, all ending in failure. In Adventure Capitalism: A History of Libertarian Exit, From the Era of Decolonization to the Digital Age, historian Raymond B. Craib examines how wealthy activist Michael Oliver conceived, financed, and helped organize three of those futile efforts.7 Oliver’s personal history clarifies the likely origins of his zealous pursuit of a libertarian paradise. Born Moses Olitzsky, a Lithuanian Jew and the only member of his family to survive the Holocaust, he immigrated to America after World War II. Adopting a new name, he made his money in Nevada real estate and by dealing in gold and silver coins—commodities long favored by those who fear the sort of social unrest and economic collapse that preceded the war. Utopian true believers are driven by an extreme push-pull of antipodal expectations: a paranoid pessimism that anticipates the imminent breakdown of their social order accompanied by the hope that such a catastrophe will clear the way for a new, uncorrupted Eden. For Oliver, with his traumatic memories of a Europe beset by anarchic violence, the social unrest America suffered in the 1960s was an ominous spectacle that he presumed would end with a new fascist regime.

There's much more at the link.

Mechanistic interpretability is necessary, but not sufficient, for understanding how LLMs work, a short note

A comment I recently posted at LessWrong:

ryan_greenblatt – By mech interp I mean "A subfield of interpretability that uses bottom-up or reverse engineering approaches, generally by corresponding low-level components such as circuits or neurons to components of human-understandable algorithms and then working upward to build an overall understanding."

That makes sense to me, and I think it is essential that we identify those low-level components. But I’ve got problems with the “working upward” part.

The low-level components of a gothic cathedral, for example, consist of things like stone blocks, wooden beams, metal hinges and clasps and so forth, pieces of colored glass for the windows, tiles for the roof, and so forth. How do you work upward from a pile of that stuff, even if neatly organized and thoroughly catalogues, how do you get from there to the overall design of the overall cathedral. How, for example, can you look at that and conclude, “this thing’s going to have flying buttresses to support the roof?”

Somewhere in How the Mind Works Steven Pinker makes the same point in explaining reverse engineering. Imagine you’re in an antique shop, he suggests, and you come across odd little metal contraption. It doesn’t make any sense at all. The shop keeper sees your bewilderment and offers, “That’s an olive pitter.” Now that contraption makes sense. You know what it’s supposed to do.

How are you going to make sense of those things you find under the hood unless you have some idea of what they’re supposed to do?

The sort of work I’ve done with ChatGPT’s storytelling or with its ontological capabilities provides clues that complement the phenomena discovered through mechanistic interpretability. Beyond that I’ve been thinking about the possibility that GPTs are associative memories in which the generation of a token is a single primitive operation for the underlying virtual machine. By that I mean there are no logical operations being performed within that operation, just straight calculation.

Am I right? It’s too early to say. But we have to start somewhere.

Friday Fotos: On the Light Rail, from Hoboken to Jersey City

GOAT Literary Critics: Part 1c, Who's Who in Lit Crit over the last half century, the view from Google Ngram

I was looking over some old posts and found this one from 2017.  It fits right in with my current series on great all-time literary critics, some I'm bumping it to the top of the queque.

I decided to do some Ngram searches on the names of important literary critics. Let's start with Northrup Frye:

He published The Anatomy of Criticism in 1957 and that, I believe, is the book that put him on the map. He tops out in the late 1980s.

Here we've got Frye, plus four other critics, Hillis Miller, George Steiner, Stanley Fish, and Harold Bloom. Frye outpaces all of them except Bloom, and Bloom passes him only in 1994 or so, where Frye is coming down and Bloom is at his apogee. Harold Bloom outpaces Fish, Steiner, and Miller, presumably because he developed a general audience readership that they did not. Notice as well that the other three (Hillis Miller, George Steiner, Stanley Fish) peaked in the 1990s.

Now let's add Derrida to the group:

Not surprisingly he sends them all to the showers. Of course, he's not a literary critic. He's a philosopher with a strong interest in literature and, of course, who exerted a strong influence on literary criticism.

Notice, in passing, that Derrida also outpaces Noam Chomsky, sometimes touted as the best-known intellectual in the world (as much for his politics, if not more, as for his linguistics):

Finally, let's recontextualize Frye and situate him among the New Critics:

John Crow Ransom is there at the bottom, while Frye rises above the others in the middle and late 1960s. Both Brooks and Warren had been students of Ransom and, of course, they had their names on two of the best-known undergraduate textbooks for literature, Understanding Poetry and Understanding Fiction. Warren also won Pulitzer prizes in both fiction and poetry (the only one to do so) and had a novel, All the King's Men, made into a major motion picture. Still, since 1970 Frye was mentioned more often in books than any of the New Critics.

Finally:

Conclusions?

UPDATE AT 76: Reflections on entering my eighth decade and why it portends to be the most productive one of my life

I posted this six years ago, a few days after my 70th birthday. It’s still a reasonable overview of things I’ve accomplished, though the future looks a bit different. GPT-3 and then ChatGPT came along unexpectedly. I’ve devoted a great deal of time on ChatGPT over the last year, over 100 blog posts and 11 working papers. I’d hoped to finish a 12th paper by today, but other things intervened so I’ve still go a day’s or so worth of work on it. Then I’ve got to look over that work, arrive at some conclusions, and set directions for future work. That’ll take several more days.

One of those working papers, Xanadu, GPT, and Beyond: An adventure of the mind, outlines the connections between this stand of work with my earlier work going back to “Kubla Khan” and even a bit before. Back in 2022 I posted Relational Nets Over Attractors, A Primer: Part 1, Design for a Mind, which knits together the work I did with David Hays, first on cognitive networks in the 1970s, and then our two papers on the brain in 1988, Principles and Development of Natural Intelligence and Metaphor, Recognition, and Neural Process. That was unexpected.

It also appears that I’ve undertaken yet another overview of literary criticism, GOAT Literary Critics. I’ve posted two pieces already and have tentative plans for five more. I don’t know just when I’ll do them as the ChatGPT wrap-up has higher priority.

Finally, back in 2017 I wrote about how I made Jersey City my home: through photographs. Now Hoboken has become my home and, once more, it’s been through photographs. I post photos to a Facebook group for Hoboken photos and am planning a small exhibit of some of my photos. I’m working with one of my Jersey City friends, Greg Edgell, on that (& Greg’s gotten married and moved to the house near Morristown where he grew up).

Things are moving along. Like my friend Al used to say, “These are the good old days. The best days are yet to come.”

In destinies sad or merry, true men can but try.

– Sir Gawain and the Green Knight 

In scientific prognostication we have a condition analogous to a fact of archery—the farther back you draw your longbow, the farther ahead you can shoot.

– Buckminster Fuller

Birthdays are generally just that, even “major” birthdays, like my most recent one, my 70th. They are an occasion for a celebration, perhaps a modest one, perhaps not quite so modest – our house was crowded with dinner guests for my father’s 50th birthday – perhaps even extravagant. I’ve never been to an extravagant birthday party. Birthdays may also be a time for reflection, but by no means necessarily so.

But in my experience birthdays rarely correspond to major life events. What’s a major life event? Getting married, whether at a small civil ceremony before a judge or an elaborate wedding with 100s of guests into the church and out to the reception where a great band – like me and my colleagues in The Out of Control Rhythm and Blues Band – performs for hours of dancing, that’s a major life event. Climbing a mountain you’ve trained for over a period of years, that’s a different kind of major life event. Graduating from school, or completing basic training in the military, passing the bar exam, all major.

Years ago, in my early 20s, I was in a rock band called “St. Matthew Passion.” It was our last gig, the sax player and I were jamming a whacked out intro to “She’s Not There” and suddenly it all disappeared, me, the musicians, the room, the world, all into a brilliant, but soft, white light. Only the light and the music. It lasted what, half a second, a second, two seconds? Whatever. Those few moments challenged me for years, changing my sense of myself and the world.

Major life events come in all forms and durations, but they rarely coincide with a birthday. Birthdays simply mark the passage of time.

And so it set out to be on Thursday, December 7, 2017, when I turned 70. I woke up, cruised the web, made four posts to New Savanna, had breakfast and then, and then I decided to go out and take some photos, including some of that green platform pump I’ve been having so much fun photographing.

That was a bit unusual because I generally write in the morning, and perhaps I was motivated by my birthday to do something a bit different. But that’s all it was, a change in routine. It’s no big deal; I do it all the time.

But then I realized, sometime in the afternoon, that this birthday IS a big deal, and that I really can make it a big deal. How? By finishing my working paper, Calculating meaning in “Kubla Khan” – a rough cut. Why is that important, major milestone important? Because I’ve been working on it almost 50 years, all my adult life.

Teaser: Calculating meaning in “Kubla Khan”, 2017

This image likely makes little sense. Don’t worry about it. There are two more images that won’t make much sense. Don’t worry about them either. Just look at them as you would displays in a museum or gallery and move on.

Not that paper, no, not that. It’s the poem, “Kubla Khan”, by Samuel Taylor Coleridge, that I’ve been tracking all my adult life. I’ve been working on it since the spring of 1969 when I read it in Earl Wasserman’s class in my senior year at The Johns Hopkins University in Baltimore – where my father had gone to school. To have completed a project that framed one’s adult life, that is indeed a major event. Not completed, not in the sense that it’s all over and done with – for it isn’t, but in some deep and fundamental sense, things have changed. I’ve got a new understanding, and new obligations to go with it.

The first five lines:

In Xanadu did Kubla Khan
A stately pleasure-dome decree:
Where Alph, the sacred river, ran
Through caverns measureless to man
          Down to a sunless sea.

There are 49 more.

Let me explain. Perhaps then you will understand why I expect the next decade to be the most productive one of my life. And not just my intellectual life. There is the Bergen Arches Project as well. And who knows what else? I wonder if Rita Moreno is available for salsa lessons?

Photoshop AI magic: Before (top) and After (bottom), one brush stroke

GOAT Literary Critics: Part 1b, Lit Crit Compared to Economics; Aristotle, No

Tyler Cowen was kind enough to put a link to my GOAT Literary Critics post over at Marginal Revolution. It got a bit of commentary over there as well as sending some folks over here, so I thought I’d make some observations in response. First up, I’m copying a longish comment I posted over at Marginal Revolution (lightly edited). Then I’m going to explain why I didn’t nominate Aristotle for the list. I don’t know why ChatGPT failed to do so, though I do doubt that Aristotle would turn up if it perused its model for literary critics. I’m attaching an appendix that lists all the critics Wikipedia includes in its entry on literary criticism.

A comparison between lit crit and economics

I've been looking over the various comments on the GOAT literary critic, which I've since christened GOATLiC (Literary Critic), and I have a couple general observations. The first is that, as far as I can tell, when Tyler set out to write about the GOAT economists the population of thinkers from which he had to draw was pretty clear. That population does not include, for example, Michael Lewis. Michael Lewis is a journalist who writes excellent books about business. But he's not an economist. Nor did he include someone like Peter Drucker, who has lots of interesting things to say about business, but he's not an economist.

The term "literary critic" is not so clear cut. Edmund Wilson, for example, is a distinguished man of letters – notice the term I used. He is not an academic literary critic – notice that I used the term "academic." When academics think of academic disciplines, they think of intellectual activities that can provide for the cumulative development and growth of knowledge. The kind of intellectual work Edmund Wilson did was valuable, but it is not the sort of thing that lends itself to cumulative elaboration and development. The same is true for the work of Michael Lewis.

Now, anyone who reads a poem or a novel or sees a play, and so forth, may want to discuss it with their friends. They may also want to read something that deepens their knowledge of and appreciation of those texts. Edmund Wilson will do that. Edward Said perhaps, politics aside, not so much. So they may well then say, “A pox on you and your mother, Edward Said, with your obscure words and knotted syntax.” Readers of Michael Lewis, I suspect, are somewhat less likely to dismiss professional economicists because of the technical vocabulary, dry prose, and the math. They're willing enough to acknowledge that business journalism is one thing, and that's for them. Professional economics is something different, and not for them.

(And, yes, Tyler, I know a number of your GOATs wrote stuff that's readable by any reasonably well-educated person. After all, you ARE interested in economists as purveyors of ideas participating in general intellectual discourse. But they are writing as economists, more or less, not journalists. And, yes, what you write for Bloomberg is journalism, as is Krugman for the Times, but both of you are writing as economists. It’s complicated.)

For as long as I can remember, academic literary critics have been anxious about their status as academics and so have worked hard to figure just what it is they do that qualifies as the basis for academic inquiry. I don’t sense the economists have had the same problem. Physicists certainly haven’t. These anxieties are not quelled by the fact that much of the time of academic literary critics is devoted to teaching undergraduates how to write, which has absolutely nothing whatever to do with the study of literature. Are economists expected to teach undergraduates how to balance their checkbooks, file their taxes, and open savings accounts? I’m almost, but not quite, serious in that comparison.

And then there is the fact that literary study is tied to nationalism. The French study French literature, the Germans study German literature, the English study British literature, and Americans, until the last 50 or 60 years, Americans studied British literature as well. Seriously, when I was an undergraduate at Hopkins the English Department have 12-15 faculty, but only one of them taught American literature. That was a fairly common arrangement, at least at elite schools – see the highlighted remarks by J. Hillis Miller in this post (Miller should probably be a candidate for GOATLic). It took a while for American literature to be thought serious enough for inclusion in college and university courses.

Why Aristotle isn’t on the list & conceptual emergence in the late 18th century

Basically, too long ago. For one thing, if I gave Aristotle serious consideration, then there’s a long list of thinkers between him and the 18th century who must be considered as well (see the list in the appendix). Grouping things by centuries is standard practice in literary criticism, and it's a bit arbitrary with respect to the phenomena being grouped; centuries are artifacts of calendars, not causal forces in cultural change.

But there’s a more principled reason why the 18th century marks a reasonable demarcation zone. Malthus is the oldest economist on Cowen’s GOAT list. He was born in the 18th century and died in the 19th. Why didn’t Cowen have an older thinker of the list? I’m guessing that there aren’t any he could have picked. It really wasn’t until the late 18th century that people arrived at a modern conception of the future. Here’s how I opened an old post:

We’ve always thought about the future: Where’s the next meal coming from? How do we prepare for the festival? Will the harvest be good this year? What’ll I leave the grand kids? I have something different in mind. When did we start thinking about the future as a time and place when things would be different from they are now, when they would be better, and made so by effort we can undertake now? When did the future become that (kind of) place?

Now, Malthus wasn’t looking forward to a better time, he was fearing a time made worse by overpopulation. But the underlying conceptual requirement is the same.

Of course economics isn’t a discipline about the future, it’s not a species of science fiction (see my post, Adam Roberts on the future as imaginative territory). It’s about, well, the economy, about resources and their allocation, exchange of goods, investment and growth – how often has Cowen said that GROWTH is THE subject of economics? The future is essential to the conceptual foundations in which its concepts are grounded.

That particular conceptual change wasn’t the only thing going on at the time. Conceptual change was widespread and thorough-going. David Hays and I wrote about the general phenomenon in “The Evolution of Cognition” (1990) and Hays went on to write a book about technological evolution from that standpoint. I put Coleridge (first) on my list of GOAT critics because, as far as I know he was the first major thinker to absorb and interpret the emerging philosophical ideas in a way that allowed him to think about the literary mind in new way. I’ll have more to say about Coleridge in a later post, so I’ll let that statement stand for now.

When I think of the discipline of literary criticism, then, I think about a discipline that followed from the line of thinking that Coleridge initiated. Thinkers have been commenting on literary texts for as long as such texts have existed. That’s where we find Aristotle and a host of others (see the appendix). They are certainly players in the larger phenomenon of literary culture, but they aren’t literary critics in the modern sense of the term. Rather, they belong to the phenomenon that literary critics study.

Appendix: Wikipedia list of literary critics

Here’s a list of the literary critics included in the Wikipedia article on literary criticism. I’ve put numbers next to the period labels to indicate the number of critics in that period. I’ve highlighted Coleridge in yellow. I’ve done the same for a number of ‘critics’ listed in the 20th century. It seems to me that their inclusion in a list of literary critics is, shall we say, problematic – and they aren’t the only problematic ones on the list. I highlighted those particular figures because they were important in a movement toward the use of linguistics in literary study, a movement that had largely fizzled out by 1980. For the most part I’m talking about structuralism, which quickly gave way to post-structuralism.

Are LLMs approaching the limiting wall beyond which performance simply wanders around in the same general region?

Hot take on Google Gemini and GPT-4:

👉Google Gemini seems to have by many measures matched (or slightly exceeded) GPT-4, but not to have blown it away.

👉From a commercial standpoint GPT-4 is no longer unique. That’s a huge problem for OpenAI, especially post drama, when many… https://t.co/0i0yyTEtRY

— Gary Marcus (@GaryMarcus) December 6, 2023

Cade Metz and Nico Grant, Google Updates Bard Chatbot With ‘Gemini’ A.I. As It Chases ChatGPT, NYTimes, 2023. From the article:

Google has built three versions of Gemini with three different sets of skills. The largest, Ultra, is designed to tackle complex tasks and will debut next year. Pro, the mid-tier offering, will be rolled out to numerous Google services, starting Wednesday, with the Bard chatbot. Nano, the smallest version, will power some features on the Pixel 8 Pro smartphone, such as summarizing audio recordings and offering suggested text responses in WhatsApp starting Wednesday. [...]

With Gemini, Google has also trained the technology on digital images and sounds. It is what researchers call a “multimodal” system, meaning it can analyze and respond to both images and sounds. If you give it a math problem that includes lines, shapes and other images, for example, it can answer in much the way a high school student would.

That portion of the technology, however, will not be available to consumers until sometime next year. Google also acknowledged that like similar systems, Gemini is prone to mistakes. It can get facts wrong or even “hallucinate” — make stuff up.

Blue Giant

With English subtitles:

Kevin Nguyen, Jazz anime Blue Giant hits all the high notes, The Verge, Oct. 7, 2023.

The setup of Blue Giant is familiar anime territory: a young boy from a small town moves to Tokyo to pursue his dream. This boy, Dai, is a saxophone obsessive. He practices all day under a bridge, honking and warbling until he wears out his reed. Naturally, Dai wants to be the greatest jazz musician in the world.

For all his audacious ambition, Blue Giant is largely restrained. The movie focuses on the trio that forms the band — and even shifts its attention away from Dai at around the halfway mark. There’s also Sawabe, a savvy and smug pianist who knows how the club performance circuit operates. And then there’s Dai’s roommate, Tamada, a high school friend who improbably becomes the band’s rhythm section after trying the drums just once. They decide to form JASS, a name that, strangely, no one bats an eye at.

As JASS, they practice, they perform, they get better. Much of the film’s success hinges on the music being excellent — which it absolutely is. This isn’t the jukebox roulette of the Cowboy Bebop. Blue Giant has a legit soundtrack composed by Japanese jazz pianist Hiromi Uehara, who does a compelling homage to the American sax legends of the ’60s. It’s less the jazz of cool sophistication but one of bravado and squeaky high notes. Think of the muscular brass of Sonny Rollins’ Saxophone Colossus or John Coltrane’s Giant Steps (probably the record the film’s title most strongly evokes).

Not to mention the fact that Sonny Rollins famously practiced on the Williamsburg Bridge in 1959 and then came out with an album entitled The Bridge.

The stage dynamics of the trio are a thrill when they come together, as the three of them get into harmony. As the film progresses, these sequences get more ecstatic, more abstract. Swirls of color, brushes with the cosmic, an attempt to go sublime. I’m at a loss for words to describe how much fun these sequences are. I guess I’ll just go with… “hot” and “intense”?

There's more at the link. 

As for Hiromi, I've got some posts about her, and an article at 3 Quarks Daily: Sukiyaki and beyond: Hiromi Uehara, music, war and peace, Chick Corea, and others.

The sun will come up tomorrow

Big Think: The chaos inside OpenAI with Karen Hao [Excellent]

Journalist Karen Hao joins Big Think’s Editor-in-Chief, Robert Chapman-Smith, to discuss the recent events at OpenAI, including the ousting and reinstatement of CEO Sam Altman, as well as the ideological clashes regarding the development and release of powerful AI models like ChatGPT.

Karen Hao is an award-winning journalist covering artificial intelligence, currently contributing to The Atlantic. Previously, she was a foreign correspondent at The Wall Street Journal focused on AI, China tech & society, and a senior editor at MIT Technology Review, where she wrote about the latest AI research & its social impacts. She was also a fellow with the Harvard Technology and Public Purpose program, the MIT Knight Science Journalism program, and the Pulitzer Center’s AI Accountability network.

There's a useful set of resources linked to the YouTube site.

Animal ( carved wood), metal (smelted mineral), vegetable (bush)

Grounding neuroscience in behavioral changes using artificial neural networks

New paper "Grounding neuroscience in behavioral changes using artificial neural networks"
This opinion piece shows how focusing on changes in brain state that cause behavior changes helps pinpoint neural mechanisms. Specifically, I show how ANN models in neuro & AI help with this pic.twitter.com/X3ESe3dMBf

— Grace Lindsay (@neurograce) December 5, 2023

Open Source LLM-techs unite!

IBM & Meta are launching the AI Alliance to advance *open* & reliable AI.
The list of over 50 founding members from industry, government, and academia include AMD, Anyscale, CERN, Hugging Face, the Linux Foundation, NASA....https://t.co/NdcMb7GMXv

— Yann LeCun (@ylecun) December 5, 2023

Moby Dick speaks (has vowels)

Previously it was thought that mostly the number of clicks and their timing matters.

We identify two discrete patterns that we call the a-vowel and the i-vowel. We also observe clear trajectories in formant frequency that we call diphthongs. pic.twitter.com/2zPrKE193f

— Gašper Beguš (@begusgasper) December 5, 2023

A young woman is born

Welcome to the arena

Cage Match in the Valley 2: Google suits up, others join the fray

Karen Weise, Cade Metz, Nico Grant and Mike Isaac, Big Tech Muscles In: The 12 Months That Changed Silicon Valley Forever, NYTimes, Dec. 5, 2023:

What played out at Google was repeated at other tech giants after OpenAI released ChatGPT in late 2022. They all had technology in various stages of development that relied on neural networks — A.I. systems that recognized sounds, generated images and chatted like a human. That technology had been pioneered by Geoffrey Hinton, an academic who had worked briefly with Microsoft and was now at Google. But the tech companies had been slowed by fears of rogue chatbots, and economic and legal mayhem.

Once ChatGPT was unleashed, none of that mattered as much, according to interviews with more than 80 executives and researchers, as well as corporate documents and audio recordings. The instinct to be first or biggest or richest — or all three — took over. The leaders of Silicon Valley’s biggest companies set a new course and pulled their employees along with them.

Over 12 months, Silicon Valley was transformed. Turning artificial intelligence into actual products that individuals and companies could use became the priority. Worries about safety and whether machines would turn on their creators were not ignored, but they were shunted aside — at least for the moment.

The night before ChatMAS:

On Nov. 29, the night before the launch, Mr. Brockman hosted drinks for the team. He didn’t think ChatGPT would attract a lot of attention, he said. His prediction: “no more than one tweet thread with 5k likes.”

Mr. Brockman was wrong. On the morning of Nov. 30, Mr. Altman tweeted about OpenAI’s new product, and the company posted a jargon-heavy blog item. And then, ChatGPT took off. Almost immediately, sign-ups overwhelmed the company’s servers. Engineers rushed in and out of a messy space near the office kitchen, huddling over laptops to pull computing power from other projects. In five days, more than a million people had used ChatGPT. Within a few weeks, that number would top 100 million. Though nobody was quite sure why, it was a hit. Network news programs tried to explain how it worked. A late-night comedy show even used it to write (sort of funny) jokes.

There's much more at the link.

Sam Altman's Brain Chips

OpenAI signed $51M deal to buy ‘brain’ chips from Sam Altman portfolio firm

Sam Reynolds, Computerworld, Dec, 3, 2023.

A story that got left out of the corporate infighting that led to Sam Altman’s firing, then re-hiring as CEO of OpenAI, was about the firm’s relationship with a startup called Rain Neuromorphics, which is developing a neuromorphic processing unit (NPU) designed to replicate features of the human brain.

Rain says their brain-inspired NPUs could potentially offer 100 times more computing power and, for AI training purposes, deliver up to 10,000 times greater energy efficiency than the GPUs predominantly used by AI developers.

In theory, these NPUs could provide a big boost in processing power for portable “edge” devices such as smartphones or vehicle infotainment devices located far from a data center. Samsung, for instance, says that the Galaxy S24 – its next-generation flagship phone – will be AI-enhanced, but smartphones are limited in their processing abilities by their portable nature.

Altman’s conflict of interest

Before Sam Altman was fired and then rehired at OpenAI, last month, the company had signed a Letter of Intent to buy $51 million worth of Brain’s NPU chips, Wired reported. Complicating the issue further is the fact that this firm is one of Altman’s portfolio companies, bringing up a potential conflict of interest, with the CEO of OpenAI personally investing $1 million in the firm.

On X, VC Jason Calacanis pointed out that Letters of Intent – what OpenAI and Altman drafted to buy chips from Rain – are non-binding making the deal far from confirmed.

There's more at the link.

Zotz!

GOAT Literary Critics: Part Ia, What do you mean, literary critic?

Back in October of this year, the 23rd to be exact, Tyler Cowen posted a link to a book he’d just written and put on the web, along with connections to current LLMs (large language models) one could use to explore it. The book: GOAT: Who is the Greatest Economist of all Time and Why Does it Matter? I decided that reading it might help me fill a major gap in my education, economics. So I downloaded it, took at look at the end to see who wins (I’m not saying) and then put it aside ‘till next year, for I figured much of my time this year would be consumed with making sense of my work with ChatGPT – over 100 posts so far, not to mention the raw records of all my sessions with the Chatster.

Then, yesterday, when I was coming down the homestretch on my current essay for 3 Quarks Daily, on the current state of AI, I decided, of all things, to think about answering his book with a post on the GOAT (Greatest Of All Time) literary critics. So I jotted down some quick notes, and set them aside to complete my essay. Once I’d uploaded the essay I came back to the GOAT critics – a very Girardian notion, BTW, as one could give “GOAT” a Straussian reading as the sacrificial goat, the pharmakos, of ancient ritual, see also (from my dictionary) Greek tragōidia, apparently from tragos ‘goat’. To that end I put the question to ChatGPT – my gesture, one of them, toward the use of modern technology in this essay. I looked at its initial answer, prodded it some, and decided that I had enough for my purposes.

Here's the plan. First I’m going to present those initial thoughts, just as I jotted them down. Then I’m going to discuss those thoughts in relationship to ChatGPT’s response to my probing. I’ve attached that response as an appendix. That’s enough for this post. I may or may not get around to writing further posts. If I do here’s my most likely title: GOAT Literary Critics: Part 2: Frye, Penn Warren, and Coleridge. If I get that far, then maybe I’ll write this: GOAT Literary Critics: Part 3: Digital Humanists Map the Territory.

We’ll see.

Initial thoughts

The fact is that Tyler is obsessed with the question, Who’s on first? in a way that I am not, and that, to be honest, I find a bit suspect, if not foolish. I’m pretty sure he would understand my reservations about this kind of activity, even if he emphatically does not share them.

With that, why not? Whatever else it is, it’s an interesting exercise.

I really shouldn’t be doing this. I’m not qualified. The man who should do it is my teacher, Richard Macksey. But he died a few years ago.

And then there’s Adam Roberts.

Which is to say, whereas economists are clearly separate from the economy itself as a social structure and mechanism, the separation between literary criticism and literature itself is not so clear.

But if you asked me to set up a committee to look into the matter, here’s a list of people I’d ask them to consider.

Coleridge
I. A. Richards
Russian Formalists?
Bakhtin
Robert Penn Warren
George Lukacs
Leo Spitzer, Kemp Malone?
William Empson?
Trilling?
Dare I put Harold Bloom on the list?
Northrop Frye
Wayne Booth
Jacques Derrida
Lévi-Strauss
Rene Girard?

Finally, take a look at the Wikipedia article on literary criticism. The article itself isn’t much, but it has a long list of literary critics from ancient times through the present. 160 total, 79 in the twentieth century. That’s worth looking at. Take my small list, that large list, and begin connecting the dots: Who’s read who? Who’s been influenced by who? Perhaps as well, who met who, talked with them, or at least corresponded with them? That last will require you to add many names to the list, and perhaps hypothesize the existences of individuals who are lost to history. The result will be a large web. Somewhere in there you’re likely to find your literary GOATS.

And we’ve now got the computing technology to help us locate them.

Hopkins Handbook of Lit Crit will help you flesh out the list.

What do I think of what ChatGPT ‘thinks’?

As I was looking through ChatGPT’s responses it occurred to me that Cowen made a big deal out of specifying criteria by which he would be judging economists. I’d given some thought to that in making my notes, but hadn’t really listed any criteria in those notes, save for this sentence: “Which is to say, whereas economists are clearly separate from the economy itself as a social structure and mechanism, the separation between literary criticism and literature itself is not so clear.” That’s an issue, and I’m not sure just how it works out. I suppose that’s one of the things this exercise might reveal.

Let me say just a word by way of indicating why that’s an issue. I put Coleridge at the top of my list. He’s best known as a poet, for two poems when it comes down to that (“Kubla Khan,” and “Rhyme of the Ancient Mariner”), but he was a prodigious essayist and he wrote some excellent and influential books and essays about the nature of literature and literary experience and about specific works (Shakespeare in particular). There are others who straddle the line between criticism and writing as well.

That’s why I suggested Adam Roberts as someone who could make an interesting job of nominating GOAT literary critics. While he’s best known for his speculative fiction, he’s a PhD-trained literary scholar, a romanticist I believe, and has made a wider range of contributions to literary culture than just about anyone I can think of. While’s he’s a creative writer, he’s also done that most fundamental of critical tasks, prepared editions of texts (e.g. Coleridge’s Biographa Literaria), written literary biography (of H.G. Wells), literary history (of science fiction), written who knows what standard-issue academic literary criticism, written reviews and other pieces for venues like The Guardian, and, of all the crazy-ass things, collaborated with Google Translate to translate Finnegans Wake into Latin. Imagine that. You take a book many have claimed to read, but haven’t, and translate it into a language hardly anyone bothers to read. Finally, I know from our tenure together at the now-defunct group blog, The Valve, that he’s a congenial and helpful colleague. He’s far more qualified as lit crit GOAT whisperer than I am.

But I digress. This is about GOAT criteria. What are they? I don’t know, perhaps I’ll figure out. Anyhow, here’s the criteria Cowen set out for his economics (p. 8):

To qualify as “GOAT the greatest economist of all time,” I expect the following from a candidate. The economist must be original, of great historical import, serve as a creator and carrier of important ideas, have a hand in both theory and empirics, have a hand in both macro and micro, and be “not too wrong” on the substance of issues. Furthermore, the person also must be a pretty good economist! That is, if you sat down with the person and discussed economic issues, you would be in some way impressed.

It could be debated how much weight should be assigned to each category, but that is better considered through concrete comparisons than in the abstract. My inclination is to take each category as a kind of absolute requirement.

You see, there’s that idea that the economists have to score well on each category of judgment. What are the categories we’re to use in judging literary critics? In particular, what are the categories of activity? In economics we’re got “theory and empirics” and “macro and micro.” In what little I skimmed through in the last chapter he also talked about policy recommendations, which doesn’t seem to appear on that list. Should our GOAT literary critics be both creative writers and commentators. How do we weigh academic criticism against journalist criticism? What about the high-class and absolutely necessary scut-work of editing literary texts (known as textual criticism)? These are real issues, and I don’t know how to deal with them.

Having said that, let’s get on with it.

Why, in the first place, would I even consult ChatGPT on the matter? Pretty much for the same reason I consult it after I’ve watched a movie I’ve liked: calibration and provocation. I don’t consider myself qualified to list the GOAT literary critics. Yes, I’ve got a degree in literature, but long ago I went off the rails into cognitive science and literature and whatnot, so I just don’t have a good knowledge of a wide enough range of literary critics to make those calls. On the other hand, ChatGPT has “read” everything. I’m willing to rely on it to produce a kind of “least common denominator” answer to the question. Between the two of us we should be able to come up with some kind of interesting list to present to the GOAT Committee.

Let’s look at what ChatGPT did. It gave me ten critics: Harold Bloom, Northrop Frye, T.S. Eliot, Virginia Woolf, Roland Barthes, Edward Said, Susan Sontag, Michel Foucault, Cleanth Brooks, and Lionel Trilling. I recognize every one of them and have read at least something by every one of them except Cleanth Brooks and Lionel Trilling.

But there’s only one critic that’s on both ChatGPT’s list and my quick and dirty list: Northrop Frye. And Chat’s listed the crucial book: Anatomy of Criticism. That’s very interesting, in part because, as far as I can tell, hardly any academics read Frye anymore. I’ll return to Frye in a later essay.

Two of its suggestions, T.S. Eliot and Virginia Woolf, are known primarily as creative writers not critics. As far as I know Susan Sontag didn’t produce any academic writing at all (she didn’t have a PhD). I’m not sure that Foucault has done any literary criticism at all, or only in passing; but he certainly has influenced a lot of post-structuralist critics. Harold Bloom is controversial within the academy, which is perhaps neither here nor there, and his contribution to academic literary criticism will likely rest of the fate of The Anxiety of Influence (I read it years ago). But he’s written a lot for the general public, including big fat books on Hamlet and on The Western Canon, and he’s done a lot of editing. I believe Lionel Trilling is best known as an essayist and though he didn’t have a degree, he had a post at Columbia. Roland Barthes, a wide-ranging philosopher and semiotician, who wrote some very important work specifically about literature. Edward Said, an important critic for sure, but also an important public intellectual who was very active on behalf of Palestinians (he’s Palestinian himself). Cleanth Brooks, why not?

There’s the issue, though, what kind of intellectual production should we take into consideration when assessing literary critics for GOAThood? We might settle on academic literary criticism. That eliminates Sontag, Eliot, Woolf, and Trilling, though he held an academic post. Do we really want to do that? Bloom’s extra academic output may be more important that his academic output. It’s an issue.

On the street: Christmas and shoe repair

Experimental Google LLM for medical diagnosis

There has been work applying AI to medical decision-making, but all used other forms of AI, not LLMs. Now Google trained LLM for diagnoses.

In a trial, AI diagnosed 59% of hard cases. 🧑‍⚕️only got 33% right (🧑‍⚕️& AI together did well, but not as good as AI) https://t.co/ls3Tu3h8Hp pic.twitter.com/Mv3XzA0zAw

— Ethan Mollick (@emollick) December 4, 2023

Cage Match in the Valley 1: Many men enter, one man leaves.

Cade Metz, Karen Weise, Nico Grant and Mike Isaac, Ego, Fear and Money: How the A.I. Fuse Was Lit, December 3, 2023:

The question of whether artificial intelligence will elevate the world or destroy it — or at least inflict grave damage — has framed an ongoing debate among Silicon Valley founders, chatbot users, academics, legislators and regulators about whether the technology should be controlled or set free.

That debate has pitted some of the world’s richest men against one another: Mr. Musk, Mr. Page, Mark Zuckerberg of Meta, the tech investor Peter Thiel, Satya Nadella of Microsoft and Sam Altman of OpenAI. All have fought for a piece of the business — which one day could be worth trillions of dollars — and the power to shape it.

At the heart of this competition is a brain-stretching paradox. The people who say they are most worried about A.I. are among the most determined to create it and enjoy its riches. They have justified their ambition with their strong belief that they alone can keep A.I. from endangering Earth.

Mr. Musk and Mr. Page stopped speaking soon after the party that summer. A few weeks later, Mr. Musk dined with Mr. Altman, who was then running a tech incubator, and several researchers in a private room at the Rosewood hotel in Menlo Park, Calif., a favored deal-making spot close to the venture capital offices of Sand Hill Road.

That dinner led to the creation of a start-up called OpenAI later in the year. Backed by hundreds of millions of dollars from Mr. Musk and other funders, the lab promised to protect the world from Mr. Page’s vision.

Thanks to its ChatGPT chatbot, OpenAI has fundamentally changed the technology industry and has introduced the world to the risks and potential of artificial intelligence. OpenAI is valued at more than $80 billion, according to two people familiar with the company’s latest funding round, though Mr. Musk and Mr. Altman’s partnership didn’t make it. The two have since stopped speaking.

Altman goes on to make a Girardian point:

“There is disagreement, mistrust, egos,” Mr. Altman said. “The closer people are to being pointed in the same direction, the more contentious the disagreements are. You see this in sects and religious orders. There are bitter fights between the closest people.”

There's much more at the link: Eliezer Yudkowsky (matchmaker to the stars), Demis Hassibis, DeepMind, Geoffrey Hinton, Microsoft, Bill Gates, Mark Zukerberg, Yan LeCun, Sergi Brin, Eric Schmidt, Darlo Amodei, Anthropic, the names keep coming moths to light.

Sunrise Serenade

* * * * *

The OpenAI Affair and the Nature of Business in an Era of Intelligent Machines

I’ve been listening to a podcast by Ezra Klein with guests Kevin Roose and Casey Newton. Here’s one bit that I think is important.

Ezra Klein

Yeah, I first agree that clearly A.I. safety was not behind whatever disagreements Altman and the board had. I heard that from both sides of this. And I didn’t believe it, and I didn’t believe it, and I finally was convinced of it. I was like, you guys had to have had some disagreement here? It seems so fundamental.

But this is what I mean the governance is going worse. All the OpenAI people thought it was very important, and Sam Altman himself talked about its importance all the time, that they had this nonprofit board connected to this nonfinancial mission. The values of building A.I. that served humanity, that could fire Sam Altman at any time or even shut down the company fundamentally if they thought it was going awry in some way or another. And the moment that board tried to do that — now, I think they did not try to do that on very strong grounds — but the moment they tried to do that, it turned out they couldn’t. That the company could fundamentally reconstitute itself at Microsoft or that the board itself couldn’t withstand the pressure coming back. [...]

So maybe they have a stronger board that is better able to stand up to Altman. That is one argument I have heard.

On the other hand, those stronger board members do not hold the views on A.I. safety that the board members who left, like Helen Toner of Georgetown and Tasha McCauley from Rand, held. I mean, these are people who are going to be very interested in whether or not OpenAI is making money. I’m not saying they don’t care about other things too, but these are people who know how to run companies. [...] I mean, am I getting that story wrong to you?

Kevin Roose

No, I think that’s right. And it speaks to one of the most interesting and strangest things about this whole industry is that the people who started these companies were weird. And I say that with no normative judgment. But they made very weird decisions.

They thought A.I. was exciting and amazing. They wanted to build A.G.I. But they were also terrified of it, to the point that they developed these elaborate safeguards. I mean, in OpenAI’s case, they put this nonprofit board in charge of the for-profit subsidiary and gave, essentially, the nonprofit board the power to push a button and shut down the whole thing if they wanted to.

At Anthropic, one of these other A.I. companies, they are structured as a public benefit corporation. And they have their own version of a nonprofit board that is capable of essentially pushing the big red shut it all down button if things get too crazy. This is not how Silicon Valley typically structures itself.

Mark Zuckerberg was not in his Harvard dorm room building Facebook thinking if this thing becomes the most powerful communication platform in the history of technology, I will need to put in place these checks and balances to keep myself from becoming too powerful. But that was the kind of thing that the people who started OpenAI and Anthropic were thinking about.

And so I think what we’re seeing is that that kind of structure is bowing to the requirements of shareholder capitalism which says that if you do need all this money to run these companies, to train these models, you are going to have to make some concessions to the powers of the shareholder and of the money. And so I think that one of the big pieces of fallout from this OpenAI drama is just that OpenAI is going to be structured and run much more like a traditional tech company than this kind of holdover from this nonprofit board.

Casey Newton

And that is just a sad story. I truly wish that it had not worked out that way. I think one of the reasons why these companies were built in this way was because it just helped them attract better talent. I think that so many people working in A.I. are idealistic and civic-minded and do not want to create harmful things. And they’re also really optimistic about the power that good technology has. And so when those people say that as powerful and good as these things could be, it could also be really dangerous, I take them really seriously. And I want them to be empowered. I want them to be on company boards. And those folks have just lost so much ground over the past couple of weeks. And it is a truly tragic development, I think, in the development of this industry.

Look Up

The Method of Loci: With some brief remarks, including transformers and evaluating AIs

The method of loci is a memory technique used from ancient Greece and Rome up through the Middle Ages, into the Early Modern Era, and still in use today, though sporadically. Rather than explain it myself, or force you to read the Wikipedia entry, though you’re welcome to do so, I thought I’d like my good buddy, Schoolmaster Chat explain it to you. After that I’ll made a few observations about why it was necessary in the first place and conclude with a remark about transformers.

The Method of Loci

What can you tell me about the classical memory technique known as the method of loci?

The method of loci, also known as the memory palace technique, is a classical mnemonic device dating back to ancient Greece and Rome. It is a memory enhancement strategy that involves associating the information you want to remember with specific locations in a familiar space, such as a building or a room. The term "loci" is Latin for "places."

Here's a basic overview of how the method of loci works:

• 1. Choose a Familiar Setting: Select a place you are very familiar with, such as your house, a route you frequently walk, or a room you know well. This place will serve as your "memory palace."
  
• 2. Mentally Navigate the Space: Take a mental walk through this space and identify specific locations or landmarks. These could be furniture, doors, windows, or any distinct features within the environment.
  
• 3. Associate Information with Locations: Associate the information you want to remember with specific locations in your chosen space. Create vivid and memorable mental images that link the information to each location.
  
• 4. Mentally Walk Through the Space: When you need to recall the information, mentally walk through your memory palace. As you reach each location, the associated information should come to mind.

The effectiveness of the method of loci is based on the human brain's natural ability to remember spatial relationships and locations. By tying information to specific places, you create a mental map that makes it easier to retrieve the information later.

This technique has been used by ancient orators, scholars, and individuals with impressive memory feats. It's a powerful tool for memorizing lists, speeches, or any sequential information. While it may take some practice to master, many people find it to be a valuable and creative approach to memory enhancement.

Remarks, transformers, evaluating AIs

The obvious question – Why go to all that trouble? – has an obvious answer – Because remembering long collections of things is difficult. Why is it difficult? For my present purposes it’s sufficient to say: Because that’s just not how the brain works. Why that’s so, that’s a different question. Psychologists have spent a great deal of time investigating human memory, but we need not go into that.

I will not in passing, however, that we also find arithmetic calculation difficult. Oh, the concepts are trivial. That’s not where the difficulty lies. It’s in the practical application of those concepts in the act of calculation. Children are drilled for hours and hours over years and years in doing such calculation – though I would hope less so now that cheap calculations (not to mention computers) are readily available to do that work. Learn to do it, yes. It's an important basic skill, upon which important concepts can be piggy-backed. But endless drill, no longer necessary.

Which brings us to transformers. Getting them to be fluent with multidigit arithmetic has proven to be an interesting and challenging problem. It’s not that anyone wants them to do it in practical applications, rather, to put it bluntly, investigators are trying to see how human their capacities are. Well, the fact that multidigit arithmetic is difficult for transformers seems to make the very human, doesn’t it? After all, numeric calculation doesn’t even exist in pre-literate cultures. It’s not part of our biological endowment the way language is.

Now, to transformers. My first point is simple: How does the method of loci work? Once you’ve established your memory palace, and then stocked it with visual associations to your information, you execute this “program,” if you will, but starting at the beginning and walking through to the end. There’s no looping, no branching, just a single stream of recall. And that’s like the single stream of token generation that transformers produce.

My second point is that you have to set your memory palace up before you can use it. I supposed that’s Phase I of pretraining. Once that’s been done you can enter Phase II where you specialize the palace for a specific use by stocking it with images linked to the items you want to remember. I suppose it would be pushing things a bit to liken this to RLHF, but I’m not aiming for anything precise here. Just a crude correspondence.

One final and general observation. Pretty much everything I read about the capacities of LLMs vis-à-vis human capacities seems to assume that there is some one thing that is human mental capacity. No, there isn’t. There is a biological endowment, the nature of which is not clear. Over that, however, there is quite a bit of cultural modification and extension.

Well, guess what, just as arithmetic isn’t part of our native biological endowment, neither are detailed logical reasoning or (complex) planning. Those are cultural extensions and elaborations. That doesn’t mean that we an effective AI doesn’t have to capable to doing those things. Sure, for some applications they’re essential. But it’s foolish to routinely and unthinkingly include those capacities in the process of assessing whether or not an AI has reached “human capacity.” The whole field needs a much more sophisticated understanding of human behavior in order to begin making such judgments in a sensible way. Without such understanding The Spirit of St. Turing is flying blind.

More later.

A little whimsy with French fried onion snacks

Three Against Two the Tumbuka Way

I'm bumping this to the top on the principle that I need to do more posts about music. This doesn't actually add a post to the blog, but it brings one out of the past and into the present.

 

Western music is based on so-called duple rhythms, patterns of two or multiples of two. There are triple rhythms as well, the waltz for example, but they aren't as prominent. What Western rhythm rarely does is superimpose the two.

Not so in much of the third world, where three against two is a way of life. Here's a passage from Beethoven's Anvil (pp. 116-117) that describes a technique for learning three-against-two that is ascribed to an origin myth. Imagine that, a culture that makes rhythm part of it's origin myth.

* * * * *

The Tumbuka of Malawi, in southeastern Africa, have an origin myth that is coupled with a thigh slapping routine. The myth concerns Mupa, who discovered the rhythms used in vimbuza music, the music played for the trance dancing central to Tumbuka healing. Mupa discovered the rhythms while slapping his thighs. He began with a simple alternation—slap the right thigh with the right hand, left thigh with the left hand, in even alternating strokes—but that quickly grew boring. So he began figuring out more interesting ways to generate rhythms. I won’t recite the whole story—you can find it in Steven Friedson’s book, Dancing Prophets—but I will briefly describe the thigh-slapping routine that Mupa developed.

First, take a comfortable seat with your feet resting on the floor. Gently slap one thigh (say, the right thigh with the right hand) and then the other; do this repeatedly with an even rhythm at a comfortable tempo. Now, group your strokes into groups of three by slapping your knee on the first of each group of three. You will probably have to count to do this. You could use number names and say “one two three” but any three syllables will do. Just repeat the sequence over and over and slap your knee on the first syllable in the series. Not only is the physical gesture a little different from before, so is the sound. Notice that the initial stroke in your groups—set in bold type—will alternate between your right and left knees:

(1) R knee (2) L thigh (3) R thigh (1) L knee (2) R thigh (3) L thigh

A full cycle is thus six strokes long, divided into two groups marked by initial knee slaps. Emphasize the knee slaps so that they are just a little louder, thus strengthening the triple grouping. Practice this at a comfortable pace until you can do it with little or no thought. then you may want to pick up the pace and see how fast you can go.

Next we are going to superimpose THREE (two-stroke groupings) on the TWO groupings that consist of three strokes each. We will do this merely by thinking. Continue the same pattern but now concentrate on only one hand at a time, perhaps your right. I find it helps simply to look at the appropriate thigh. In the following representation the right strokes have been set in bold type while the initial strokes of the two groups of three have been set in italics:

(1) R knee (2) L thigh (3) R thigh (1) L knee (2) R thigh (3) L thigh

One could also choose to concentrate on the left-hand strokes:

(1) L knee (2) R thigh (3) L thigh (1) R Knee (2) L thigh (3) R thigh

Either way, get six beats in two different ways. When you use knee slaps as your marker, the six beats are divided into two groups or three beats each. When you use left or right side as your marker, they are divided into three groups of two beats each. Further, you can switch your concentration back and forth from the right-hand strokes to the left-hand ones.

If you are not already practiced in this sort of thing, you should be slow and deliberate. As you become comfortable, pick up the pace. You will reach a point where you no longer explicitly think in six, with overlays about how each stroke must be executed, and think instead in three/two.

The pattern of physical gestures you establish by practicing this exercise can then be applied to playing the ng’oma drum, the lead or master drum of Tumbuka music. The thigh stroke and knee stroke of the exercise become two different ways of striking the drum. But the basic pattern the Mupa left his people forms the core pattern of Tumbuka drumming; all other patterns derive from it.

* * * * *

See also: Three Against Two: Splitting the Mind in Two.