What’s been going on at New Savanna over the last year?

Here’s the action at New Savanna over the last year:

Notice the large increases in August and September of last year and in March through May of this year along with a more modest increase in December.

Now look at the all-time action:

There’s that big jump in 2017 and something similar this past year. I have no idea what’s going on in either case. In particular, in the current case, there doesn’t seem to be a big increase in interest in current posting, at least nothing I could see simply by eye-balling the hit rate. Whatever’s happening would seem to be more widely distributed. As I recall, it was like that for 2017 as well.

Here's a post from November 2017 where I note that older action, and one from January 2017, nearer the beginning of that older rise.

FWIW, here's the top 10 posts for the last year:

Friday Fotos: Irises, yellow & white, with cars

Globally, songs and instrumental melodies are slower and higher and use more stable pitches than speech

Yuto Ozaki, Adam Tierney, Peter Q. Pfordresher, et al. Globally, songs and instrumental melodies are slower and higher and use more stable pitches than speech: A Registered Report, Science Advances, 15 May 2024m Vol 10, Issue 20, DOI: 10.1126/sciadv.adm9797

Abstract: Both music and language are found in all known human societies, yet no studies have compared similarities and differences between song, speech, and instrumental music on a global scale. In this Registered Report, we analyzed two global datasets: (i) 300 annotated audio recordings representing matched sets of traditional songs, recited lyrics, conversational speech, and instrumental melodies from our 75 coauthors speaking 55 languages; and (ii) 418 previously published adult-directed song and speech recordings from 209 individuals speaking 16 languages. Of our six preregistered predictions, five were strongly supported: Relative to speech, songs use (i) higher pitch, (ii) slower temporal rate, and (iii) more stable pitches, while both songs and speech used similar (iv) pitch interval size and (v) timbral brightness. Exploratory analyses suggest that features vary along a “musi-linguistic” continuum when including instrumental melodies and recited lyrics. Our study provides strong empirical evidence of cross-cultural regularities in music and speech.

From the introduction:

Culturally relativistic hypotheses appear to be dominant among ethnomusicologists. For example, in a 13 January 2022 email to the International Council for Traditional Music email list entitled “What is song?,” International Council for Traditional Music Vice-President Don Niles requested definitions for “song” that might distinguish it from “speech” cross-culturally. Much debate ensued, but the closest to such a definition that appeared to emerge, was the following conclusion published by Savage et al. (25) based on a comparative analysis of 304 audio recordings of music from around the world:

“Although we found many statistical universals, absolute musical universals did not exist among the candidates we were able to test. The closest thing to an absolute universal was Lomax and Grauer’s (30) definition of a song as a vocalization using “discrete pitches or regular rhythmic patterns or both,” which applied to almost the entire sample, including instrumental music. However, three musical examples from Papua New Guinea containing combinations of friction blocks, swung slats, ribbon reeds, and moaning voices contained neither discrete pitches nor an isochronous beat. It should be noted that the editors of the Encyclopedia did not adopt a formal definition of music in choosing their selections. We thus assume that they followed the common practice in ethnomusicology of defining music as “humanly organized sound” (31) other than speech, with the distinction between speech and music being left to each culture’s emic (insider and subjective) conceptions, rather than being defined objectively by outsiders. Thus, our analyses suggest that there is no absolutely universal and objective definition of music but that Lomax and Grauer’s definition may offer a useful working definition to distinguish music from speech.”

However, the conclusion of Savage et al. (25) was based only on an analysis of music; thus, the contrast with speech is speculative and not based on comparative data. Some studies have identified differences between speech and song in specific languages, such as song being slower and higher-pitched (32–35). However, a lack of annotated cross-cultural recordings of matched speaking and singing has hampered attempts to establish cross-cultural relationships between speech and song (36). [...]

Our study overcomes these issues by creating a unique dataset of matched singing and speaking of diverse languages, with each recording manually segmented into acoustic units (e.g., syllables, notes, and phrases) by the coauthor who recorded it in their own first/heritage language.

Tulip, pale yellow with a wash of red

Unfrosted: Caught in a network of reference and allusion [Media Notes 119 F]

As counterintuitive as it might seem at first glance, Unfrosted is fundamentally an intellectual movie. It starts out that way and never strays far from it. What do I mean by “intellectual”? Rather than give you an answer right away, let’s take a look.

The movie opens with a little mystery. First a slide that says “NETFLIX Presents,” and then our mystery unfolds. These objects appear on the screen, one after the other, accompanied by quiet nostalgic music:

• a red bandana, opened and spread flat,
• a Woody Woodpecker comic book placed there by a child’s hands,
• three packets of Bazooka bubble gum,
• a rubber ball,
• a pack of baseball cards held together by a rubber band,
• GI Joe,
• a wallet of some kind,
• a penknife,
• a Slinky, and
• a red cloth that I don’t recognize, but it’s shaped a bit like a ping-pong paddle and has lettering on it.

I don’t know about you, but as I saw those little hands place those (precious) objects on the bandana I was identifying the objects and asking myself: What is going on?

We discover soon enough. A boy is running away from home and those are the things he’s taking with him. He goes into a diner, takes a seat next to some guy – why’d he do that, all the other seats were empty? – and orders: “Two Pop-Tarts, please. Leave the box.” [Did diners serve Pop-Tarts back then?]

As it so happens, the kid sat next to none other than Bob Cabana (Jerry Seinfeld), the executive who brought Pop-Tarts into the world. He starts telling the story: “Well, in the early ‘60s, the American morning was defined by milk and cereal.” And we’re off.

We’re back in the early 1960s. If you’re old enough – Seinfeld was born in 1954, I was born in 1947 – you might feel a warm glow of nostalgia wash over you as see those styles and objects from your childhood. If you’re somewhat younger, perhaps you’ve seen Mad Men or various movies set in that period; they’re all over the place. As these things unfold before you, may even nod to yourself, “I recognize that, and that, ah, so nice...” And there are more specific things. The car Cabana/Seinfeld was driving was styled like a Chevy “unsafe at any speed” Corvair, but I didn’t remember a Corvair station wagon. I eventually got around to checking on that and, yes, they made one, from 1961 to 1963, just in time for Pop-Tarts! I’m guessing that many/most in the audience may not have recognized the car (they only lasted until 1969), but I’m probably not the only one who was curious about the wagon.

Here's the point: The whole (freakin’) movie is like that. Those complicated plots within plots (Kellogg’s vs. Post, cereal vs. milk, Russia vs. America, mascots vs. management) seem to be mostly a device on which to hang an elaborate network of references and allusions, to things, events, and people, but also to movies and TV shows, not to mention the zillion actors who traipse through the movie. Whatever else you’re doing, you’re also marking those those things one by one, one after another, all the way through the film. Many of them are wry or funny, a few are hilarious, but most of them are just there. If the film were showing in theaters, they’d give you a small sheaf of papers listing all the various allusions so you could check them off, one by one, as they whizzed by. I don’t know what you’d do at the end, when all those product mascots, not just cereal mascots, stormed the Kellogg’s headquarters building. Were all of them real mascots for real products, or were some of them made up? I don’t know. Do I get a prize if I correctly identify every one of them? If so, I’d rather have diner Danish than Pop-Tarts.

And THAT’s what I mean when I say the movie is fundamentally intellectual. Whatever else is going on, you’re always thinking about this or that thing that’s just popped up on the screen.

Seinfeld has a bit about the “weight problem in this country.” It has these two lines:

The donut hole. The donut hole. Let’s stop right there. What a horrible little snack. If you want a donut, have a donut. Why are you eating the hole?

It’s such a freaky metaphysical concept to begin with. You can’t sell people holes. They… A hole, a hole does not exist. Words have meaning.

That’s intellectual. It’s also funny. A great deal of humor is about exploiting and exploding cracks in concepts. That’s fine for stand-up. But you can’t make a compelling feature-length movie out all those little cracks. That’s why Unfrosted feels unfocused. All those jokes and allusions make it impossible to build and sustain narrative momentum. This comes down to a case of not being able to see the forest for the trees.

More later.

The Pop-Tarts Portfolio

Industrial Strength Pop-Tarts

Pop-Tarts, the Monolith

Pop-Tarts, the Myth

Pop-Tarts in the Wild

Pop-Tarts, Sports Edition

Pop-Tarts: Hot! Hot! Hot!

86 Billion Neurons [& who knows how many synapses]

Bumping this to the top 1) on general principle, and 2) as a reminder.

 

Steven Mitten reviews The Human Advantage: A New Understanding of How Our Brain Became Remarkable by Suzana Herculano-Houzel, MIT Press, 256 pp., $29.95.

Energy requirements and cooking our food:

Our 86 billion neurons need so much energy that if we shared a way of life with other primates we couldn’t possibly survive: there would be insufficient hours in the day to feed our hungry brain. It needs 500 calories a day to function, which is 25 percent of what our entire body requires. That sounds like a lot, but a single cupful of glucose can fuel the brain for an entire day, with just over a teaspoon being required per hour. Nevertheless, the brains of almost all other vertebrates are responsible for a mere 10 percent of their overall metabolic needs. We evolved and learned a clever trick in our evolutionary past in order to find the time to feed our neuron-packed brains: we began to cook our food. By so doing, more energy could be extracted from the same quantity of plant stuffs or meat than from eating them raw.

Brain soup:

Brain soup was the method Herculano-Houzel devised to deal with the problem of a brain’s heterogeneity. Her procedure was to dissolve a brain of whatever species, with its millions or billions of cell membranes, in detergent to create a homogeneous distribution of free-floating cell nuclei. She could then sample the suspension, use a blue dye to stain the nuclei, count them up, and confidently extrapolate to the number of cells in the entirety of the brain, or whatever part of the brain she had begun with.

Those cells would be of three types—neurons, glial cells, and endothelial cells. Glial cells are crucial to the synaptic transmission of information across neurons, while endothelial cells form the walls of the capillaries that take oxygen and nutrients to the brain via the blood. Fortunately, the neurons could be distinguished by tagging them with a red-colored neuron-specific antibody, one that attaches to the NeuN protein within the cell nuclei. By counting the number that turned from blue to red once the antibody was added to the suspension, she could establish the proportion of the total cell count that was neurons

Human uniqueness:

Here are the numbers she found: the average human brain has 16 billion neurons in the cerebral cortex, 69 billion in the cerebellum, and slightly fewer than one billion in the rest of the brain. This fitted almost perfectly with the neuronal scaling rules derived for nonhuman primates: we have a perfectly normal primate brain, just the right number of neurons for the mass of our brain and also our body size.

That finding flew in the face of conventional wisdom, which argued that when correlations are drawn between body size and brain size for living primates (including the great apes), humans appear to have a brain size three times larger than expected. But Herculano-Houzel argues that it is the great apes, not humans, that are the exception. While the great apes also conform to the neuronal scaling rules—i.e., the average size of their neurons doesn’t increase exponentially as they gain more neurons—their brains are much smaller than should be expected for their body size.

The evolutionary story she tells by way of explanation is one of choosing between brain and brawn. Being restricted to eight hours of foraging a day, the ancestral great apes chose brawn (which, of course, means they underwent natural/sexual selection for a larger body size): the amount of energy that could be acquired was invested in building a bigger body rather than a bigger brain. At seventy-five kilograms a 30 billion–neuron brain was the maximum size that could be fueled. Ancestral Homo went a different way: it increased the energetic uptake from foraging by increased scavenging and hunting while maintaining a relatively small body size, enabling its brain to expand to an estimated 40 to 50 billion neurons for Homo habilis two million years ago. But that was the limit: there was no time left in the day and no other sources of food to exploit. Further expansion of the brain required securing more energy from the same type and quantity of foodstuffs. As from 1.5 million years ago that is just what our ancestors achieved by cooking their food.

Scaling rules:

But even though the human cerebral cortex constitutes 82 percent of the total brain mass, the largest when compared to all mammals, it was found to contain only 19 percent of the total number of neurons in the brain, the same percentage as in the guinea pig and capybara, and midway in the 15 to 25 percent range found in most mammals.

How can the human cerebral cortex have expanded so greatly in comparison to the rest of the brain while maintaining a proportion of neurons equivalent to that found in the cerebral cortex of other small-brained primates? Herculano-Houzel’s answer lies partly in the absolute number of neurons in the human cerebral cortex and partly in the fact that different scaling rules apply to the cerebral cortex and the cerebellum.

These rules are constant across all primates: when additional neurons are added to the brain, the cerebral cortex increases in mass at a much faster rate than does the cerebellum. This is because the cerebral cortex requires larger neurons than the cerebellum—neurons that have long-range connections of several centimeters to link different cortical areas; neurons in the cerebellum need to span no more than a few millimeters. As a result, the cerebral cortex becomes proportionally larger even though the ratio of cortical to cerebellar neurons remains the same. So with humans, the 16 billion neurons in the cerebral cortex result in its forming 82 percent of the total brain mass, despite the human brain’s remaining entirely typical for a primate with regard to the proportions of neurons in the cerebral cortex and in the cerebellum.

Some 1st class jazz: David Sanborn, Dizzie Gillespie, Diane Reeves, David Peaston, Onaje Allan Gumbs

At about 31:44 we have an absolutely INSANE version of "Stormy Monday" with Diane Reeves and David Peaston sharing vocal honors. I thought Sanborn and Gillespie were smokin' on "Tin Tin Deo," by Chano Pozo and Gil Fuller, but "Stormy Monday" lasted the whole damn week.

Assorted Flowers in a variety of colors and shapes

From Pop-Tarts to Jackie O’s and beyond [Media Notes 119 E]

One of the problems I’ve been having with Unfrosted is that it’s unfocused. As I mentioned in my original review, Seinfeld’s a miniaturist. His core artistic discipline is stand-up comedy, a discipline based on two-, three-, and five-, maybe six-minute bits. The Seinfeld Show was a half-hour sitcom, which means it had roughly twenty-two-and-a-half-minutes of show interrupted by commercial announcements. Those didn’t involve much of a plot. Similarly, Comedians in Cars Getting Coffee consisted of a conversation between Seinfeld and some comedian, which each show being ten to twenty minutes long – edited, I believe, from four or five hours of footage.

Unfrosted is a feature-length movie that runs an hour and twenty-five minutes, twenty-eight if you count the final song. While movies are constructed of shorter scenes, they have to have a set of events the builds and resolves over the course of the whole movie. Unfrosted doesn’t do that. Oh, we’ve got lots going on: Kellogg’s vs. Post, Big Milk vs. Big Cereal, Cuban sugar vs. Puerto-Rican sugar, Russia vs. America, mascots vs. Kellogg’s, and all the while NASA’s shooting for the moon. And somehow Pop-Tarts is supposed to hold all that together. It does that nominally, but not emotionally. There were two or three places in the movie where I felt like it should end. But it didn’t, too soon.

There’s a gag about Jackie O that illustrates one aspects of this diffuseness. The gag plays on the fact that there’s a very popular cereal called “Cheerios.” That presents the opportunity for a pun. But in order to get that pun into the movie we’ve got to get Jackie O into the movie. How do we do that? Jackie O is Jacqueline Onassis, formerly Jacqueline Kennedy, wife of President John F. Kennedy, who was in office between Jan, 1961 and Nov. 1963, when he was assassinated. Since Pop-Tarts were introduced about a year later, Kennedy died too early to play a role in this story. But that’s OK, since Unfrosted doesn’t pretend to be historically accurate about anything beyond costumes, cars, furniture, and the general physical setting.

Still, we need a pretext. What’s available. Ah, the Cuban Missile Crisis in October 1962, two years before. OK. The time frame’s close enough. But how do we make a connection between Pop-Tarts and the Cuban Missile Crisis? I can tell you easily enough. But I’m making some kind of point here, so I’m not going to do that, but if you really want to know, you can find out in the plot summary in the Wikipedia entry for the movie.

As a result of this turn of events, the top brass of Kellogg’s are summoned to the oval office at about 52 minutes into the film. President Kennedy enters the oval office: “Have you fellas ever considered calling a cereal Jackie O’s? Just a pitch but I think she’d get a big kick out of it.” There’s another second or two of chitchat and they get down to business. It seems that Post is now working with the Russians to secure a supply of sugar from Cuba. Kennedy: “The idea of our American children waking up in the morning to a commie breakfast pastry really burns my britches.” There’s more chit chat, this time a bit more serious, and Kennedy agrees to help with the situation: “I will instruct my brother Bobby to tighten the screws on those cow huggers.” If you’re wondering how THAT got in there, consult the Wikipedia entry. Out Kellogg’s honchos leave the meeting as the Doublemint Twins enter the oval office for some “executive privilege.” The scene ends at about 54 minutes.

The movie then moves back to other business. We’ve got a two-minute scene in a darkened bar between the Thurl Ravenscroft, the disgruntled actor who plays Tony the Tiger, and a representative of the milk mob. Then we have a minute with Walter Cronkite that’s split between on-air time and off. And now we have two major scenes on the main plot-line. First Kellogg’s conducts a test of their new pastry that is a parody of a rocket launch. The pastry is good, but the “taste pilot” died in a freak accident. Given the general NASA motif I assume this is a reference to the death of three astronauts when the crew cabin of Apollo 1 broke out in flames on the launchpad in 1967. That takes a two and a half minutes or so. Then we segue the funeral for the deceased. That runs for roughly three minutes.

We’re now eight or nine minutes beyond the end to the oval office scene where Kennedy make the Jackie O’s suggestion. That’s not much time at all, but it seems longer than it is because our minds have had to move through four distinctly different settings: a darked bar, a TV program, a pastry launch, and a funeral. That’s a lot of information packed into a short time.

And now we’re on the street in Battle Creek (at roughly 1:03:00). Our three Kellogg’s honchos, Edsel Kellogg (President), Bob Cabana (Head of Development?), and Donna “Stan” Stankowski (Chief Scientist?), are driving along when then see a crowd gathered in front of the local TV store where everyone is watching the news. The pull over and join the crowd President Kennedy is talking to the nation about a Russian ship near Cuba: “We believe it’s illegal Cuban sugar meant to destabilize a balanced breakfast.”

Kellogg: “I tell you the wife’s a knockout.”
Cabana: “Yeah”
Stankowski: “I got the boys working on the Jackie O thing.”
Kellogg: “Her last name’s ‘Kennedy.’ What does that even mean?”
Stankowski: “No, the shape of the cereal’s an O. Cheerios, Oreos, Jackie-O’s.”
Woman in the crowd: “Shussh! He’s talking about nuclear war, you idiots.”

And there you have it. Ten minutes after the seed was planted in the oval office, the Jackie O gag is brought to fruition. And they had to have lines in the script that told us what the joke was about.

But to what point? What does that gag have to do with Pop-Tarts? Well, Cheerios is a cereal, Jackie-0’s would be a cereal. And Pop-Tarts is being positioned as a cereal substitute. That’s the connection.

I wonder just how, in the process of developing the script, that gag got created. Perhaps the Cuban Missile Crisis bit came first, as a way of amplifying the rivalry between Kellogg’s and Post, and the Jackie O bit was tacked on because, why not? It’s such a little thing. But I could almost imagine that it went the other way. One of the writers, Seinfeld or one of his partners (Spike Feresten, Barry Marder, and Andy Robin) came up with the idea of a cereal connection between “Jackie O”, Cheerios, and Pop-Tarts, and the whole Cuban sugar connection was then worked into the plot so the Jackie O gag can be used. It really doesn’t matter.

What does matter is that Unfrosted is full of gags and references that are locally funny and clever, but that don’t build a compelling overall story-arc. Perhaps the largest such intrusion is the climactic scene where the mascots go on a rampage against the Kellogg’s building while the brass are inside overseeing the certification of Pop-Tarts. Tony the Tiger leads the charge and he’s costumed with Viking horns like the so-called QAnon shaman. Many of the shots in the scene parody shots from video footage of the attack on the United States Capitol on January 6, 2021.

Why? That event is six decades after the launching of Pop-Tarts and has absolutely nothing to do with it. Let the mascots attack the Kellogg’s building, but why distract the audience by forcing us to recall the various video shots they saw? “I saw that, and that one, I recognize that shot...” And so forth. If it’s intended to be a parody of January 6, well, it doesn’t play that way. Nor does it somehow inform or amplify the underlying story of corporate competition. Rather, it’s just management vs. labor that’s been tacked onto that story. To what end?

The Marx Brothers didn’t have me asking such questions of Duck Soup. If you want movies from the 1960s, what about Dr. Strangelove (1964) or Putney Swope (1969)? True, they’ve very different from Unfrosted. But both are comedies, though Strangelove is rather grim. And they are tightly focused. And gags? What about Slim Pickens waving his cowboy hat and riding the bomb? And if you want a comedy that makes effective use of children, how about Galaxy Quest from 1999? If Unfrosted has just turned the Butchie&Cathy line up to eleven, bringing them more closely into the action like the (somewhat older) kids in Galaxy Quest, that might have helped bring more coherence to Unfrosted. 

More later.

Just Coffee

A look at A.I. Hype

Julia Angwin, Will A.I. Ever Live Up to Its Hype? NYTimes, May 15, 2024. 

...some of A.I.’s greatest accomplishments seem inflated. Some of you may remember that the A.I. model ChatGPT-4 aced the uniform bar exam a year ago. Turns out that it scored in the 48th percentile, not the 90th, as claimed by OpenAI, according to a re-examination by the M.I.T. researcher Eric Martínez. Or what about Google’s claim that it used A.I. to discover more than two million new chemical compounds? A re-examination by experimental materials chemists at the University of California, Santa Barbara, found “scant evidence for compounds that fulfill the trifecta of novelty, credibility and utility.”

Meanwhile, researchers in many fields have found that A.I. often struggles to answer even simple questions, whether about the law, medicine or voter information. Researchers have even found that A.I. does not always improve the quality of computer programming, the task it is supposed to excel at. [...]

And consider for a moment the possibility that perhaps A.I. isn’t going to get that much better anytime soon. After all, the A.I. companies are running out of new data on which to train their models, and they are running out of energy to fuel their power-hungry A.I. machines. Meanwhile, authors and news organizations (including The New York Times) are contesting the legality of having their data ingested into the A.I. models without their consent, which could end up forcing quality data to be withdrawn from the models.

There's more at the link.

Pop-Tarts, they aren’t that good [Media Notes 119 D]

I’ve been working away on Unfrosted and have been having trouble bringing a long article to a close. I came across an interesting review by Owen Gleiberman in Variety. Here’s the third and fourth paragraphs:

As a kid growing up in the late ’60s and ’70s, I confess that I never understood Pop-Tarts. My family would buy them, and every so often I would put one in the toaster, wanting it to be a tasty treat. Such is the power of advertising that I always thought it was my fault that I found Pop-Tarts to be…just okay. Twinkies, by contrast, were junky but succulent. And even good old dry cereal, when you were in the mood for it, was pretty great — the delicate crunch of Rice Krispies, the sweet-milk-bath rapture of Sugar Frosted Flakes. To me, though, Pop-Tarts never lived up to their billing. They were bland when untoasted (though a lot of folks ate them that way). Once you toasted them, the hot fruit filling had a soothing tasty tang, but the rectangular pastry was still cardboard pie crust. It wasn’t awful, but it’s not like biting into it gave you a rush of joy. Prefab and a little dull, the Pop-Tart was a “product of the future” that seemed stuck in the past, like astronaut food.

I bring all this up because “Unfrosted” treats the origin story of the Pop-Tart with such a derisive, backhand flippancy that it’s not at all clear what Jerry and his team of screenwriter-producers [...] actually think of the Pop-Tart. Is the movie a goof because they’re making fun of what a mediocre product it was? Perhaps. Yet if the memory of Pop-Tarts actually strikes a chord of Proustian reverence in Jerry — if it’s his madeleine stuffed with fake-fruit chemicals — then why make such a misanthropic satire of it?

YES!

I’m a bit older than Seinfeld. I remember the Cuban missile crisis. And I remember Pop-Tarts.

I don’t think I ever ate one. We certainly didn’t keep them around the house. My mother prided herself on her pastry skills too much to put up with Pop-Tarts. Of course, making good pastry takes time, so we didn’t have it often, but her Danish, from a recipe she got from my grandmother...to die for! Anyhow, I actually went out and bought a package of Pop-Tarts, mostly do I could get a photo or two. They’re not that good. Putting some jelly on toast isn’t hard – ordinary jelly and standard supermarket white bread, nothing artisanal – and it tastes better.

So I’m with Gleiberman on Pop-Tarts in and of themselves. And I agree with him on that second paragraph as well. It’s not clear what Seinfeld’s attitude toward them actually is. The whole thing is a bit scattered and unfocused. I think Gleiberman’s reference to Proust is spot-on. As my friend David Porush just remarked (on the phone), this is very much an act of memory. Seinfeld is remembering the old days, but framing them in a series of ironic comic miniatures to keep them from being too seductive.

Brain encoding models that can transfer across language and vision

Jerry Tang, Meng Du, Vy Vo, VASUDEV LAL, Alexander Huth, Brain encoding models based on multimodal transformers can transfer across language and vision, Advances in Neural Information Processing Systems 36 (NeurIPS 2023) Main Conference Track

Abstract: Encoding models have been used to assess how the human brain represents concepts in language and vision. While language and vision rely on similar concept representations, current encoding models are typically trained and tested on brain responses to each modality in isolation. Recent advances in multimodal pretraining have produced transformers that can extract aligned representations of concepts in language and vision. In this work, we used representations from multimodal transformers to train encoding models that can transfer across fMRI responses to stories and movies. We found that encoding models trained on brain responses to one modality can successfully predict brain responses to the other modality, particularly in cortical regions that represent conceptual meaning. Further analysis of these encoding models revealed shared semantic dimensions that underlie concept representations in language and vision. Comparing encoding models trained using representations from multimodal and unimodal transformers, we found that multimodal transformers learn more aligned representations of concepts in language and vision. Our results demonstrate how multimodal transformers can provide insights into the brain’s capacity for multimodal processing.

Street lamp and lots of other stuff around it

The Matrix [Media Notes 120]

I saw it when it came out, and perhaps I saw in again on some streaming service, of maybe I got it on Netflix back in the DVD days. Don’t know what I thought of it back then. But now (on Netflix)...YIKES!!

I’m thinking that The Matrix is mostly a pretext for a Sci-Fi urban martial arts computational tentacle-porn pseudo-mystical mashup. Really? I’m sure I didn’t take it seriously, for whatever meaning of “seriously” makes sense in this context. It’s all about the action and the tidbits of mystical-mumbo jumbo about belief and self. And the vibe, it’s about the vibe.

And the mystical power of true love? It all comes down to a woman’s kiss? Trinity kisses Neo, All Better. The Oracle said so. He is the one! And soaring brass! Gimme a break!

* * * * *

On action, see my post, John Wick 1, 2, 3 [Media Notes 116], for some discussion.

* * * * *

This one film gave birth to three more features, one of which had a bit role for Cornel West, and a whole lot more. I’ve watched the first two sequels, but not the fourth. Apparently, there’s a fifth in the works.

Here’s a paragraph from the Wikipedia article on the franchise:

The series features a cyberpunk story of the technological fall of humanity, in which the creation of artificial intelligence led the way to a race of powerful and self-aware machines that imprisoned humans in a neural interactive simulation — the Matrix — to be farmed as a power source. Occasionally, some of the prisoners manage to break free from the system and considered a threat, become pursued by the artificial intelligence both inside and outside of it. The films focus on the plight of Neo (Keanu Reeves), Trinity (Carrie-Anne Moss), and Morpheus (Laurence Fishburne and Yahya Abdul-Mateen II) trying to free humanity from the system while pursued by its guardians, such as Agent Smith (Hugo Weaving, Abdul-Mateen II, and Jonathan Groff). The story incorporates references to numerous norms, particularly philosophical, religious, and spiritual ideas, among others the dilemma of choice vs. control, the brain in a vat thought experiment, messianism, and the concepts of interdependency and love. Influences include the principles of mythology, anime, and Hong Kong action films (particularly "heroic bloodshed" and martial arts movies). The film series is notable for its use of heavily choreographed action sequences and "bullet time" slow motion effects, which revolutionized action films to come.

Will this multimedia franchise go down in history as the end of civilization as we know it? The hinge of history heralding the Singularity (properly understood?). Who knows? (Who cares?)