Monday, May 10, 2021

Improving prediction accuracy and interpretability of deep neural netowrk models

Curious about group theory? Visual is the way to go

Don't know the books, or group theory for that matter. But Strogatz is sharp so I take his advice seriously.

Blue green yellow orange red

Paganini Caprice No. 24 performed on various instruments [what a glorious thing, the human body]

It’s not something I set out to investigate, Caprice No. 24 for violin by Niccolò Paganini. Paganini was an Italian violinist whose life straddled the 18th and 19th centuries. His skill was so formidable that some believed he had made a pact with the devil. Others believed it was his mother who sold him out.

Think about that. Why would people believe such a thing? Yeah, they were superstitious, but still, why believe that Paganini’s skill must have been beyond the merely human?

Paganini composed 24 caprices, as they were called, for unaccompanied solo violin. The 24th one, the last, is the best known. From Wikipedia:

Caprice No. 24 in A minor is the final caprice of Niccolò Paganini's 24 Caprices, and a famous work for solo violin. The caprice, in the key of A minor, consists of a theme, 11 variations, and a finale. His 24 Caprices were probably composed in 1807, while he was in the service of the Baciocchi court.

It is widely considered one of the most difficult pieces ever written for the solo violin. It requires many highly advanced techniques such as parallel octaves and rapid shifting covering many intervals, extremely fast scales and arpeggios including minor scales, left hand pizzicato, high positions, and quick string crossings. Also, there are many double stops, including thirds and tenths.

There you have it, “one of the most difficult pieces ever written.” The point of such pieces is that they ARE difficult, but a virtuoso makes playing them seem effortless.

Well, if you are talented – whatever that is – and you’ve spent years and years practicing, as Paganini certainly did, then, yes, such pieces are effortless. Almost. They are effortless – if that’s the right word, and I’m not sure that it is – if and only if you practice every day. Otherwise your skills deteriorate and at some point even the most heroic effort won’t get you through the piece.

Simply making it through the piece, that’s one thing. Making it appear easy, that’s another. THAT’s what gets you rewarded. As for music, it seems to me that the musical value of these caprices, these bonbons as they are sometimes called, is slight. They are not even remotely comparable to a major concerto or sonata or, for that matter, a Charlie Parker tune (see my recent post, Kids and Music 5: Kwak DaKyung, Jazz Trumpeter [Born to Groove]). They were created for virtuoso display and simply do not have the musical substance needed to subordinate virtuosity to musical expressiveness.

Fun? Yes. Substantial? Alas, no.

Cheap thrills, that’s what they are. Nothing wrong with that, but accept them for what they are. Have fun working on them and performing them.

Ratcheting up technical capacity

This post is about the influence this piece, and (many) others like it, have had on subsequent instrumental technique. Once Paganini had composed and performed these pieces other violinists figured out how to perform them and thus the overall level of technical accomplishment rose. Violinists got better and better.

What particularly interests me, however, is what happened when other instrumentalists got ahold of the piece – and others like it.

All instruments have their characteristic affordances, the opportunities they offer the player for musical realization and expression. Some instruments can play only one note at a time; others can play several. Some instruments don’t sound strongly defined pitches, many but by no means all percussion instruments. And some instruments are more flexible than others; that is to say, it is relatively easy to move from note to note over wide intervals on the instrument and to do so rapidly.

The violin is very flexible and has a wide range, though not at wide as the piano. While it is primarily a single note instrument, it is possible to play two, three, and even four notes at a time on it. That’s one of the things Paganini did, extend the capacity of violin technique to play two, three, and four notes.

What happens when instruments with quite different affordances (attempt to) play music originally created to extend violin technique? That’s what we’re going to look at.

Violin – Jascha Heifetz

Heifetz was one of the great violinists of the previous century, some say he was the best since Paganini. I’m in no position judge violin technique – I’ve never even held a violin, much less attempted to play one – so I’m in no position to second guess such judgments. I’m content to take them at face value.

Here he is performing Caprice No. 24. Notice that’s he’s accompanied by piano. I don’t know who wrote that part, but it wasn’t Paganini. Since we’re looking at instrumental technique, watch carefully. Fortunately we even get a close-up of his left hand as it performs a very difficult pizzicato passage (plucking the strings) with the left hand starting at roughly 3:51.


Marimba – Naoko Takada

The marimba is a very different kind of instrument. Look at the size. Takada has to move her whole body from side to side to play it. She strikes the keys with whole-arm movements. But she doesn’t need the intricate finger control necessary for the violinist’s left hand. Think of the problems involved in precisely targeted large movements of the sticks, especially at soft volume levels. Note, however, that she can play several notes simultaneously.

Great playing, though it seems to me that some passages don’t quite flow together.


Trombone – Peter Steiner

I do know something about brass playing. I’m a semi-virtuoso trumpet player and I’ve also played the valve trombone. The basic problem in playing a brass instrument is creating and sustaining a sound. That’s trivial for violin, though controlling the sound is not. Creating sounds on a marimba is easy as well, though there is no way to sustain a pitch other than using a tremolo (which, I would imagine, is tricky).

Brass instruments are not flexible. The further two notes are apart, the more difficult it is to move from one to the other. Rapid wide interval jumps (e.g. 1:54 ff.) are very difficult. The trombone slide adds another dimension of difficulty. While it allows you to get any pitch you want, that’s not what you want here. You want to get exactly the right pitches. This also makes rapid runs very difficult to execute as the pitches tend to run together, forcing the trombonist to use a technique called double-tonguing. Don’t worry about just what it is, it IS difficult.

Embouchure control (lips and jaw) is critical, as is breath control. That, and not slide movement, is what makes wide-interval jumps so difficult. This, obviously, is quite different from both violin and marimba.

This is heroic playing, working ferociously against the inherent tendencies of the instrument. But also, I fear, a bit ponderous for a cappricio.


Trumpet – Alison Balsom

Almost everything I said about the trombone applies to the trumpet. Difficult to sound and consequently inflexible. However, the trumpet is played with valves, not a slide. Thus instead of large movements of the fight arm and hand you have intricate movements of the fingers on the right hand. Because the trumpet is a smaller instrument than the trombone the resulting performance is, it seems to me, more successful, not so ponderous. 

Balsom is a superb soloist.

But why is every shot in very soft focus and diffused lighting?


Brass Quintet – Vladimir Mezentsev, French Horn Soloist

Now we have four different brass instruments, two trumpets, a trombone, a tuba, and a French horn. More heroic playing all around.

The tuba is even more ponderous than the trombone (I assure you there are tuba players who can execute very rapid, albeit heavy, renditions of “Flight of the Bumblebee”). As for the French horn, for technical reasons we need not go into, it is an extraordinarily treacherous instrument to wrangle. Mezentsev is insane to be playing this.

But really, guys, come on. We know you’re fabulous players but did you have to, really? Perhaps that’s why they threw in a bit of Rachmaninoff. Starting at 4:33 they toss in a passage from his “Rhapsody on a Theme of Paganini” (from Caprice 24) as though to say their tongues were in their cheeks the whole time.

That boomy hall didn’t do them any favors.


Electric Guitar – The Commander-In-Chief & Craig Ogden

Now we’re getting somewhere. Irony, that’s the way to play it. Dig the Commander's military-themed threads, not to mention the high-heels. Adds a bit of an S&M vibe to the performance, which is more than appropriate for this piece. She got her start in metal.

The Commander-in-Chief performs the caprice as a duet. Always cool, plus it gives us insight into the music. Paganini’s piece is really “Dueling Banjos” for solo fiddle. Dig the unison at 2:53 and after. That’s skill. There’s some very interesting interplay further on. This is a wonderful performance.

I leave it as an exercise for the reader to ponder the affordances of the guitar, both acoustic and electric.


Gypsy Style – Florian Cristea and Friends


Now this is fun. This is music. They don’t perform Paganini’s caprice in the most direct sense of the word. The fiddler starts with the caprice and then the group uses it here and there and however as the basis for the rest of the performance. Their humor, virtuosity, and musicality all but redeems Paganini’s piece from all the excesses visited upon it over the years.

Psst! You know what? I'd love to hear the Commander-in-Chief jam with these guys.

Also, if the performance makes you laugh for joy, they're doing it right.

Sunday, May 9, 2021

Branches, twigs, and leaves

Thoughts about recent work, where it’s going [philosophy new, GPT-3 & AI]

Two recent lines of work have gotten me thinking, my reconsideration of John Horgan’s The End of Science and my dialog with GPT-3 about a Jerry Seinfeld bit. Let’s start with the end of science and work our way to Seinfeld.

Horgan set himself a difficult task in arguing for the end of science, for he is, in effect, predicting the future. And that’s difficult. When I wrote about his argument 25 years ago I took him to be making a philosophical claim, though I didn’t think about it in those terms, and countered with a claim of my own. Twenty-five years have passed and Horgan is still claiming that science is in trouble and I’m still agreeing with him, but also arguing that there’s a difference between being in trouble and being at an end.

I’ve done a lot of work in the interim and that’s what I’ve been thinking about. There’s my book on music, Beethoven’s Anvil, my return to neuroscience through Walter Freeman’s neurodynamics, a great deal of descriptive work in literature and film, along with my methodological and quasi-theoretical piece on literary form (Literary Morphology: Nine Propositions in a Naturalist Theory of Form), my excursions into computational criticism, and my encounter with Latour, culminating with Living with Abundance in a Pluralist Cosmos: Some Metaphysical Sketches, which sure looks like an outline of a philosophical system.

And that’s what I’ve been thinking about: Am I sitting on the scattered pieces of a philosophical system? If so, what would it take to pull it together? In a way it begins with the little piece David Hays an I wrote on complexity and natural selection, A Note on Why Natural Selection Leads to Complexity (1990), for that provides a justification for why, over the long course, living systems become more complex, cultural systems to. So that links our earlier paper on the brain (Principles and Development of Natural Intelligence, 1988), in which we located functional principles at major junctures in vertebrate phylogeny, with our more recent work on cultural evolution (e.g. The Evolution of Cognition, 1990). What would it take to spell out those connections and then to link the whole contraption to my more work on pluralism?

I note that, while Hays and I talked about abundance, under the rubric of fecundity, in private conversation, we hadn’t published about it at the time he’d died. But I introduced it in my original treatment of Horgan’s book, Pursued by Knowledge in a Fecund Universe (1997), where I also introduced the idea of implementation, taken from computing. The general idea is that the routines of a higher realm (of biological behavior, of cultural practices) are implemented in the mechanisms of the pre-existing and lower realms. The idea is that evolution in complexity proceeds by elaborating variations and extensions of existing mechanism until, somehow, a phase shift happens so as to reorganize the system in a new way. In that reorganized system new mechanisms emerge to govern older systems, using them as vehicles to implement its new goals and behaviors. My immediate objective is not, however, to explain how such phase changes occur – I’ll happily leave that to others, others with technical skills I lack – but only to (provisionally) recast our work on the principles of natural intelligence and those of cognitive evolution in culture in the same terms. And then to take that and see how it resonates with the Realms of Abundance notion from the pluralism paper.

And that brings me to my recent interaction with GPT-3 over a Seinfeld bit. I don’t myself have access to GPT-3, but I know someone who does, Phil Mohun. And so one day last week we spent 15 or 20 minutes chatting back and forth on Slack while Phil quizzed GPT-3. It was a lot of fun. When you read the post you’ll see that it took awhile to get a sense of how GPT-3 ‘understood’ the joke’s punch line.

We need to free these systems from our narcissistic investment in them.

THAT’s what’s interesting. I don’t really care whether GPT-3 understands language in the way you and I do – it doesn’t – nor whether or not it’s one Big Giant Step toward AGI – I don’t find the idea of AGI (artificial general intelligence) very interesting, or even coherent. But it does seem to me the GPT-3 is ¬one Big Giant Step toward SOMETHING. But just what that is, I don’t know, nor do I think anyone else does either.

At the moment I don’t even believe we’ve got the terms and concepts we need to think about that. If GPT-3 doesn’t understand Seinfeld’s bit in some way, if it isn’t thinking about its knowledge when responding to the questions Phil and I put to it, then what IS it doing? What words do we use? “Process” seems too generic. It’s a cop-out, useless. If you will, we need a phenomenology of machine intelligence that's different from our phenomenology of ourselves.

As long as we insist on anthropomorphizing these systems we're deceiving ourselves about them. They really are radically new and not merely halting attempts to re-create ourselves in machine form. We need to free these systems from our narcissistic investment in them.

I keep thinking about a comment Yann LeCun made about end-to-end systems (GPT-3 is such a system) in an article by Kenneth Church and Mark Liberman: “If you can train the entire thing end to end—that means the system learns its own features. You don’t have to engineer the features anymore, you know, they just emerge from the learning process.” I believe that’s analogous to the argument that Sydney Lamb has made about meaning in networks, that the meaning of a node is a function of its position in the network (I discuss this in, for example, a recent post, Minds are built from the inside [evolution, development]).

I’m tempted to say that the meaning of a node in a relational network is a function of its relationship to both the input and the output side of the network. What does the mean in the case of GPT-3? Does it make any sense to say that its internal structures ‘gain’ meaning in the process of interacting with humans? It seems to me that such systems do have something in common with the human brain, something deep and fundamental, that they ‘construct’ themselves. That’s something we must understand, and we’re not going to create that understanding as long as we’re worried about whether or not it’s really got a human-like mind. It doesn’t. But it has something.

In my working paper, GPT-3: Waterloo or Rubicon? Here be Dragons, Working Paper, Version 2, I talk about why GPT-3 is always going to have trouble with so-called common-sense reasoning. Someone with mathematical competence needs to take that argument and develop it. Common-sense knowledge is ‘close’ to our physical interaction with the world. Ultimately it depends on that knowledge. But GPT-3, and similar systems, are trained on text only, so they simply don’t have access to the sensorimotor foundation on which common-sense reasoning is based. That places severe limitations on what they can do, making flexible and robust common sense reasoning always just beyond reach.

How do you make up that deficit? Sure, you can train a system on images and sounds, touches, and smells, but it also has to move about in the world. You can see where this is going, can’t you? We have to create an android that starts out the capabilities of human infant and then grows, matures, and learns. How do we create an android that grows like a human? At the moment I’m thinking that if we want to create an artificial creature with all the capabilities of a human being, we’re going to have to create an artificial human being. We haven’t got the foggiest idea of how to do that. But what’s the point?

More later.

Saturday, May 8, 2021

Reuben and fries

Hey, kids! Let's start our own country! [all things virtual]

At, How to Start a New Country, 9 April 2021.


We want to be able to peacefully start a new country for the same reason we want a bare plot of earth, a blank sheet of paper, an empty text buffer, a fresh startup, or a clean slate. Because we want to build something new without historical constraint.

What kind?

And finally we arrive at our preferred method: the cloud country. Our idea is to proceed cloud first, land last. Rather than starting with the physical territory, we start with the digital community. We recruit online for a group of people interested in founding a new virtual social network, a new city, and eventually a new country. We build the embryonic state as an open source project, we organize our internal economy around remote work, we cultivate in-person levels of civility, we simulate architecture in VR, and we create art and literature that reflects our values.

Over time we eventually crowdfund territory in the real world, but not necessarily contiguous territory. Because an under-appreciated fact is that the internet allows us to network enclaves. Put another way, a cloud community need not acquire all its territory in one place at one time. It can connect a thousand apartments, a hundred houses, and a dozen cul-de-sacs in different cities into a new kind of fractal polity with its capital in the cloud. Over time, community members migrate between these enclaves and crowdfund territory nearby, with every individual dwelling and group house presenting an independent opportunity for expansion.

What we've described thus far is much like the concept of ethnic diasporas, which are internationally dispersed but connected by communication channels with each other and the motherland. The twist is that our version is a reverse diaspora: a community that forms first on the internet, builds a culture online, and only then comes together in person to build dwellings and structures. In a sense you can think of each physical outpost of this digital community as a cloud embassy, similar to the grassroots Bitcoin embassies that have arisen around the world. New recruits can come to either the virtual or physical environment, beta test, and decide to leave or stay.

Now, with all this talk of embassies and countries one might well contend that cloud countries, like the aforementioned micronations, are also just a LARP. Unlike micronations, however, they are set up to be a scaled LARP, a feat of imagination practiced by large numbers of people at the same time. And the experience of cryptocurrencies over the last decade shows us just how powerful such a shared LARP can be.


Could a sufficiently robust cloud country with, say, 1-10M committed digital citizens, provable cryptocurrency reserves, and physical holdings all over the earth similarly achieve societal recognition from the United Nations? A cloud country with a population of this size would actually fit right in the middle of the pack globally, as out of the 193 UN-recognized sovereign states approximately 20% of existing countries have a population of less than 1M and ~55% have a population of less than 10M. This includes many countries people typically think of as "real", like Luxembourg (615k), Cyprus (1.18M), Estonia (1.3M), New Zealand (4.7M), Ireland (4.8M), Singapore (5.8M), and so on.

More at the link. 

Think of this as another variety of virtual feudalism.

Friday, May 7, 2021

Friday Fotos: Hoboken Spring [Hallucinated City]

Analyze This! Screaming on the flat part of the roller coaster ride [Does GPT-3 get the joke?]

Here’s Seinfeld’s first television appearance. It is from 1977 on Celebrity Cabaret, a nationally syndicated show. He’s doing a bit that starts with the Roosevelt Island tramway. You know what that is?

Background knowledge and common sense

Or don’t you? Just to be sure Seinfeld helpfully explains what a tramway is. What it is, really, is something he uses to set up the joke, but that’s not what I’m interested in. I’m interested in background knowledge, often known as common sense knowledge in the rarified world of artificial intelligence (AI).

If you look at the version of this bit that Seinfeld published in his book, Is This Anything?, you’ll see that he doesn’t explain it at all (I’ve placed it immediately below). He just assumes – Bam! – you know it is. He also assumes you know that the South Bronx is a rather sketchy neighborhood – back in the day it was said that “the Bronx is burning.” Because sometimes it was. But if you didn’t already know that you wouldn’t get the joke.

But Seinfeld doesn’t say “the South Bronx” in the version in the clip. He simply refers to “the ghetto.” That tells you want you need to know to get the joke. Though I’ve not consulted him on this, I assume he did that because he figured that most people in a national audience would not know about the sad state of the South Bronx. New Yorkers would know that; it’s background knowledge for them – unless of course they’re actually in the South Bronx, in which case it’s in their face. But others are not likely to know that.

So that’s what interests me, the background knowledge, the common sense knowledge, that holds the bit together. You also have to know that roller coasters go up and down (did you notice the gesture he made during the bit?), that they’re a little scary on the downslope, that bankruptcy isn’t consistent with amusement park rides, that cities have governments and that it’s those governments that do things, etc. We know all this stuff without thinking about it.

But computers do not. So we’re going to quiz a computer about the punch line.

The Bit: Roosevelt Island Tramway

I see they just finished the Roosevelt Island Tramway.

That’s nice…

The city’s going bankrupt,

they’re putting up rides for us.

Next thing you know, there’ll be a roller coaster through the South Bronx.

That would be the first roller coaster where the people scream on the flat part of the ride.

Analysis and Commentary from GPT-3

GPT-3 is a massive AI engine that OpenAI revealed in June 2020. Massive? It’s got 175 billion parameters. Parameters? Don’t worry about it. Think of a parameter as a control knob on your TV – do TV’s have knobs anymore? Imagine a TV with 175 billion, with a “B”, knobs. That’s a lot of twiddling. It was trained on 499 billion tokens (think, roughly, words) of text sucked in from the internet. So it has processed text about Roosevelt Island, tramways, roller coasters, the South Bronx and a lot of other things, such as Antarctica, green cheese, Mata Hari, redwoods, giant Komodo dragons, fried rice, the Saturn V rocket, Jerry Seinfeld, pencils, The Tale of Genji, whales, catapults, looms, fruit flies, and anything else laying around on the Internet.

What GPT-3 does is to produce text. You feed it a bit of text and it takes it a runs with it by producing a continuation of the text. The text it produces is often astonishingly natural. It seems like a human wrote it. But no human did, just a dumb, but huge, just ginormous, machine.

If you feed GPT-3 some text and then ask a question, it will answer the question. That’s what I did with GPT-3. Well, not me, my friend Phil Mohun is the one who quizzed the machine. He gave it Seinfeld’s bit, and then asked a question. GPT-3 answered. Since I was particularly interested in the last line, the punch line, that’s what Phil asked about.

1] Here’s the first interaction. I’ve set Phil’s words in boldface.

Jerry Seinfeld tells the following joke:

I see they just finished the Roosevelt Island Tramway.
That’s nice…
The city’s going bankrupt,
they’re putting up rides for us.
Next thing you know, there’ll be a roller coaster through the South Bronx.
That would be the first roller coaster where the people scream on the flat part of the ride.

Q: Why would the people scream on the flat part of the ride?
GPT-3: Because it’s the Bronx.

Well, yeah. So what? I wanted to see if GPT-3 understood the relationship between being scared on a roller coaster and being scared on the ground in the South Bronx. So Phil tried again.

2] He increases the “temperature.” Don’t worry about just what that is, but increasing the temperature raises the amount of randomness in GPT-3’s response. Note that on this try Phil also gave GPT-3 the whole bit, as in example 1. That’s true of the other examples as well, but I’m omitting that part because it’s redundant. Just assume GPT-3 got the whole deal.

Q: Why would the people scream on the flat part of the ride?
GPT-3: There’ll be nothing to see for miles but rubble.

We’re getting somewhere. It seems like GPT-3 “knows” something about the South Bronx. But what does rubble have to do with screaming?

3] Here’s another response:

Q: Why would the people scream on the flat part of the ride?
GPT-3: So the people in Queens can see them.

That’s very interesting, and indicates some knowledge of geography as Queens and the South Bronx adjoin one another. But the roller coaster would have to be pretty close to Queens in order for the screams to reach people there. How far does GPT-3 think screams can be heard, under what wind conditions, and so forth. It’s not clear what’s going on inside GPT-3 – why would people on the roller coaster want to be noticed by people in Queens? – but the reply is not flat-out nonsense.

Thursday, May 6, 2021

Ferry at dawn

Sketches After Latour: A Working Paper

Title above, abstract, contents, and introduction below. Download at:

Research Gate:

Abstract: Thirteen various observations on the thinking of Bruno Latour, from We Have Never Been Modern through Reassembling the Social and up to modes. Excursions into Stanley Fish, Paul Feyerabend, and Claude Lévi-Strauss. Topics: religion, history, social organization, culture, mind, language, truth and felicity. Thread running throughout: Latour’s “flat ontology”, in which everything has agency, has fruitful, perhaps even substantial and revelatory, implications. COI: 10.13140/RG.2.2.15423.64164


Introduction: Latour in the bushes 2
Latour Modern, Not 5
From Fish to Latour, Really 8
Latour's Modes of Existence 12
Lévi-Strauss and Latour: Transformations of a Myth? 14
Presuppositions and Conclusions: Some Comments on Latour 16
Ontology and Epistemology: gone, together? 18
Some Latour Litanies from Feyerabend 19
The Living Cosmos 20
Chapter 1 of the New Latour Available, Apocalypse Awaits 22
Latour and the End of the End of History 24
Latour and Culture 26 Of Factish Gods and Modes of Existence 27
Latour, Language, and Translation 33

Introduction: Latour in the bushes

Just yesterday, May 5, 2021, I was looking through my hard-drive for something-I-forget-what and I came upon this, an all but complete draft of a working paper on Latour. It was from late November of 2017. All I had had to do was complete the front matter, this introduction plus the abstract, and I would have been done.

Why’d I drop it? I haven’t the foggiest idea.

In any event, I’ve now completed it. This working paper collects posts written between April 1, 2012 and August 12, 2015. I’d started blogging about Latour on August 17, 2011, with my first post on Assembling the Social; that project continued until October 11. I issued those posts as a working paper two days later, Reading Latour: Reassembling the Social. And I suppose that my engagement with Latour reached its peak when I issued Living with Abundance in a Pluralist Cosmos: Some Metaphysical Sketches in January of 2013, which is what its title implies: I’d taken Latour’s ideas, sifted them through my own work and coughed up an outline for a metaphysics. Surprised me, but there it is.

In some way, reading Latour was a return to my intellectual roots, in philosophy, in Continental thought. But a strange return. Figuring I wanted his recent thinking, I deliberately began reading him with a late work, Reassembling the Social, which turned out to be a high-level methodological manual rather than a systematic exposition of a philosophical position. It was only some months afterward that I took Graham Harman’s advice and read We Have Never Been Modern. I found it strange, a segment from an ongoing-conversation where I missed the beginning. Graham Harman thought it was easy, I thought it was tough. I marked it up quite a bit, a habit from my undergraduate days, and continued on. The posts I’ve collected here represent some of that continuing on.

* * * * *

Some of these are little squibs, some are longer.

Latour Modern, Not: In the process of reading (three chapters in) We Have Never Been Modern: “...this virtual Constitutional Convention is mostly a conversation among humans about how they’re going to talk and think about relations among humans and non-humans. In the end, though, in that Parliament of Things that Latour leaves to his successors, does he not want the Things to be among the constitutional conveners?”

From Fish to Latour, Really: Stanley Fish talks about ideas as though they are just out there in the ether. Latour seems them as embodied in a vast meshwork of intersecting networks: “While I can’t imagine Fish denying the existence of such networks, I don’t see that they’re uppermost in his mind either. It’s not what drives his intuitions. It’s like the Gestaltist’s duck-rabbit. Fish and Latour may look at the same state of affairs, but one is driven by the intuition that it is a rabbit while the other sees a duck.”

Latour's Modes of Existence: For Latour, each mode of existence has its own characteristic type of veridication. Charles Tart advanced a similar idea in the 1970s: “When you take a psychedelic drug you (may well) have a (so-called) peak experience in which deep truths are revealed to you. Those truths often seem like trite nonsense to those who've not experienced a psychedelic peak. Let's posit that those deep truths are, in fact, specific to one's biochemical neural state and can ONLY be judged and appreciated from within that state.”

Lévi-Strauss and Latour: Transformations of a Myth?: Lévi-Strauss had a tremendous influence on me early in my career. Latour has an interesting passage about him in We Have Never Been Modern. And so: “...are the two thinkers so very different as Latour’s little critique aims to suggest? There would seem to be at least one major difference, one at the center of Latour’s thinking, and at the center of Lévi-Strauss’s. Both talk a lot about the opposition between nature and culture. Latour aims to undermine it while Lévi-Strauss would seem to be enmeshed in it, as Derrida argued in ‘Structure, Sign, and Play in the Discourse of the Human Sciences.’”

Presuppositions and Conclusions: Some Comments on Latour: Latour talks of risk-taking rocks and networks of humans and everything else. But he’s rather thin on how people think and reason.

Ontology and Epistemology: gone, together?: Once you allow for multiple modes of existence, things get tricky.

Some Latour Litanies from Feyerabend: Allies in pluralist syntax.

The Living Cosmos: In an oblique way, consider this squib an homage to the John Cage of Silence.

Chapter 1 of the New Latour Available, Apocalypse Awaits: I’m all of three pages into the just-released English translation of An Inquiry Into Modes of Existence. Latour talks of climate science; I remember Hurricane Sandy. I conclude, “The weird aliens have come home to roost.”

Latour and the End of the End of History: Fukuyama was wrong about the end of history, an idea born out of washed up Hegel/Marx. “...lurking in Politics of Nature [...] are the seeds of a philosophy of history as a process of negotiations beyond the boundaries of society, of existing institutions, negotiations of collectives in dragon territory.”

Latour and Culture: He’s missing something. He fails to distinguish between society, conceived as a group of people oriented toward and interacting with one another, and culture, the beliefs, practices, a mores of those people. A tricky distinction to be sure: “Still, it changes how I think about Latour to realize that, in effect, those actor networks are important because they are the genetic substrate of culture. Finally, I’ve thought from the beginning that Latour is missing a psychology. He has little to nothing to say about the mind, and yet it is the mind that holds those actor networks together.”

Of Factish Gods and Modes of Existence: About Latour’s modes of existence, each with its own “felicity” conditions, with an excursion into a sermon delivered by a local preacher in a Black church. Religion is one mode. Science, waaayy over there, is different (Reference perhaps?). I conclude, however, that “Latour is missing some modes. He’s flattened a bunch of them into this single mode, Reproduction. Thus he’s giving us a scheme that bears an echo of that old nature/culture distinction he’s worked so hard to replace. His scheme has 14 modes which center on human activity and this one mode that, while humans as physical beings must participate in it, seems to be a single mode only in contrast to, by comparison with, the other fourteen.”

Latour, Language, and Translation: Language, whether written or spoken, is a double system in which a string of signifiers carries or encodes a string of signifieds, to use Saussure’s terms. The signifiers are physical events in the public domain, vibrations in air, marks on a page, while the signifieds exist within the privacy of individual minds and brains. In Latour’s terms, signifiers act like intermediaries while signifieds are mediators. In translation from one language to another, intermediaries must function as mediators.

Sunk boat

On bringing bioligcal data to bear on artificial neural networks [I think we're getting somewhere]

Comedy gold, the missing comma [much ado about nothing]

Wednesday, May 5, 2021

Do Wide and Deep Networks Learn the Same Things? [I think we're getting somewhere]

From Google's AI Blog: Do Wide and Deep Networks Learn the Same Things?

A common practice to improve a neural network’s performance and tailor it to available computational resources is to adjust the architecture depth and width. Indeed, popular families of neural networks, including EfficientNet, ResNet and Transformers, consist of a set of architectures of flexible depths and widths. However, beyond the effect on accuracy, there is limited understanding of how these fundamental choices of architecture design affect the model, such as the impact on its internal representations.

In “Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth”, we perform a systematic study of the similarity between wide and deep networks from the same architectural family through the lens of their hidden representations and final outputs. In very wide or very deep models, we find a characteristic block structure in their internal representations, and establish a connection between this phenomenon and model overparameterization. Comparisons across models demonstrate that those without the block structure show significant similarity between representations in corresponding layers, but those containing the block structure exhibit highly dissimilar representations. These properties of the internal representations in turn translate to systematically different errors at the class and example levels for wide and deep models when they are evaluated on the same test set. 

More on the blog.

Go West, into the purple

The relationship between religiosity and authoritarian values varies with education and level of societal development

Reality is not perceived, it is enacted

Reality is not perceived, it is enacted – in a universe of great, perhaps unbounded, complexity.

David Hays and I published that in a little article, we called it a note, we published over thirty years ago (see below). It was Hays’s formulation, but I liked it then. And I still like it?

But what does it mean? Forget the stuff after the dash, it’s that first part I’m thinking about. It clearly implies that without actors there’s no reality. Since we humans are the actors does that mean that before us, there was no reality? That’s absurd. The earth and the universe existed before we did, no?

Yes. Perhaps it means that they weren’t real. You mean they were imaginary, or illusions? No, more like the difference between the real and the not-real had no meaning.

That seems a little better. Let’s try something else. Let’s go the Latour route and say that everything is an actor, not just us humans. That leaves plenty of actors to enact reality prior to us.

That’s beginning to make sense.

It seems to me that we’re haunted by an old picture that we might as well attribute to Descartes, though it’s likely older. In this picture we are over here and the (external) world is way way over there somewhere. Our problem is to figure out what that’s like. How do we do that? By perception.

But that’s never really worked out. There’s always the problem of figuring out whether or not perception really matches the world. It never does or, what amounts to the same thing, there’s no place where we can stand to verify the match. The only way out of that bottle is to stop playing that game. We drop the convention of assigning agency only to humans and all the creatures get to enact reality.

We still have to verify our perceptions and theories, etc. But we accept that they’re always going to be provisional. But as long as we can use them to act effectively in the world, we’re OK.

Where does that leave the foundations of physics, with all that fancy mathematics and no way to test it with empirical observation? It means that the foundations of physics is no longer enacting science, which requires empirical verification. It’s doing something else. Ironic science, as John Horgan has it?

Let’s consider a very different example, “The Rite of Spring” from Disney’s 1941 Fantasia. It depicts an enactment of the solar system and the earth prior to human habitation. It’s certainly not a perception. You can call it an artistic imagining, with considerable license taken, if you will – for example, the T. Rex and the Stegosaurus couldn’t have fought because they lived a hundred million years apart (and Disney knew that). But that’s an action, no? Yes. Disney isn’t doing science, though it obviously interests him a great deal in this episode. He’s doing art.

Science and art are two different ways of enacting reality. Two different modes. Which brings us back to Latour again, who has advanced a theory of modes. Note, however, that I have decided to take a somewhat different view of modes and have offered a preliminary sketch: Living with Abundance in a Pluralist Cosmos: Some Metaphysical Sketches, Working Paper, January 2013,

I could go on and on, and probably should. But not now. Later. Meanwhile here’s the paper where Hays and I offered that formulation along with the immediate context.

* * * * *

Rest your eyes on the balloon

Does human habitation cause extinction of other species? Not until the Holocene [c. 10 kya]

Well-trained dogs

Tuesday, May 4, 2021

Humans have been terraforming the earth for 12,000 years ['pristine' habitats aren't so pristine]

From the linked article, People have shaped most of terrestrial nature for at least 12,000 years:


The current biodiversity crisis is often depicted as a struggle to preserve untouched habitats. Here, we combine global maps of human populations and land use over the past 12,000 y with current biodiversity data to show that nearly three quarters of terrestrial nature has long been shaped by diverse histories of human habitation and use by Indigenous and traditional peoples. With rare exceptions, current biodiversity losses are caused not by human conversion or degradation of untouched ecosystems, but rather by the appropriation, colonization, and intensification of use in lands inhabited and used by prior societies. Global land use history confirms that empowering the environmental stewardship of Indigenous peoples and local communities will be critical to conserving biodiversity across the planet.


Archaeological and paleoecological evidence shows that by 10,000 BCE, all human societies employed varying degrees of ecologically transformative land use practices, including burning, hunting, species propagation, domestication, cultivation, and others that have left long-term legacies across the terrestrial biosphere. Yet, a lingering paradigm among natural scientists, conservationists, and policymakers is that human transformation of terrestrial nature is mostly recent and inherently destructive. Here, we use the most up-to-date, spatially explicit global reconstruction of historical human populations and land use to show that this paradigm is likely wrong. Even 12,000 y ago, nearly three quarters of Earth’s land was inhabited and therefore shaped by human societies, including more than 95% of temperate and 90% of tropical woodlands. Lands now characterized as “natural,” “intact,” and “wild” generally exhibit long histories of use, as do protected areas and Indigenous lands, and current global patterns of vertebrate species richness and key biodiversity areas are more strongly associated with past patterns of land use than with present ones in regional landscapes now characterized as natural. The current biodiversity crisis can seldom be explained by the loss of uninhabited wildlands, resulting instead from the appropriation, colonization, and intensifying use of the biodiverse cultural landscapes long shaped and sustained by prior societies. Recognizing this deep cultural connection with biodiversity will therefore be essential to resolve the crisis.

Leaves above [green red new]

Brockman’s Third Culture and the emergence of a new philosophical regime

I’m wondering whether what literary agent John Brockman has called the third culture isn’t, in effect, philosophy in a mode of broad-ranging intellectual synthesis, an attempt to pull things together in an era of intellectual hyper-specialization. Let me ease into it.

What is philosophy?

For some time I’ve been aware of a distinction between philosophy as an academic discipline and philosophy as a mode of thought, a distinction I have from Peter Godfrey-Smith (himself a philosopher, who is fond of octopi). It was a philosophy as a mode of thought that led me to major in philosophy as an undergraduate at Johns Hopkins. Alas, academic philosophy turned out to be intellectually specialized in a way I sense to be at odds with the mode of thought.

But then what did I know?

By the time I’d figured that out I’d racked up enough philosophy course to fulfill the requirements of the major, so I saw no point in switching to something else. But what would I have switched to? Literature is what I’d come to love, but I didn’t have enough courses in English to satisfy score a major there. But I did take lots of literature courses, many of them the polymathic Richard Macksey. Now, that’s what I was looking for in philosophy, an opportunity to range far and wide, to synthesize, and literary study, as Macksey conceived it, was a good vehicle for that. I had, in effect, majored in Macksey.

More recently Eric Schlisser has argued for synthetic philosophy, which he found in Daniel Dennett’s From Bactria to Bach and Back. Yes, Dennett is a card-carrying philosopher, but that book is not philosophy as it is practiced in philosophy departments. It is a wide-ranging look at life and mind under the rubric of evolution. [I should add that I’ve not read the book, but I’ve seen many Dennett lectures on it and am familiar with a number of his papers and have blogged about them.] That is to say, it is philosophy as a mode of thought rather than as an intellectual discipline.

Other books in the mode include Steven Pinker’s How the Mind Works, Jared Diamond’s Guns, Germs, and Steel, and Steven Mithen’s The Singing Neanderthals. These investigators are card-carrying academics with specialized credentials who have chosen to address themselves to a general audience in works of larger scope than is typical of the academy. And they are writing as much for themselves and for fellow specialists as for that general public. It is the only way they have of putting things together. Still more recently I have argued that journalist John Horgan has written such a synthetic book in his (ironically titled) The End of Science, in which he takes the via negativa toward outlining the whole.

Brockman’s third culture

And that brings me to John Brockman who, in his capacity as literary agent, has helped bring many of these books to light. Here is what he says in the introduction to an edited collection, The Third Culture (1995):

The third culture consists of those scientists and other thinkers in the empirical world who, through their work and expository writing, are taking the place of the traditional intellectual in rendering visible the deeper meanings of our lives, redefining who and what we are.

In the past few years, the playing field of American intellectual life has shifted, and the traditional intellectual has become increasingly marginalized. A 1950s education in Freud, Marx, and modernism is not a sufficient qualification for a thinking person in the 1990s. Indeed, the traditional American intellectuals are, in a sense, increasingly reactionary, and quite often proudly (and perversely) ignorant of many of the truly significant intellectual accomplishments of our time. Their culture, which dismisses science, is often nonempirical. It uses its own jargon and washes its own laundry. It is chiefly characterized by comment on comments, the swelling spiral of commentary eventually reaching the point where the real world gets lost.


The wide appeal of the third-culture thinkers is not due solely to their writing ability; what traditionally has been called “science” has today become “public culture.” Stewart Brand writes that “Science is the only news. When you scan through a newspaper or magazine, all the human interest stuff is the same old he-said-she-said, the politics and economics the same sorry cyclic dramas, the fashions a pathetic illusion of newness, and even the technology is predictable if you know the science. Human nature doesn't change much; science does, and the change accrues, altering the world irreversibly.” We now live in a world in which the rate of change is the biggest change. Science has thus become a big story.

That sounds an awful lot like Schlisser’s synthetic philosophy.

I regard the emergence of this synthetic mode of discourse, this philosophical mode, as another indication of fundamental changes in our culture, something Dave Hays and I have called rank-shift. Not only do intellectual specialists have to change how they think and work, but it is necessary to rework a civilization-wide general view of the world. That can only be done in the public arena.

* * * * *

Note: I have decided to gather my posts on this topic under the label “philosophy new.”

Looks like Europe is the world center of war, or maybe just has the best records

Monday, May 3, 2021

This is post number 7500 at New Savanna [Milestone – To infinity and beyond!]

I made my first post at New Savanna on April 14, 2010. Before that I’d been blogging at The Valve, a now defunct group blog centered on literature. I made my first post there, I believe, in December of 2005, but I’d been active on the web in one way or another for over a decade before that. Back in 1994 Bill Berry and I created Meanderings – actually, Bill created it, I just joined in – and I’d been on CompuServe before that. I tell the story about my digital travels in this working paper: Personal Observations on Entering an Age of Computing Machines, Working Paper, November 2015. It started life as a series of blog posts here, as most of my working papers do.

Anyhow, it’s time for another accounting. First I look at the overall total, then I make some observations about the top 10 posts, and I conclude with some remarks of a more personal nature.

Overall page visits and top 10

I still haven’t got the foggiest idea what was going on in 2017. Maybe Russian bots, by why would they spend a year cruising New Savanna? The recent spike looks interesting. I have no idea where that will go. Top posts

Here at the ten most popular posts. Those numbers are not accurate, however. If you go through the numbers post by post you’ll see higher numbers. And I’m pretty sure there are at least one or two posts that should be in this list by aren’t. But I’m not going to try tracking them down. This will do my present purposes.

What’s interesting about the behavior of individual posts is that a number of them show large spikes at some point. With one exception, I do not know what’s going on.

1. Sex, Power, and Purity in Kawajiri’s Ninja Scroll [Rape]

Posted September 2010.

This has been the most popular post since it went up. There’s no mystery about that. Ninja Scroll is a popular anime and the post is about sex. But what’s that spike at about January of 2019?

2. Secrets of Pink Elephants Revealed

Posted October 2010.

This has been popular for a long time as well, though it took two years to gain traction. No surprise here either. It’s about a very popular sequence from a popular Disney cartoon, albeit one that’s eight decades century old. Think about that.

3. Horikoshi’s Wife: Affective Binding and Grief in The Wind Rises

Posted November 16, 2015.

This one pleases me a great deal. Why? Because it’s Miyazaki. I posted it near the end of 2015 and, as I recall, it just coasted along. Then for some reason it spiked early in 2020, and then tapered off at midyear.

4. Two Rings in Fantasia: Nutcracker and Apprentice

Posted December 28, 2010.

This one saw a lot of action in its first two years and then tapered off. I haven’t got the foggiest idea why

5. Joe Rogan and Joey Diaz call “Bruce Lee vs. Chuck Norris” – a rough transcription of their conversation

Posted July 26, 2018.

This went up in July 2018, so it’s relatively recent. It’s also Joe Rogan. What’s that spike early in 2019, January to April?

Yellow, white, red, purple, green

Analyze This! Seinfeld’s narcissistic parakeet [Hi, Mom!]

Notice what Seinfeld does with his voice, his face, and his gestures.


The ancient Greeks had a myth about a myth, well, not exactly, but close enough. Poor Narcissus saw his reflection in pond, became entranced, and stayed there until he died. Thus we have a flower, the kind that sprouted from the spot where Narcissus rotted into the ground, and the name of a mental disorder, narcissism.

Think about it, though, how would an animal recognize its own reflection? Chimpanzees can do it, but monkeys can’t. I don’t know about other species, but I believe the capacity for mirror recognition is rare.

In the first place, mirrors don’t exist in the natural world. There are ponds and streams, but how many animals actually stare at the surface as opposed to drinking? Nor does will a surface produce a very good reflection unless it is still. Should an animal come across a mirror, how would it recognize its reflection as itself? If it attempts to touch the creature in the mirror what happens? It bumps into the mirror. It may still be able to see the other creature, but how can he tell that that other creature is itself?

It’s faced with contradictory evidence of its senses. Vision tells it that there’s a space over there and a creature in it. But when it moves, it bumps into the surface of the mirror. Touch says there’s a barrier there and the motor system is trying to move, but can’t. It just doesn’t make any sense.

And that’s what mirrors are about. Frankly, I don’t quite understand what’s going on in this joke. But it has something to do with our confusion over and, yes, obsession with, mirrors.

The Bit: Parakeet Mirror

You’ll that this version, from Seinfeld’s book, Is This Anything?, is a bit different from the version he delivers in the clip. Jokes are like that.

My mother would always talk to me about what she’s going to do with the living room.

This was her obsession.

She was gonna fix the living room.

“I want to change the living room.”

My mother would say,

“You know, if you make one wall of a room a mirror people think

you have an entire other room.”

She believed this.

What kind of an idiot walks up to a mirror and goes,

“Hey look, there’s a whole other room in there.

There’s a guy in there that looks just like me.”

My parakeet would fall for this.

I would let him out of his cage.

He would fly around and he would go “BANG” right into the mirror.

With his little head that was very smooth at the front.

And the feathers would fly.

And he’d hit the ground.

Then he’d fly off in another direction a little askew.

But even if he thinks the mirror is another room,

why doesn’t he at least try and avoid hitting the OTHER parakeet?


What happened to bird’s-eye view and all that?

There’s another parakeet coming right at you!

Analysis and Comments: The Parakeet

My mother would always talk to me about what she’s going to do with the living room.

This was her obsession.

She was gonna fix the living room.

“I want to change the living room.”

Seinfeld’s heading toward a parakeet, but there isn’t a hit of it here. He’s starting out at home, with his mother talking to him about the living room. Their living room. Her living room. He’s got two voices, his own, and his mother’s.

My mother would say,

“You know, if you make one wall of a room a mirror people think

you have an entire other room.”

She believed this.

Again, he gives voice to his mother, who is expressing her belief in the home-improvement efficacy of an illusion. Why does he say, “She believed this”? She wouldn’t have said it if she believed it, would she? He says it to take up a bit of time, give us room to think, and simply to emphasize her state of mind, her belief.

What kind of an idiot walks up to a mirror and goes,

“Hey look, there’s a whole other room in there.

There’s a guy in there that looks just like me.”

Add another voice, for the idiot.

What if Seinfeld had said, “What kind of an idiot believes such a thing?” Then he’d be accusing his mother of being an idiot, which is not a nice thing to do. When he starts the sentence we don’t know where he’s going with it. If he’d gone to his mother we’d have been a bit shocked, but not surprised. Before we have a time to anticipate that far he heads off in a benign direction, toward some unnamed idiot – definitely not his mother – walking up to the mirror. This guy IS an idiot, though, because he doesn’t recognize that he’s looking at a mirror. Moreover, he explicitly remarks on the doppelgänger in the room, mistaking them for someone else. In fact, any reasonable competent human being would see the mirror for what it is.

My parakeet would fall for this.

I would let him out of his cage.

He would fly around and he would go “BANG” right into the mirror.

With his little head that was very smooth at the front.

Now Seinfeld brings the parakeet on stage. Why bring up the detail of the smooth little head? It does emphasize the parakeet’s vulnerability, perhaps leading us to worry about the parakeet’s fate.

And the feathers would fly.

And he’d hit the ground.

Then he’d fly off in another direction a little askew.

Whew! The little guy’s OK.

But even if he thinks the mirror is another room,

why doesn’t he at least try and avoid hitting the OTHER parakeet?


What happened to bird’s-eye view and all that?

There’s another parakeet coming right at you!

But how could the bird possibly avoid illusory parakeet? If the parakeet moves to the left, the illusory one follows. To the right? Same thing. Up? Down? No difference. There’s no way out.

And what about mom? Where’d she go. Would the joke work without using her to set it up? I think not. If you start with the parakeet it’s just a dumb bird flying into a mirror. They fly into windows too. So what?

No, what makes this joke work is Seinfeld’s mother’s desire to get something out of the illusory room, some improvement to the home, and what happens when the parakeet attempts to fly into that illusion. Between mom and the parakeet, that’s where the laughter comes from.

And it has to be his mom and his parakeet; it has to be in the family. It won’t work with some other person and a third party parakeet. This is a very intimate joke. How does a young child see themselves in their mother’s face? The French psychoanalyst Jacques Lacan calls this the stade du miroir, the mirror stage. But it’s not merely a stage. It continues, on and on.

Note: See the story about a chimpanzé named Vicki in this post, Two Puzzles Concerning the Self.