Street Art in the NYTimes – "these works point beyond the self"

Seph Rodney, New York’s Sidewalk Prophets Are Heirs of the Lascaux Cave Artisans, NYTimes,Aug. 6, 2020.

Those early artisans [Lascaux, France] drew these creatures over and over, likely fascinated by their forms and their powers, but also intuiting that whatever happened to the animals would almost certainly be a harbinger of what would happen to humans. The presence of the bison and stags, their physical fitness and numbers, their mass migrations would have indicated the onset of plagues or cataclysmic weather systems. Containing some 15,000 paintings and engravings from the Upper Paleolithic era, the caves in Southwestern France were not simply an exhibition space for local talent. They essentially constituted a public square where a community shared critical knowledge.

These portraits and discrete stories are not very different from our contemporary forums: the street art adorning boarded-up storefronts in New York City. They tell us about our shared political realities, the people we coexist with in social space and the ways in which our stories and fates are tied together. If you walk the streets of SoHo, the alleys of the Lower East Side, and heavily trafficked avenues in Brooklyn, as I did over the last few weeks, you will see these symbols and signs and might wonder at their meanings. What became apparent to me is that in the intervening millenniums between those cave paintings and the killing of George Floyd, the messages we share, like the sociopolitical circumstance that impel them, have become more complex.

Now street artists take account of the qualified legal immunity protecting police officers, the Black Lives Matter movement and the ramifications of a dysfunctional democracy, among other realities, using a well-developed visual language of cultural memes that illustrate the ideological battles among regional, racial and cultural factions. When we see the image of thin, green-skinned, bipedal beings with teardrop-shaped black apertures for eyes, we typically read “alien.” But when I see the image of such a creature holding a sign that reads “I can’t breathe,” I grok an urgent message: Even aliens visiting from light years away understand the plight of Black people in the United States because this situation is so obviously dire.

Today’s street paintings contain dispatches that proliferate across the city sphere — lovely, challenging, angry, remonstrative and even desperate. There are two critical things to note about them. They are different from graffiti, which to my eyes is egocentric and monotone, mostly instantiating the will of the tagger over and over again. I am here and you must see me, is the message.

The street artists in these works point beyond the self, to larger, collective issues. The other pressing point is that these images in chalk, paint and oil stick are ephemeral. Between the time I walked these districts and alerted the photographer to document them, five images had already disappeared. One was a depiction of the transgender freedom fighter Marsha P. Johnson, whose image was marked in chalk on the sidewalk in the ad hoc tent city created near Chambers Street a few weeks ago. It’s since been cleared out by police officers.

Gonna' fly now [one of those old time good ones, as Pops used to say]

FDR in Philadelphia

DIY in Jersey City, no longer there

DIY in Jersey City, no longer there

How I Found a Home in Jersey City and Got Steve Fulop Elected Mayor, Part 3

I'm bumping this to the top of the blog in honor of the fact that today, August 6, 2020, almost thirteen years after we started working for it in November, 2013, Jersey City opens it first legitimate, purpose built SK8 park.

When the second installment ended I was photographing graffiti and had started volunteering in the Hamilton Park Neighborhood Association. There I found out that Janice Monson, who taught in the Jersey City schools and was a hardcore activist, used to take her students up on the Newport Wall and take their pictures upon graduation.

Back in November of 2006, before I’d shown up at an HPNA meeting, I was walking around, or perhaps I was in the car on the way back from my Sunday AM grocery run. One or the other, it doesn’t much matter. Anyhow, I spotted some color:

That’s the stuff, says I, that’s the stuff. When I got closer, I noticed a ramp against a wall:

And then this:

Someone was obviously using this site—the floor slab of an abandoned industrial building of some sort—as a park for skateboarding and BMX bike riding. See:

I took that one in July of 2007. Notice that the art on the walls has changed. It seems that some local, and not so local, graffiti writers used this site as something of an experimental gallery even as the skateboarders and BMXers used it to hone their athletic skills.

All off the books, so to speak. They were trespassing on this land. But no one cared. The cops certainly knew what was going on. Sure, it was a little off the beaten path, but only a little. The site’s not particularly remote or hidden. Oh, they knew, the cops. But why hassle the kids? They weren’t hurting anyone; the land wasn’t being used for anything. Let ‘em use it; keeps ‘em outa’ trouble.

Then this appeared:

Someone was hauling concrete back there and building mounds and ramps. Someone knew about construction. Someone was investing in improvements in this property that they didn’t own. Call it crazy; call it initiative; call it real.

Grounded [etherium]

GPT-3: Waterloo or Rubicon? Here be Dragons

I've published a new working paper. Title above, download links, abstract, table of contents, and introduction below.

Download at:

• academia: https://www.academia.edu/43787279/GPT_3_Waterloo_or_Rubicon_Here_be_Dragons_Version_2
• SSRN: https://ssrn.com/abstract=3667608
• ResearchGate: https://www.researchgate.net/publication/343444766_GPT-3_Waterloo_or_Rubicon_Here_be_Dragons

---

GPT-3 is a significant achievement.

But I fear the community that has created it may, like other communities have done before – machine translation in the mid-1960s, symbolic computing in the mid-1980s, triumphantly walk over the edge of a cliff and find itself standing proudly in mid-air.

This is not necessary and certainly not inevitable.

A great deal has been written about GPTs and transformers more generally, both in the technical literature and in commentary of various levels of sophistication. I have read only a small portion of this. But nothing I have read indicates any interest in the nature of language or mind. That seems relegated to the GPT engine itself. And yet the product of that engine, a language model, is opaque. I believe that, if we are to move to a level of accomplishment beyond what has been exhibited to date, we must understand what that engine is doing so that we may gain control over it. We must think about the nature of language and of the mind.

That is what this working paper sets out to achieve, a beginning point, and only that. By attending to ideas by Adam Neubig, Julian Michael, and Sydney Lamb, and by extending them through the geometric semantics of Peter Gärdenfors, we can create a framework in which to understand language and mind, a framework that is commensurate with the operations of GPT-3. That framework can help us to understand what GPT-3 is doing when it constructs a language model, and thereby to gain control over that model so we can enhance and extend it.

It is in that speculative spirit that I offer the following remarks.

---

Abstract: GPT-3 is an AI engine that generates text in response to a prompt given to it by a human user. It does not understand the language that it produces, at least not as philosophers understand such things. And yet its output is in many cases astonishingly like human language. How is this possible? Think of the mind as a high-dimensional space of signifieds, that is, meaning-bearing elements. Correlatively, text consists of one-dimensional strings of signifiers, that is, linguistic forms. GPT-3 creates a language model by examining the distances and ordering of signifiers in a collection of text strings and computes over them so as to reverse engineer the trajectories texts take through that space. Peter Gärdenfors’ semantic geometry provides a way of thinking about the dimensionality of mental space and the multiplicity of phenomena in the world, about how mind mirrors the world. Yet artificial systems are limited by the fact that they do not have a sensorimotor system that has evolved over millions of years. They do have inherent limits.

Contents

0. Starting point and preview 1
1. Computers are strange beasts 4
2. No meaning, no how: GPT-3 as Rubicon and Waterloo, a personal view 8
3. The brain, the mind, and GPT-3: Dimensions and conceptual spaces 16
4. Gestalt switch: GPT-3 as a model of the mind 24
5. Engineered intelligence at liberty in the world 26

0. Starting point and preview

GPT-3 is based on distributional semantics. Warren Weaver had the basic idea in his 1949 memorandum, “Translation” (p. 8). Gerard Salton operationalized the idea in his work using vector semantics for document retrieval in the 1960s and 1970s (p. 9). Since then distributional semantics has developed as an empirical discipline. The last decade of work in NLP has seen remarkable, even astonishing, progress. And yet we lack a robust theoretical framework in which we can understand and explain that progress. Such a framework must also indicate the inherent limitations of distributional semantics. This document is a first attempt to outline such a framework, as such its various formulations must be seen as speculative and provisional. I offer them so that others may modify them, replace them, and move beyond them.

It started with a comment at a blog

On July 19, 2020, Tyler Cowen made a post to Marginal Evolution entitled “GPT-3, etc.” It consisted of an email from a reader who asserted, “When future AI textbooks are written, I could easily imagine them citing 2020 or 2021 as years when preliminary AGI first emerged,. This is very different than my own previous personal forecasts for AGI emerging in something like 20-50 years…” While I have my doubts about the concept of AGI – it’s too ill-defined to serve as anything other than a hook on which to hang dreams, anxieties, and fears – I think GPT-3 is worth serious consideration.

Cowen’s post has attracted 52 comments so far, more than a few of acceptable or even high quality. I made a long comment to that post. I then decided to expand that comment into a series of blog posts, say three or four, and then to collect them into a single document as a working paper. When it appeared that those three or four posts would grow to five or six I decided that I would issue two working papers. This first one would concentrate on GPT-3 and the nature of artificial intelligence, or whatever it is. The second would speculate about the future and take a quick tour of the past.

Here is a slightly revised version of the comment I made at Marginal Revolution. This paper covers the shaded material. The rest will be covered in the second paper.

Yes, GPT-3 [may] be a game changer. But to get there from here we need to rethink a lot of things. And where that's going (that is, where I think it best should go) is more than I can do in a comment.

Right now, we're doing it wrong, headed in the wrong direction. AGI, a really good one, isn't going to be what we're imagining it to be, e.g. the Star Trek computer.

Think AI as platform, not feature (Andreessen). Obvious implication, the basic computer will be an AI-as-platform. Every human will get their own as an very young child. They're grow with it; it’ll grow with them. The child will care for it as with a pet. Hence we have ethical obligations to them. As the child grows, so does the pet – the pet will likely have to migrate to other physical platforms from time to time.

Machine learning was the key breakthrough. Rodney Brooks’ Gengis, with its subsumption architecture, was a key development as well, for it was directed at robots moving about in the world. FWIW Brooks has teamed up with Gary Marcus and they think we need to add some old school symbolic computing into the mix. I think they’re right.

Machines, however, have a hard time learning the natural world as humans do. We're born primed to deal with that world with millions of years of evolutionary history behind us. Machines, alas, are a blank slate.

The native environment for computers is, of course, the computational environment. That's where to apply machine learning. Note that writing code is one of GPT-3's skills.

So, the AGI of the future, let's call it GPT-42, will be looking in two directions, toward the world of computers and toward the human world. It will be learning in both, but in different styles and to different ends. In its interaction with other artificial computational entities GPT-42 is in its native milieu. In its interaction with us, well, we'll necessarily be in the driver’s seat.

Where are we with respect to the hockey stick growth curve? For the last 3/4 quarters of a century, since the end of WWII, we've been moving horizontally, along a plateau, developing tech. GPT-3 is one signal that we've reached the toe of the next curve. But to move up the curve, as I’ve said, we have to rethink the whole shebang.

We're IN the Singularity. Here be dragons.

[Superintelligent computers emerging out of the FOOM is bullshit.]

* * * * *

ADDENDUM: A friend of mine, David Porush, has reminded me that Neal Stephenson has written of such a tutor in The Diamond Age: Or, A Young Lady's Illustrated Primer (1995). I then remembered that I have played the role of such a tutor in real life, The Freedoniad: A Tale of Epic Adventure in which Two BFFs Travel the Universe and End up in Dunkirk, New York.

While the portion of the comment to be elaborated in the next working paper is considerably longer than the portion being elaborated in this one, I do not expect that paper to be proportionately longer. This paper covered quasi-technical matters requiring fairly careful exposition. The next paper will go by more quickly and will, in sections, approach science fiction.

* * * * *

1. Computers are strange beasts – They’re obviously inanimate, and yet we communicate with them through language. The don’t fit pre-existing (19th century?) conceptual categories, and so we are prone to strange views about them.

2. No meaning, no how: GPT-3 as Rubicon and Waterloo, a personal view – Arguing from first principles it is clear that GPT-3 lacks understanding and access to meaning. And yet it produces very convincing simulacra of understanding. But common sense understanding remains elusive, as it did for old school symbolic processing. Much of common sense is deeply embedded in the physical world. GPT-3, as it currently functions is, in effect, an artificial brain in a vat.

3. The brain, the mind, and GPT-3: Dimensions and conceptual spaces – GPT-3 creates a language model by examining the distances and ordering of signifiers in a collection of text strings and computes over them so as to reverse engineer but the trajectories texts take through a high-dimensional mental space of signifieds. Peter Gärdenfors’ semantic geometry provides a way of thinking about the dimensionality of mental space and the multiplicity of phenomena in the world.

4. Gestalt switch: GPT-3 as a model of the mind – GPT-3 creates: 1) a model of a body of natural language texts, and only a model. 2) Those texts are the product of human minds. 3) Though the application of 2 to 1 we may conclude that GPT-3 is also a model of the mind, albeit a very limited one. 3 requires a Gestalt switch.

5. Engineered intelligence at liberty in the world – The “intelligence” in systems such as GPT-3 is static and reactive. To liberate and mobilize it we need to endow AI systems with mental models of the kind investigated in “old school” symbolic AI.

Sparkychan wondering why John Searle couldn't keep it locked up in that Chinese room of his

Once more into the Chinese Room

I’ve been crashing on my GPT-3 working paper and I had a new thought about Searle’s infamous Chinese room [1].

Yet if you would believe John Searle, no matter how rich and detailed the world model included in an AI, understanding would necessarily elude them. When I first encountered the Chinese room argument years ago my reaction was something like: interesting, but irrelevant. Why irrelevant? Because it said absolutely nothing about the techniques AI or cognitive science investigators used and so would provide no guidance toward improving that work. He did, however, have a point: If the machine has no contact with the world, how can it possibly be said to understand anything at all? All it does is grind away on syntax.

What Searle misses, though, is the way in which meaning is a function of relations among concepts, as I pointed out earlier (see [2]). It seems to me, however – and here I’m just making this up off the top of my head – we can think of meaning as having both a intentional aspect, the connection of signs to the world, and a relational aspect, the relations of signs among themselves. Searle’s argument concentrated on the former and said nothing about the latter.

What of the intentional aspect when a person is writing or talking about things not immediately present, which is, after all quite common? In this case the intentional aspect of meaning is not supported by the immediate world. Language use thus must necessarily be driven entirely by the relations of signifiers among themselves, Sydney Lamb’s point which we have already investigated [again, see it in [2]).

Have I at long last wrestled that pesky argument to the ground?

We'll see.

* * * * *

[1] I have written a number of blog posts about this argument. Here’s one of them: Another romp around Searle’s Chinese room, New Savanna, blog post, July 18, 2018, http://new-savanna.blogspot.com/2018/07/another-romp-around-searles-chinese-room.html. You can find others at the Searle link, which, however, contains other Searle posts as well, http://new-savanna.blogspot.com/search/label/Searle.

[2] New Savanna blog post, 2. The brain, the mind, and GPT-3: Dimensions and conceptual spaces, July 29, 2020, http://new-savanna.blogspot.com/2020/07/2-brain-and-gpt-3-part-1-dimensions-and.html.

Orange bouy on the Hudson at dawn with the George Washington Bridge in the background

Can we adapt to climate change?

Ashutosh Jogalekar reviews Bjorn Lomborg, False Alarm: How Climate Change Panic Costs us Trillions, Hurts the Poor and Fails to Fix the Planet, in 3 Quarks Daily. From the review:

The benefits of climate adaptation become clear in the case of large-scale flooding due to sea level rise, one of the biggest concerns of climate change proponents. The book talks about a report based on two papers that estimate the damage as a percentage of GDP caused by sea level rise. With no adaptation, the damage is worth 5.3% of GDP and will lead to 187 million people displaced by the year 2100 even after spending 24 billion dollars on dikes. But strikingly, by just doubling the spending on dikes to 48 billion dollars, the models predict that the cost of flooding will now drop to only 0.008% of GDP. Even the authors of the studies from which these dire estimates are drawn say, “Damages of this magnitude are very unlikely to be tolerated by society and adaptation will be widespread.” Even the earlier study reported in the New York Times said in its abstract that “coastal defenses aren’t considered.”

The example of dikes shows that not only will adaptation be inevitable but that it would be relatively cheap and lead to disproportionate benefits. Constructing dikes to minimize water damage is cheap. Putting better building and fire codes in place and discouraging people from living in dry forested areas is cheap for minimizing the impact of forest fires. If malaria caused by mosquitoes breeding in warmer areas because of climate change is a concern, making nets and spraying insecticide is cheap. Smog and air pollution have already been drastically reduced in major cities by local and national government action. All these things are vastly cheaper than large-scale cuts to carbon emissions which would be like trying to drive a nail in a wall using a bulldozer; the nail may possibly be driven in, but the entire house would likely collapse. If flooding is threatening your town or state, in general in seems foolhardy to try to address it by asking China or the United States to expensively reduce CO2 emissions over the next fifty years and much more sensible to cheaply build dikes.

The point that Lomborg makes is that even if we assume that extreme events are caused by warmer weather, it does not mean directly tackling carbon emissions is the best way to address these problems. An elephant in the room is the so-called “bullseye effect”. The bullseye effect simply says that as you expand a bullseye, the chances of hitting the target increase. In case of flooding, forest fires or hurricanes, a big reason why they are causing more damage every year is simply because more people are living in and building expensive assets in the areas in which they strike. So while adaption certainly will be valuable, the simple but perhaps challenging policy of discouraging more people to live in disaster-prone areas will also minimize the damage because of global warming.

A word of caution:

One of the things that Lomborg does not focus on is that all these models – both ones predicting severe effects and one predicting mild ones – are ultimately models. They contain many assumptions that don’t always hold, they are simplified representations of reality and and they often don’t account for non-linear network effects. One of my concerns is that while Lomborg is rightly skeptical of models that portend catastrophe, he is less skeptical about models that forecast the low economic costs and relative benefits of climate change. However it is fair to say that it is precisely because of uncertainties like these that models of the kind Nordhaus uses have a 25% margin of error baked in. Lomborg’s argument is that even with these error margins, the impact is not half as bad as we are made to believe. Nonetheless, keeping that these are models in mind all the time can greatly finesse discussions on climate change and set the right expectations on both sides.

About fossil fuels:

Unfortunately the advantages of fossil fuels lie in basic laws of physics which are almost impossible to circumvent. Because there’s so much oil and natural gas around, fossil fuels are cheap. Because they consist of the most reduced source of carbon (hydrocarbons), they are energy-dense. Mirroring what Gore said, depriving the average citizen of a developing country of fossil fuels would indeed be not just impractical but immoral. In fact this is one of the central problems rich countries like the US have faced in climate change negotiations in which they ask China and India to cut down on fossil fuels and growth while having enjoyed unprecedented growth from fossil fuel burning for decades themselves.

Sadly, most of the climate change negotiations until now have pushed this central fact about the basic energetics and economics of fossil fuels under the rug. The Paris Agreement is sadly the latest culprit in this parade of failed policy prescriptions. The Paris Agreement basically asks every country to voluntarily cut greenhouse gas emissions until 2050. Quite aside from the fact that shifting political and economic realities will make it impossible for most countries to keep these promises, many countries like Mexico are consigning their citizens to an impoverished existence by promising deep cuts. The costs of the agreement are huge – about a trillion dollars per year. However, this vast expenditure of money would result in a minuscule temperature drop of 0.05 degrees Fahrenheit. [...] It goes without saying that a trillion dollars a year would allow any country in Africa or a country like India to improve its citizens’ welfare significantly. In other words, the Paris Agreement calls for very expensive and unrealistic actions resulting in a very small benefit. [...]

The problem with cutting the kinds of emissions as stated in the Paris Agreement is that while their benefits will be minuscule, their costs in terms of stalling national and personal progress will be enormous. Rich people will not be impacted much because their losses will be small, but there will be more people left in poverty and less money for them to climb out of it through investments in education, healthcare and simple technological adaptation. It’s a good example of why climate change policy will hurt the poor the most.

The fallacy of keeping people poor by tying all their problems to climate change is a significant part of the issue. Poverty, air pollution, disease and a lack of educational opportunities are all critical challenges faced by a majority of the world’s population. These problems may partly be tied to climate change, but most of them are separate from it.

Alternative energy sources:

When governments say they are funding “alternative energy”, it almost always means solar and wind energy. But the problems with both of these sources are well known and yet strangely persistent. Neither of these sources work when the sun isn’t shining or the wind isn’t blowing, and baseline energy still has to come from fossil fuels. Wind farms take up land and both wind turbines and solar panels use significant energy from fossil fuels in their production. Efficient energy storage may provide a partial solution, but we aren’t there yet. The harm of pivoting significantly to solar and wind energy becomes clear when we look at Germany. Under a sweeping energy policy, Germany moved away from nuclear power to solar and wind energy. As a result, they pay the highest electricity prices of anybody in Europe. This might still be feasible for a German, but it’s not so for an average rural African or Indian.

Technology:

We came up with technological solutions like GMOs for fighting hunger, technological solutions like vaccines for fighting disease and technological solutions like catalytic converters for fighting air pollution; we did not think that implementing these solutions reflected resignation in any way. It shouldn’t be any different for global warming. We have already implemented cheap technological adaptations for fighting floods, hurricanes and forest fires and should undoubtedly implement more of these. Most importantly, we need to have a more optimistic view of technology as an important part of the solution. One of the most spectacular stories of using technology to address climate change is fracking, and it was technology that was discovered serendipitously and deployed quickly. [...]

Unfortunately as Lomborg points out, funding for renewables and green energy R&D had actually reduced in the last ten years. What is even worse is that both the government and the private sector pick winners and losers like solar and wind energy. As with most R&D, the best solution to combat climate change will be one which we cannot predict, so the best policy is to fund a wide range of options. Lomborg lists a few including algae that can produce biofuels by consuming CO2, better storage technology for wind and solar power, salt spray that can create whiter clouds that reflect sunlight and new kinds of nuclear fission and fusion reactors that can provide clean energy. I think one of the most promising avenues might be genetically engineering trees so that they can bury more carbon in their roots and in topsoil and less in their shoots.

Fly me to the moon

August 2, July in review: poetry, GPT-3 and a couple other things

It’s been an exciting, and exhausting two weeks. I’ve been wanting to ramble on for a week now, but kept putting it off because my series on GPT-3 kept calling. It’s still calling me, but I’m at a strange place and so have stepped back from it.

A strange place [of corpus and the mind]

I don’t quite know what I intended when I started this series of GPT-3 posts. Oh, now I recall. I’d made a long comment about GPT-3 at Marginal Revolution on July 19, 2020. I wanted to elaborate on that. I suppose I had three or four or five posts in mind, certainly less than ten.

Nor will it be over ten. But the first three posts have proved complex enough that I decided to break them into two groups so that I can issue a working paper when the first series is done. That series is strictly about GPT-3 and related issue and will be entitled: GPT-3: Waterloo or Rubicon? Here be Dragons. The second series will be about the future of AI and has this working title: GPT-3: And beyond, the future.

I reached a turning point in the series sometime between July 27, when I uploaded the second post, and July 29, when I uploaded the third. In that interval...ah, I found it. And it was July 24 (my sister’s birthday). I’d linked my first post in the series to Facebook. John Lawler made a comment in which he pointed me to a post by Julian Michael, To Dissect an Octopus: Making Sense of the Form/Meaning Debate. That gave me a clue about what’s going on in engines like GPT-3, and that clue turned my head around. It also made this series of posts somewhat more complicated and demanding than it had been when I set out to write it.

I now have a fundamentally new understanding about what’s going on in these statistical models of bodies of texts. At least I think I do. I’ve now reached the point where I’m wondering if I’ve actually done that, whatever that is. It seems so obvious to me that surely everyone must know it already. And maybe they do, but they don’t recognize it yet. It’s about point of view. From one point of view it’s a duck, from another point of view it’s a rabbit. But the lines and their relations are the same in both cases.

If been chasing this idea for three years or so. This blog post is a good marker:

Borges Redux: Computing Babel – Is that what’s going on with these abstract spaces of high dimensionality? [#DH], New Savanna, blog post, October 2017, https://new-savanna.blogspot.com/2017/10/borges-redux-computing-babel-is-that.html.

I’ve gathered a number of those posts into this working paper:

Toward a Theory of the Corpus, Working Paper, December 31, 2018, 46 pp., https://www.academia.edu/38066424/Toward_a_Theory_of_the_Corpus.

African-American sonnets and literary criticism

It started with the interview I conducted with Hollis Robbins about her new book, out: Forms of Contention: Influence and the African American Sonnet Tradition. We emailed back and forth for two weeks or so and then I edited it into the form of an interview. The last thing we did was to set an African American poet, Marcus Christian, in competition with GPT-3, thus updating the contest between John Henry and the steam drill. The interview was published in 3 Quarks Daily on July 20.

That’s what got me thinking about GPT-3, which I already knew about. By staging a competition – I got GPT-3 to attempt to write a sonnet – I got a little skin in this game and that, I suspect, got me thinking about these language engines in a different way, from a different point of view. It was no longer that strange and interesting technology over there. It was now right here, beneath my nose.

I dealt with it.

As a bonus, I made some progress in conceptualizing the relationship between what I have been calling naturalist literary criticism and the standard criticism that has been practiced since the 1950s or so, On the nature of academic literary criticism as an intellectual discipline: text, form, and meaning [where we are now]. I’ve been chewing on that one for a long time. A long time. The critic has a choice: they can seek meaning in the text (the standard approach) or they can seek to analyze and describe a text’s form (which relatively few choose to do). You can’t do both, not in the same discourse. Oh, I know critics say they’re dealing with form, but they’re not really...And I don’t want to go into that here. Read the post.

Other things: Stagnation, identity, freedom and dignity in music

All this work on sonnets and GPT-3, however, has taken my attention away from a series of posts I’d been working on about Peter Thiel and economic stagnation. Looks like late June was the last I’d posted on that, and the post was a short one presenting a conversation between Thiel and David Graeber.

The conversation with Robbins brought up the issue of cultural identity, another topic I’ve been chewing on for a long time. I’m due for another post on that. Perhaps next week, or the week after. We’ll see.

Finally, I need to do a post about music. When people are absorbed in making music, they cease being old or young, rich or poor, male or female, and so forth. The become, simply, musicians.

Like a Miyazaki movie (Mononoke)

静寂と緑#写真好きな人と繋がりたい #ファインダー越しの私の世界 #キリトリセカイ #photography #photo #樹海 pic.twitter.com/d1Gcp7BziC

— よしと＠故障中 (@yoshito_photo1) August 2, 2020

And now the machines recognize emotion categories embedded in the visual system

A neural network could recognize which #emotions, out of 20 different categories, a human would feel in response to an image, challenging the prevailing view that emotions are independent from the sensory environment. https://t.co/m67bzLgH6k @ScienceAdvances #ScienceMagArchives

— Science Magazine (@ScienceMagazine) August 1, 2020

Abstract: Theorists have suggested that emotions are canonical responses to situations ancestrally linked to survival. If so, then emotions may be afforded by features of the sensory environment. However, few computational models describe how combinations of stimulus features evoke different emotions. Here, we develop a convolutional neural network that accurately decodes images into 11 distinct emotion categories. We validate the model using more than 25,000 images and movies and show that image content is sufficient to predict the category and valence of human emotion ratings. In two functional magnetic resonance imaging studies, we demonstrate that patterns of human visual cortex activity encode emotion category–related model output and can decode multiple categories of emotional experience. These results suggest that rich, category-specific visual features can be reliably mapped to distinct emotions, and they are coded in distributed representations within the human visual system.

Two views, one tree

What Louis Milic saw back in 1966 [digital humanities]

Willard McCarty just reminded me of an article Louis Milic published in the first issue of Computers and the Humanities:

Milic, L. (1966). The next step. Computers and the Humanities, 1(1), pp. 3-6. doi:10.1007/bf00188010

DOIs didn’t exist at that time, much less our helpful Russian friends who have made the world’s scholarly literature freely available through Sci-Hub. At that time we were, of course, locked in a deadly Cold War with the Soviet Union, a war that lasted until the fall of the Berlin Wall in 1989.

Computers and the Humanities aside, 1966 was an important year in humanistic thought in the United States, for that’s the year the French landed in Baltimore and stormed the stacks of The Johns Hopkins University to discourse on “The Languages of Criticism and the Sciences of Man,” [“Les Langages Critiques et les Sciences de l'Homme”]. I was a sophomore at the time and, though I didn’t attend any of the sessions of that (in)famous structuralist symposium, I was in the orbit of Dick Macksey, one the its organizers. I rather doubt that humanities computing was on the minds of any speakers at the symposium, but the informatic and cybernetic culture that informed computing would have been very much in the air, as it also informed structuralism.

As Milic mentions machine translation in his remarks, I observe that MT was one of the founding disciplines of computer science, with its origins in the early 1950s. In America the objective was to translate Russian technical documents into English. The Soviets had broader needs, as they had a multi-cultural multi-lingual nation to govern. The American enterprise collapsed in the mid-1960s and was deep into re-conceptualizing itself as computational linguistics at the time of the structuralist symposium.

Did Milic attend any of those sessions? I do not know. Was he aware of the symposium? Perhaps, though there would come a time when everyone was aware of it. In any event, he was on the faculty of Columbia’s Teacher’s College at the time.

Let us take a look at his (prophetic) essay:

We are still not thinking of the computer as anything but a myriad of clerks or assistants in one convenient console. Most of the results I have just described could have been accomplished with the available means of half a century ago. We do not yet understand the true nature of the computer. And we have not yet begun to think in ways appropriate to the nature of this machine.

I fear that this is more or less true in much of the (so-called) digital humanities to this day, over a half-century later. We cannot hope to come to grips with GPT-3 (and its kin) without learning “to think in ways appropriate to the nature of this machine.”

Milic goes on to point out:

The true nature of the machine is unknown to us, but it is neither a human brain nor a mechanical clerk. The computer has a logic of its own, one which the scholar must master if he is to benefit from his relations with it.

I would underline the last clause of that first sentence, “neither a human brain nor a mechanical clerk.” If neither of those, then what is it? I rather like the phrase “artificial being” (jinzo ningen in Japanese), which Osamu Tezuka used to describe Michi, the central character of his 1949 manga, Metropolis. But it is, I suppose, too general for our purpose.

Milic continues:

Its intelligence and ours must be made complementary, not antagonistic or subservient to each other. For example, understanding in the arts and letters is based on the perception, identification and recognition of patterns. But the patterns must be small and traditional enough to be perceived by the human apparatus.

Such patterns are central to so-called “close reading” (As you may know, I am deeply suspicious of this metaphor of distance. What kind of distance is this, with respect to what? Not physical, nose to page as it were. Metaphysical? I digress.)

He concludes the paragraph:

Perhaps for that reason Aristotle questioned whether a large object could be beautiful. In literature, we sense this when we read a long novel. Unlike the human perceiver, however, the computer can be made to detect the longest and best-concealed pattern, no matter how random an appearance it presents to the human eye. Thus, we must learn to ask it larger questions than we can answer and to detect what escapes our unaided senses. This may involve not only proposing old questions in new ways but even thinking up new questions. The computer can be made an extension of man only if it opens avenues we have not suspected the existence of.

There you have it, a remit for much of the best and most imaginative work being done in computational criticism.

Let me bring this post to a close by quoting Milic’s next paragraph in full – it is by no means the end of his essay. I leave commentary and annotation as an exercise for the reader:

Thinking in a new way is not an easy accomplishment. It means re- orientation of all the coordinates of our existence. Necessarily, therefore, our first motions in that direction are likely to be tentative and fumbling. The most interesting direction, to my mind, for this new work to take is in the imitation of the process of literary composition. For a long time, we have asked ourselves how the mind worked when it tried to articulate its experience with linguistic symbols. Many kinds of analysis (grammatical, statistical, psychological) nave provided us with only a fractional insight into this mystery. The notable failure of machine translation has been paradoxically a very instructive development. Computers were instructed to behave like human translators, and they could not. What was learned about the complexity of linguistic structure, however, far exceeds what might have been gained from translating Chinese or Russian political speeches or scientific papers. That use of the computer was constructive, if not creative. It moved in the direction of synthesis rather than analysis.