Purple Flowers

Machine-learning can predict the evolution of chaotic systems

Natalie Wolchover, Machine Learning’s ‘Amazing’ Ability to Predict Chaos, Quanta Magazine:

The findings come from veteran chaos theorist Edward Ott and four collaborators at the University of Maryland. They employed a machine-learning algorithm called reservoir computing to “learn” the dynamics of an archetypal chaotic system called the Kuramoto-Sivashinsky equation. The evolving solution to this equation behaves like a flame front, flickering as it advances through a combustible medium. The equation also describes drift waves in plasmas and other phenomena, and serves as “a test bed for studying turbulence and spatiotemporal chaos,” said Jaideep Pathak, Ott’s graduate student and the lead author of the new papers. [...]

The algorithm knows nothing about the Kuramoto-Sivashinsky equation itself; it only sees data recorded about the evolving solution to the equation. This makes the machine-learning approach powerful; in many cases, the equations describing a chaotic system aren’t known, crippling dynamicists’ efforts to model and predict them. Ott and company’s results suggest you don’t need the equations — only data. “This paper suggests that one day we might be able perhaps to predict weather by machine-learning algorithms and not by sophisticated models of the atmosphere,” Kantz said.

Knockin’ on Heaven’s Door (at Howard’s Party)

I'd originally posted this on April 3, 2011. Last night, April 14, 2018, I attended Howard's jam, again. And we played “Knocking on Heaven's Door", just like we had that time almost 20 years ago that I wrote up in my book. Another good version. Some of the same people were there, I'm sure. And some new ones. A good time was had by all.

* * * * *

Last night was Howard’s Annual Jam and Birthday Bash. He’s been doing this for twenty-five years, though I’ve only been going for about ten or so. As always, a good time was had by all.

And this year was especially good, perhaps the best I remember – though I don’t remember everything that happened this time, so it may have been even better than I think it was. Here’s a passage from Beethoven’s Anvil, my book about music, that recounts an incident from ten about years ago (pp. 69-70):

A couple I know threw a big party to celebrate their new house and her pregnancy. As he is deep into the Hoboken folk-music scene, about a dozen guitars were there, and some other instruments as well: a flautist, a few pianists, a woman who brought a dozen or so shakers that folks could play, a soprano saxophonist, and me, on flugelhorn and clavé. We played one or two songs as old as dirt, but also lots of Beatles, van Morrison, Bob Dylan—very Sixties.

The front room on the ground floor served as the music room. The music would start and stop, musicians of all levels of ability came and went, and the boundary between players and others was wonderfully fluid. The music was ragged and rambling and occasionally confused and the rhythm would get lost every now and then and all that. From an evolutionary point of view it was just a bunch of apes hanging out and grooming one another while munching on some choice leaves and termites.

But there was at least one moment quite unlike anything exhibited by bands of apes. It was 1:30 or 2 in the morning and we were jamming on Bob Dylan's “Knocking on Heaven's Door.” I took a flugelhorn solo early in this long jam and then, when I was done, went to the bottom register of the horn and started a simple repetitive swelling figure which I played more or less continuously to the end. The soprano sax played harmony to my line, and I to his, and sometimes did a little obbligato, and a guitar solo floated up here, a piano solo there, vocal choruses and refrains happened as needed. At some point I decided to see how much I could drive this train by leaning on my simple line and bearing down. A half minute or so later, four or five or six voices chimed in on the refrain at the same time. A lump came to my throat. There we were, knocking on heaven's door.

It's a bird, it's a plane, it's Bhagwan Shree Rajneesh!

Netflix is currently streaming a 6-part documentary about how central Oregon was "invaded" by Bhagwan Shree Rajneesh and his followers, intent upon establishing a small city devoted to his teachings. These events happened in the early 1980s. Here's the trailer:

The documentary is constituted by news and documentary footage from the 1980s and by current-day interviews with followers of Rajneesh who were there, local people involved in events, and government officials at the local, state, and federal levels. 

In brief, the Rajneeshees bought a 64,000 acre ranch, arrived by the 100s and low 1000s in the early 1980s, got into conflict with the locals over land use as they proceeded on construct their city, which included a small airport, and had left by the mid-1980s after Rajneesh had been deported for immigration fraud. It's a strange and rich story, well worth your attention. I found it a bit unsettling, in part because I'd never looked at anything quite like this, this closely. This is one of those cases where, as the cliche has it, truth is stranger than fiction – assuming we can figure out just what the truth is.

Some links:

• Netflix site: Wild Wild Country.
• Rajneesh (Wikipedia)
• Rajneeshpuram (the city, Wikipedia) 
• The Rise and Fall of the Rajneesh Cult (a series of articles about these events)

A small mechanical device that plays a simple tune

Common Sense in Artificial Intelligence

Sometime back in the 1970s, I believe it was, David Marr observed something of a paradox (I believed he used that word) in the development of artificial intelligence (AI). Much of the early work, which did meet with some success, involved modeling fairly sophisticated forms of knowledge, mathematics and science, but when researchers started working in simple domain, like ordinary narrative, things got more difficult. That is, it seemed easier to model the specialized knowledge of a highly trained scientist than the general knowledge of a six year old. That problem has come to be known in AI as the problem of common sense, and its intractability has was one reason that old school research programs grounded in symbolic reasoning fell apart in the mid-1980s. During the 1990s and continuing on to the present various machine learning techniques have become quite successful in domains that had eluded symbolic AI. But common sense reasoning has continued to elude researchers.

Earlier this year Microsoft co-founder Paul Allen announced that he was giving $125 million to his nonprofit Allen Institute for Artificial Intelligence (AI2) to study common sense reasoning. Here's a short paper from that lab that gives and overview of the problem.

Niket Tandon, Aparna S. Varde, Gerard de Melo, Commonsense Knowledge in Machine Intelligence, SIGMOD Records 2018.

Abstract: There is growing conviction that the future of computing depends on our ability to exploit big data on the Web to enhance intelligent systems. This includes encyclopedic knowledge for factual details, common sense for human-like reasoning and natural language generation for smarter communication. With recent chatbots conceivably at the verge of passing the Turing Test, there are calls for more common sense oriented alternatives, e.g., the Winograd Schema Challenge. The Aristo QA system demonstrates the lack of common sense in cur- rent systems in answering fourth-grade science exam questions. On the language generation front, despite the progress in deep learning, current models are easily confused by subtle distinctions that may require linguistic common sense, e.g. quick food vs. fast food. These issues bear on tasks such as machine translation and should be addressed using common sense acquired from text. Mining common sense from massive amounts of data and applying it in intelligent systems, in several respects, appears to be the next frontier in computing. Our brief overview of the state of Commonsense Knowledge (CSK) in Machine Intelligence provides insights into CSK acquisition, CSK in natural language, applications of CSK and discussion of open issues. This paper provides a report of a tutorial at a recent conference with a brief survey of topics.

Pagoda forms

Why should humanists adopt evolutionary concepts in thinking about culture?

From three years ago. More timely than ever.

Poets are the unacknowledged legislators of the world.
– Percy Bysshe Shelley, A Defense of Poetry

I started working on this post a month or two ago. I’ve written a number of long and complicated posts in the last year or so where I’ve ended up asserting that humanists really need to think about cultural evolution because it provides us with a way to think about that (quasi)autonomous realm that Ed Said believed in, but couldn’t justify on the basis of the literary theory that had developed during his career. I figured some scholars probably never read those arguments because they didn’t want to plough through longs posts making strange arguments.

I figured that the thing to do, then, was to make the assertion, with perhaps a bit of argumentation, in a relatively short post where THAT’s the whole point. So I started drafted that post, and it started growing, and I kept on thinking and before I knew it I’d decided I needed to gather a bunch of stuff together and write a book. So I’ve started on that project – Mind-Culture Co-Evolution is my provisional title – and abandoned that post.

Well, this is that post, resurrected, and relatively short. Why do humanists need to think about cultural evolution? Because

1) it is a way to think about how expressive culture plays a causal role in history, and

2) it is way to put macroscale and microscale work within the same conceptual framework.

Note that when I say cultural evolution I mean just that, cultural evolution, not biological evolution, not evolutionary psychology. Evolutionary psychology is neither here nor there with respect to cultural evolution. Biological evolution, of course, is the ground from which cultural evolution springs, but it operates in a different realm. Culture does exist in the pre-human world, but it’s thin stuff.

There’s been a fair amount of work on cultural evolution in the past two or three decades or so, but it’s rather scattered. There’s no off-the-shelf model that’s ready to go for students of literature, or the arts in general, and there’s a fair amount of nonsense. So we’re going to have to make it up ourselves, and that’s not easy.

The remarks in the rest of this post do not constitute an argument on those points. Such an argument is way beyond the scope of a blog post. That’s why I’ve decided to write a book. The purpose of these remarks is to indicate what I regard as the intellectual scope of a robust approach to cultural evolution.

Expressive culture is a causal force in history

That, I take it, is what Shelley had in mind when he asserted, “poets are the unacknowledged legislators of the world.” And you can’t legislate unless you’ve got a place to stand, unless you aren’t merely a puppet of historical forces.

But what, pray tell, are historical forces? I’d hoped that a trip to the Stanford Encyclopedia of Philosophy would have a helpful article on the philosophy of history. It didn’t. What the article said is that inferring historical causes is difficult. Well, yes, I know that. But are there distinct kinds of causes that have been investigated? No luck there.

Massive Human Entrainment

Fusaroli R, Perlman M, Mislove A, Paxton A, Matlock T, Dale R (2015) Timescales of Massive Human Entrainment. PLoS ONE 10(4): e0122742. doi:10.1371/journal.pone.0122742

Published: April 16, 2015DOI: 10.1371/journal.pone.0122742

Abstract: The past two decades have seen an upsurge of interest in the collective behaviors of complex systems composed of many agents entrained to each other and to external events. In this paper, we extend the concept of entrainment to the dynamics of human collective attention. We conducted a detailed investigation of the unfolding of human entrainment—as expressed by the content and patterns of hundreds of thousands of messages on Twitter—during the 2012 US presidential debates. By time-locking these data sources, we quantify the impact of the unfolding debate on human attention at three time scales. We show that collective social behavior covaries second-by-second to the interactional dynamics of the debates: A candidate speaking induces rapid increases in mentions of his name on social media and decreases in mentions of the other candidate. Moreover, interruptions by an interlocutor increase the attention received. We also highlight a distinct time scale for the impact of salient content during the debates: Across well-known remarks in each debate, mentions in social media start within 5–10 seconds after it occurs; peak at approximately one minute; and slowly decay in a consistent fashion across well-known events during the debates. Finally, we show that public attention after an initial burst slowly decays through the course of the debates. Thus we demonstrate that large-scale human entrainment may hold across a number of distinct scales, in an exquisitely time-locked fashion. The methods and results pave the way for careful study of the dynamics and mechanisms of large-scale human entrainment.

Introduction

Interest in the collective behaviors of complex systems composed of many agents has dramatically increased over the past couple of decades. This interest may stem in no small part from a new ability to measure and model collective behaviors. In a canonical case, Strogatz and Stewart [1] highlight firefly behavior as illustrative of fundamental principles underlying entrained systems [2, 3]. In parts of Southeast Asia, one may happen upon a sea of fireflies, in which each firefly’s intrinsic oscillatory dynamics have become entrained to others around it. The result is a large-scale collective behavior: The fireflies fire in sync in an impressive display brought on by subtle mutual influences. They are entrained in that they match their behavior to the temporal structure of events in the environment [4–6]. This process might involve elements of reciprocal influence between individual agents as in the case of the fireflies, or it might depend predominantly on external environmental events. The firefly model has inspired the investigation of entrainment across many physiological and technological phenomena, from neuronal firing to electric power networks [7]. However, it is still unclear how complex cognitive agents, such as human beings, might also exhibit patterns of large-scale entrainment.

Row upon row of seeds

John Horgan asks: "Is Science Hitting a Wall?"

Back in 1996 John Horgan kicked up a mighty fuss with The End of Science: Facing the Limits of Knowledge in the Twilight of the Scientific Age, which was recently reissued. Put rather crudely, Horgan argued that in field after field, science seems to be spinning its wheels. Perhaps we've run up against limits to our knowledge? In a review-essay I published in 1997 I suggested that perhaps the limits are imposed by our current systems of thought, but that other systems are possible. "What has come to an end, I argue, is a certain view of the world which sees reality as reducible to simple laws about simple systems underpinning the superficial complexity of phenomenal experience. On the contrary, reality is fundamentally complex and reductionism is doomed. The universe is fecund in that it has evolved multiple Realms of Being, with the later ones being implemented in the former."

Be that as it may, he's at it again.  He opens a recent post with some observations:

In “Are Ideas Getting Harder to Find?”, four economists claim that “a wide range of evidence from various industries, products, and firms show[s] that research effort is rising substantially while research productivity is declining sharply.” The economists are Nicholas Bloom, Charles Jones and Michael Webb of Stanford and John Van Reenen of MIT.

As an counter-intuitive example, they cite Moore’s Law, noting that the “number of researchers required today to achieve the famous doubling every two years of the density of computer chips is more than 18 times larger than the number required in the early 1970s.” The researchers found similar trends in research related to agriculture and medicine. More and more research on cancer and other illnesses has produced fewer and fewer lives saved.

These findings corroborate analyses presented by economists Robert Gordon in The Rise and Fall of American Growth and Tyler Cowen in The Great Stagnation. Bloom, Jones, Webb and Van Reenen also cite “The Burden of Knowledge and the ‘Death of the Renaissance Man’: Is Innovation Getting Harder?”, a 2009 paper by Benjamin Jones. He presents evidence that would-be innovators require more training and specialization to reach the frontier of a given field. Research teams are also getting bigger, and the number of patents per researcher has declined.

The economists are concerned primarily with what I would call applied science, the kind that fuels economic growth and increases wealth, health and living standards. Advances in medicine, transportation, agriculture, communication, manufacturing and so on. But their findings resonate with my claim in The End of Science that “pure” science—the effort simply to understand rather than manipulate nature--is bumping into limits.

He continues with a grab-bag of observations he made at recent on the subject of whether or not science is slowing down. Here's one of them:

How much are “pure” discoveries like the big bang or out-of-Africa hypothesis worth? I’d like to say they are priceless, but that answer won’t suffice when we’re talking about government funding. Should we spend billions of tax dollars on a next-generation particle accelerator, gravitational-wave detector or manned mission to Mars when millions of people lack decent health care, housing and education?

I call this the Whitey-on-the-Moon Problem in honor of rap pioneer Gil Scott-Heron. In his 1970 song “Whitey on the Moon” Scott-Heron says, “A rat done bit my sister Nell/(with Whitey on the Moon)./…The man just upped my rent last night/('cause Whitey's on the moon)./No hot water, no toilets, no lights/(but Whitey's on the moon).”

Winsor McCay: The Pet

This post from 2012 is about one of the creepiest films I've ever seen and a classic of early animation.

Winsor McCay was a cartoonist and a pioneering animator who did most of his work in the second decade or so of the 20th Century. He was a skilled and fluent draftsman and, as far as I can tell, had relatively little stylistic influence on subsequent animators, possibly because his style would have been impossible in the commercial animation world as it emerged.

This post consists mostly of notes I made on his next to the last film, The Pet, which is also one of the creepiest films I’ve EVER seen. EVER. This little gem is not kid stuff.

Point of reference: The Pet was made in 1921 while King Kong was made in 1933.

Running Time: c. 10 min 30 sec

Now, let's take a closer look at one of McCay's 1921 films, "The Pet." This is one of those dream films. In this case the dream is the husband's dream and it is about a stray animal that his wife takes in as a pet.

There are two defining characteristics of this creature:

1.) It just grows and grows and eats and eats and grows and grows.

2.) In both its being and its actions it violates boundary after boundary.

These are both obvious enough, but the first could be pointed out by a six-year old while the latter requires some considerable sophistication to formulate explicitly. The six-year old can easily tell you that it both is and is not a cat – and be puzzled by this, that it eats things it shouldn’t – like a coffee pot and a pile of coal; a twenty-six year old could tell you these things as well. But summing it all up as a succession of boundary violations, that would require an article-length piece of academic analysis.

The eating and eating and growing and growing are perfectly visible, concrete events. Each and every one of the boundary violations is also concrete and visible; but the characterization of all those events as “boundary violation” is abstract. One easily notices all those violations, they seem odd, strange, unsettling, and so forth. So, what’s the relationship between that abstract pattern and the “primary process” thinking of the Freudian unconscious?

Caught in a net

The demise of the nation state

I've been reading about this for several years now. Rana Dasguta has an article of that title in The Guardian for April 5, 2018. Here's some excerpts:

The most momentous development of our era, precisely, is the waning of the nation state: its inability to withstand countervailing 21st-century forces, and its calamitous loss of influence over human circumstance. National political authority is in decline, and, since we do not know any other sort, it feels like the end of the world. This is why a strange brand of apocalyptic nationalism is so widely in vogue. But the current appeal of machismo as political style, the wall-building and xenophobia, the mythology and race theory, the fantastical promises of national restoration – these are not cures, but symptoms of what is slowly revealing itself to all: nation states everywhere are in an advanced state of political and moral decay from which they cannot individually extricate themselves.

Why is this happening? In brief, 20th-century political structures are drowning in a 21st-century ocean of deregulated finance, autonomous technology, religious militancy and great-power rivalry. Meanwhile, the suppressed consequences of 20th-century recklessness in the once-colonised world are erupting, cracking nations into fragments and forcing populations into post-national solidarities: roving tribal militias, ethnic and religious sub-states and super-states. Finally, the old superpowers’ demolition of old ideas of international society – ideas of the “society of nations” that were essential to the way the new world order was envisioned after 1918 – has turned the nation-state system into a lawless gangland; and this is now producing a nihilistic backlash from the ones who have been most terrorised and despoiled.

Once upon a time...

The reason the nation state was able to deliver what achievements it did – and in some places they were spectacular – was that there was, for much of the 20th century, an authentic “fit” between politics, economy and information, all of which were organised at a national scale. National governments possessed actual powers to manage modern economic and ideological energies, and to turn them towards human – sometimes almost utopian – ends. But that era is over. After so many decades of globalisation, economics and information have successfully grown beyond the authority of national governments. Today, the distribution of planetary wealth and resources is largely uncontested by any political mechanism.

But to acknowledge this is to acknowledge the end of politics itself. And if we continue to think the administrative system we inherited from our ancestors allows for no innovation, we condemn ourselves to a long period of dwindling political and moral hope. Half a century has been spent building the global system on which we all now depend, and it is here to stay. Without political innovation, global capital and technology will rule us without any kind of democratic consultation, as naturally and indubitably as the rising oceans.

Can we change?

It will be objected, inevitably, that any alternative to the nation-state system is a utopian impossibility. But even the technological accomplishments of the last few decades seemed implausible before they arrived, and there are good reasons to be suspicious of those incumbent authorities who tell us that human beings are incapable of similar grandeur in the political realm. In fact, there have been many moments in history when politics was suddenly expanded to a new, previously inconceivable scale – including the creation of the nation state itself. And – as is becoming clearer every day – the real delusion is the belief that things can carry on as they are.

The first step will be ceasing to pretend that there is no alternative. So let us begin by considering the scale of the current crisis.

Dasgupta then recounts how we got here, starting with the Treaty of Westphalia in 1648, moving through the 19th century, to the decolonization that followed WWII, leaving us with:

There is every reason to believe that the next stage of the techno-financial revolution will be even more disastrous for national political authority. This will arise as the natural continuation of existing technological processes, which promise new, algorithmic kinds of governance to further undermine the political variety.[...] Governments controlled by outside forces and possessing only partial influence over national affairs: this has always been so in the world’s poorest countries. But in the west, it feels like a terrifying return to primitive vulnerability. The assault on political authority is not a merely “economic” or “technological” event. It is an epochal upheaval, which leaves western populations shattered and bereft. There are outbreaks of irrational rage, especially against immigrants, the appointed scapegoats for much deeper forms of national contamination. The idea of the western nation as a universal home collapses, and transnational tribal identities grow up as a refuge: white supremacists and radical Islamists alike take up arms against contamination and corruption.
Advertisement

The stakes could not be higher. So it is easy to see why western governments are so desperate to prove what everyone doubts: that they are still in control. It is not merely Donald Trump’s personality that causes him to act like a sociopathic CEO. The era of globalisation has seen consistent attempts by US presidents to enhance the authority of the executive, but they are never enough. Trump’s office can never have the level of mastery over American life that Kennedy’s did, so he is obliged to fake it.

Sales dynamics of best-selling books

Burcu Yucesoy, Xindi Wang, Junming Huang and Albert-László Barabási, Success in books: a big data approach to bestsellers, EPJ Data Science 2018 7:7. https://doi.org/10.1140/epjds/s13688-018-0135-y

Abstract: Reading remains the preferred leisure activity for most individuals, continuing to offer a unique path to knowledge and learning. As such, books remain an important cultural product, consumed widely. Yet, while over 3 million books are published each year, very few are read widely and less than 500 make it to the New York Times bestseller lists. And once there, only a handful of authors can command the lists for more than a few weeks. Here we bring a big data approach to book success by investigating the properties and sales trajectories of bestsellers. We find that there are seasonal patterns to book sales with more books being sold during holidays, and even among bestsellers, fiction books sell more copies than nonfiction books. General fiction and biographies make the list more often than any other genre books, and the higher a book’s initial place in the rankings, the longer the book stays on the list as well. Looking at patterns characterizing authors, we find that fiction writers are more productive than nonfiction writers, commonly achieving bestseller status with multiple books. Additionally, there is no gender disparity among bestselling fiction authors but nonfiction, most bestsellers are written by male authors. Finally we find that there is a universal pattern to book sales. Using this universality we introduce a statistical model to explain the time evolution of sales. This model not only reproduces the entire sales trajectory of a book but also predicts the total number of copies it will sell in its lifetime, based on its early sales numbers. The analysis of the bestseller characteristics and the discovery of the universal nature of sales patterns with its driving forces are crucial for our understanding of the book industry, and more generally, of how we as a society interact with cultural products.

* * * * *

"Reading remains the preferred leisure activity for most individuals"–really?

See also Chaos in the Movie Biz: A Review of Hollywood Economics [#DH].

Three for the sky