From Simon’s ant to machine learning, a parable

Simon’s ant is a well-known thought experiment from Chapter 3, “The Psychology of Thinking: Embedding Artifice in Nature,” in Herbert A. Simon (1981), The Sciences of the Artificial. It’s a parable about computation, about how computational requirements depend on the problem to be solved. Stated that way, it is an obvious truism. But Simon’s thought experiment invites you to consider this truism where the “problem to be solved” is an environment external to the computer – it is thus reminiscent of Braitenberg’s primitive vehicles (which I discussed in this post).

Think of it like this: the nervous system requires environmental support if it is to maintain its physical stability and operational coherence. Note that Simon was not at all interested in the physical requirements of the nervous system. Rather, he was interested in suggesting that we can get complex behavior from relatively simple devices, and simplicity translates into design requirements for a nervous system.

Simon asks us to imagine an ant moving about on a beach:

We watch an ant make his laborious way across a wind- and wave-molded beach. He moves ahead, angles to the right to ease his climb up a steep dunelet, detours around a pebble, stops for a moment to exchange information with a compatriot. Thus he makes his weaving, halting way back to his home. So as not to anthropomorphize about his purposes, I sketch the path on a piece of paper. It is a sequence of irregular, angular segments--not quite a random walk, for it has an underlying sense of direction, of aiming toward a goal.

After introducing a friend, to whom he shows the sketch and to whom he addresses a series of unanswered questions about the sketched path, Simon goes on to observe:

Viewed as a geometric figure, the ant’s path is irregular, complex, hard to describe. But its complexity is really a complexity in the surface of the beach, not a complexity in the ant. On that same beach another small creature with a home at the same place as the ant might well follow a very similar path.

What does this tell us about machine learning?

Think of the ant as the machine, that is, the computer system, and the environment as the HUMONGOUS pile of data it is walking through. What the machine does is quite simple. In the case of large language models, which is what I’ve thought the most about, the machine makes a guess about the next word and adjusts parameter weights accordingly. That’s all it does, guess-adjust, guess-adjust, guess-adjust...for all the texts in the corpus.

Taken collectively those texts constitute an enormously complex environment. Correlatively, the model our machine builds with its simple procedure is enormously complex as well, though not quite so complex as that environment.

Now, and here’s where things get tricky, think of ourselves as the ant, and the world as the complex environment we’re walking about it. I know, we’re not so simple as the ant, but in comparison to the vast complexity of the world, yes, we’re simple. In the process of walking through this vast world we write things. Those things, in turn, constitute the environment of some machine learning system. So, we “consume” the world and emit writings. The computer “consumes” our writings and emits a large language model.

Think about that very carefully. The model we build of the world is, in fact, complex. But not so complex as the world itself. The model the machine builds of its textual world is complex as well. But is it as complex as those texts? Is it as complex as human understanding of those texts? Does that suggest why some thinkers insist that AIs have direct access to the world, sensing it and moving about in it, if they are to be truly “intelligent”?

* * * * *

This post from 2020 covers similar territory, albeit in different and more abstract terms, World, mind, and learnability: A note on the metaphysical structure of the cosmos, August 15, 2020, https://new-savanna.blogspot.com/2020/08/world-mind-and-learnability-note-on.html.

I have included that in my working paper, GPT-3: Waterloo or Rubicon? Here be Dragons, Version 4.1, May 7, 2022, pp. 23-26, https://www.academia.edu/43787279/GPT_3_Waterloo_or_Rubicon_Here_be_Dragons_Version_4_1.

This post is based on material from an older one, Computational Thinking and the Digital Critic: Part 2, An Ant Walks on the Beach and a Pilot is Alone, July 25, 2017, https://new-savanna.blogspot.com/2014/04/computational-thinking-and-digital_30.html.

The last bite [yum]

Remember the Sabbath...[a different world]

Ezra Klein, Sabbath and the Art of Rest, NYTimes, December 3, 2022. From the introduction:

Do we know how to truly rest? Who would we be if we did?

I’ve been wrestling with these questions since I read Abraham Joshua Heschel’s stunning book “The Sabbath” in college. The ancient Jewish ritual of the Sabbath reserves a full day per week for rest. As it’s commonly practiced, that means about 25 hours every week of no work, very little technology and plenty of in-person gathering.

But the Sabbath is a much more radical approach to rest than a simple respite from work and technology. Implicit in the practice of the Sabbath is a stinging critique of the speed at which we live our lives, the ways we choose to spend our time and how we think about the idea of rest itself. That, at least, is a central argument of Judith Shulevitz’s wonderful book, “The Sabbath World: Glimpses of a Different Order of Time.”

Shulevitz is a longtime culture critic and currently a contributing writer for The Atlantic. Her book isn’t just about the Sabbath itself, it’s about the world the Sabbath tries to create: one with an entirely different conception of time, morality, rest and community. It’s the kind of world that is wholly different from our own, and one whose wisdom is urgently needed.

It is about engaging a different range of behavioral modes, ways the brain has of engaging us with the world. I suspect it engages the brain’s so-called default mode network, which is more dream-like, less driven, more social, more conducive to synching with others.

* * * * *

Addendum: Judith Shulevitz observes:

You are supposed to let your mind wander. We’ll gather, have a moment of just letting go. Not all the time, some of the time. But these are the positive things that the frame gives a special gloss to the way it does a work of art and says, this space here, it’s meaning. Make sense of it.

That's definitely the default mode network.

On the naturalistic study of the linguistic behavior of artificial intelligence

This is draft material from the introduction of a working paper I’m about to finish: Discursive Competence in ChatGPT, Part 1: Talking with Dragons.

* * * * *

When I started playing with ChatGPT on December 1, 2022, I had no specific intentions. I wanted to poke around, see what I could see, and then...As I said, I had no specific intentions. I certainly did not intend to spend hours interacting with it to produce a Microsoft Word document currently (1.1.22) containing 59,197 words of transcripts – the vast majority from ChatGPT – on 173 pages.

One of the earliest things I did with ChatGPT – not THE first, it was my third session, on December 1, 2022 – was to dialog about Steven Spielberg’s Jaws and the ideas of Rene Girard. I took that and wrote it up for 3 Quarks Daily.[1] Then I had some fun with “Kubla Khan,” quizzed it about trumpets, had a long session about Gojira/Godzilla, and then returned to Spielberg, this time to A.I. Artificial intelligence. By this time I was developing a feel for how ChatGPT responded. Both the Jaws and the A.I. posts are included in this paper.

I became a bit more systematic, looking for things, testing them out. That led to a post with a rather baroque title, Of pumpkins, the Falcon Heavy, and Groucho Marx: High level discourse structure in ChatGPT,[2] which I’ve also included in this paper. In that post I advanced the argument that there are parameters in the language model that govern ligher level discourse structures independently of the specific words and strings that realize them.

The alternation pattern is something like this:

A, B, A, B....

That can be repeated as often as one will. The text in the A sections is always drawn from one body of material while the text in the B sections is drawn from a different body of material. That’s the pattern ChatGPT has learned. Where is it in the net? How’s it encoded.

The frame structure is a bit more complicated:

A (B, C, B, C....) A’

The embedded alternation draws on two bodies of material, any two bodies. The second part of the frame, A’, must complement the first, A.

Again, it’s not a complex structure. But it’s not defined directly over particular words. It’s defined over groups of words, placing the groups, not the individual words, into specified relationships in the discourse string.

I then suggested that the patterns I had identified in Jaws and A.I. where similar, but, if anything, more complex.

I had become all but convinced that ChatGPT had explicit control over high-level aspects of its discourse. Here humans make statements like those, we take it as obvious that they have some “grammar” of high-level discourse structures. Narratologists, linguists, and psycholinguists study them. But ChatGPT is not a human. It is, shall we say, a machine, a machine that was trained to guess the next word, word after word after word....and so forth, for jillions of texts. All that’s in there is statistics about those texts. It’s a “stochastic parrot”, as one well-know paper argued.

Perhaps, in a sense, that is a true. But it is also terribly reductive and, I have come to believe, all but beside the point. Large language models issue one word at a time for the same reason that humans do: That’s the nature of the communication channel, and tells us relatively little about the device that is pushing words through the channel. LLMs develop rich and complicated structures of parameter weights during the training process. Yes, those structures are statistical in nature, but they are also structures. Perhaps there are aspects of those structures that we can investigate without having to “open the hood” and examine parameter weights.

I made that suggestion in a post, Abstract concepts and metalingual definition: Does ChatGPT understand justice and charity?, also included in this paper.[3] Chomsky famously distinguished between competence and performance, where the study of linguistic performance is about the mechanism that produces and understands texts while the study of linguistic competence is about the structure of the texts independent of underlying mechanisms. When I analyze ChatGPT’s output, as I have been doing for the past month, I am investigating its competence. When researchers pop the hood and examine parameter weights, they are investigating performance mechanisms. I further suggest that a better understanding of an LLM’s competence will aid in studying those performance mechanisms by giving us clues about what they are doing.

Nor am I the only one who believes that. Others have come to that conclusion as well, though perhaps not quite in those terms. Here is the abstract of a recent preprint from Marcel Binz and Eric Schulz from the Max Planck Institute:

We study GPT-3, a recent large language model, using tools from cognitive psychology. More specifically, we assess GPT-3’s decision-making, information search, deliberation, and causal reasoning abilities on a battery of canonical experiments from the literature. We find that much of GPT-3’s behavior is impressive: it solves vignette-based tasks similarly or better than human subjects, is able to make decent decisions from descriptions, outperforms humans in a multi-armed bandit task, and shows signatures of model-based reinforcement learning. Yet we also find that small perturbations to vignette-based tasks can lead GPT-3 vastly astray, that it shows no signatures of directed exploration, and that it fails miserably in a causal reasoning task. These results enrich our understanding of current large language models and pave the way for future investigations using tools from cognitive psychology to study increasingly capable and opaque artificial agents.[4]

My methods are different, but my purpose is the same, “to study increasingly capable and opaque artificial agents” and thus to render them less opaque. The insights we gain thereby will aid us to improve the capabilities of the next generation of artificial agents.

* * * * *

[1] Conversing with ChatGPT about Jaws, Mimetic Desire, and Sacrifice, 3 Quarks Daily, December 5, 2022, https://3quarksdaily.com/3quarksdaily/2022/12/conversing-with-chatgpt-about-jaws-mimetic-desire-and-sacrifice.html.

[2] Of pumpkins, the Falcon Heavy, and Groucho Marx: High level discourse structure in ChatGPT, New Savanna, December 8, 2022, https://new-savanna.blogspot.com/2022/12/of-pumpkins-falcon-heavy-and-groucho.html.

[3] New Savanna, December 16, 2022, https://new-savanna.blogspot.com/2022/12/abstract-concepts-and-metalingual.html.

[4] Binz, Marcel, and Eric Schulz. 2022. “Using Cognitive Psychology to Understand GPT-3.” PsyArXiv. June 21. doi:10.31234/osf.io/6dfgk.

"Mondrian" house along the river in Hoboken

Does a manned mission to Mars make sense?

Michael Ceglowski, Why Not Mars, Idle Words, 1.1.23.

Like watching paint peel:

The buildup to Mars would not look like Apollo, but a long series of ISS-like flights to nowhere. If your main complaint about the International Space Station is that it’s too exciting and has a distracting view of Earth out the window, then you’ll love watching ISS Jr. drift around doing bone studies in deep space. But if you think rockets, adventure, exploration, and discovery are more fun than counting tumors in mice, then the slow and timorous Mars program will only break your heart.

Sticking a flag in the Martian dust would cost something north of half a trillion dollars, with no realistic prospect of landing before 2050. To borrow a quote from John Young, keeping such a program funded through fifteen consecutive Congresses would require a series “of continuous miracles, interspersed with acts of God”.

Let the robots do it:

But fifty years of progress in miniaturization and software changed the balance between robots and humans in space. Between 1960 and 2020, space probes improved by something like six orders of magnitude, while the technologies of long-duration spaceflight did not. Boiling the water out of urine still looks the same in 2023 as it did in 1960, or for that matter 1060. Today’s automated spacecraft are not only strictly more capable than human astronauts, but cost about a hundred times less to send (though it’s hard to be exact, since astronauts have not gone anywhere since 1972).

The imbalance between human and robot is so overwhelming that, despite the presence of a $250 billion International Space Station National Laboratory, every major discovery made in space this century has come from robotic spacecraft. In 2023, we simply take it for granted that if a rocket goes up carrying passengers, it’s not going to get any work done.

Hmmm, and the way AI is unfolding the case it looking better and better to send the robots up.

What about microbial life, on earth and Mars?

The fact that we failed to notice 99.999% of life on Earth until a few years ago is unsettling and has implications for Mars. The existence of a deep biosphere in particular narrows the habitability gap between our planets to the point where it probably doesn’t exist—there is likely at least one corner of Mars that an Earth organism could call home. It also adds support to the theory that life may have started as an interplanetary infection, a literal Venereal disease that spread across the early solar system by meteorite. If that is the case, and if our distant relatives are still alive in some deep Martian cave, then just about the worst way to go looking for them would be to land in a septic spacecraft.

What’s the point?

Somehow we’ve embarked on the biggest project in history even though it has no articulable purpose, offers no benefits, and will cost taxpayers more than a good-sized war. Even the builders of the Great Pyramid at Giza could at least explain what it was for. And yet this project has sailed through an otherwise gridlocked system with the effortlessness of a Pentagon budget. Presidents of both parties now make landing on Mars an official goal of US space policy. Even billionaires who made their fortune automating labor on Earth agree that Mars must be artisanally explored by hand.

The cargo cult to Mars:

When you hold on to a belief so strongly that neither facts nor reason can change it, what you are doing is no longer science, but religion. So I’ve come to believe the best way to look at our Mars program is as a faith-based initiative. There is a small cohort of people who really believe in going to Mars, the way some people believe in ghosts or cryptocurrency, and this group has an outsize effect on our space program. At NASA, the faith takes the form of a cargo cult. The agency has persuaded itself that re-enacting the Moon landings with enough fidelity will reward them with a trip to Mars, bringing back the limitless budgets, uncomplicated patriotism, and rapt public attention of the early sixties. They send up their rockets with the same touching faith that keeps Amtrak hauling empty dining cars across the prairie, dreaming of the golden age of rail.

Let's hear it for robots:

We’re at a rare moment when the United States is in between white elephant space projects. The ISS is nearing the end of its life, and tensions between NASA and Roscosmos have filled all hearts with hope that we can soon drop the thing into the ocean. For the first time since Nixon, Americans have a chance to choose a bolder future for their space program.

One path forward would be to build on the technological revolution of the past fifty years and go explore the hell out of space with robots. This future is available to us right now. Simply redirecting the $11.6 billion budget for human space flight would be enough to staff up the Jet Propulsion Laboratory and go from launching one major project per decade to multiple planetary probes and telescopes a year[63]. It would be the start of the greatest era of discovery in history.

There’s much more at the link.

H/t Tyler Cowen.

Just passing thru

Half-way through my eighth decade: What next?

At the end of 2017 I wrote a post in which I reflected back on my intellectual life, Reflections on entering my eighth decade and why it portends to be the most productive one of my life. That remains a decent overview of what I’ve been up to in the last 50+ years, though it is hardly complete. It gives little sense of my interest in literature beyond “Kubla Khan,” my work on ring-composition, my thinking about form and computation, my commentary on digital humanities. Nor does it say much about my work on cognitive science, the brain, or cultural evolution. And then there’s all the descriptive work I’ve done on films. And music!

So, yeah, I guess it’s not such a good overview. I have no intention to fill in those gaps in this post. But it does give a sense that I’ve been doing other things than thinking and writing, about my involvement in my local community. In that sense, it’s good.

As for what’s next, let’s set that aside for a moment. Every so often I’ve written about my posting habits, how they vary wildly during the course of the year. My most recent such post was in November of 2018, A Mind Over the Long Haul: My Posting Patterns. Let’s see where that stands. The chart below shows my posting habits from the beginning of 2017 through the end of 2022. Each column indicates the number of posts for a single month.

The up and down cycling of my posting productivity is clearly visible. Notice that the slump came in the late Winter and early Spring in 2017, a bit earlier for 2018, and about the same in 2019. Notice that there’s a steep drop at the beginning of 2020, then a rise in summer and into the Fall, with a slump coming in November and continuing through March of 2021, when things move back up, only to drop back down in October and continuing through March of 2022.

So, in 2020, we have slump in November. In 2021 the slump moves to October. Guess when it hits in 2022? That’s right, September. That’s not good. But then we have a major – and unexpected rebound – in December. I know what that’s about; that’s when I started playing with and blogging about ChatGPT. I expect that to continue in this year.

What happened in the Summer of 2020 that brought me back to life? That’s when GPT-3 hit, in June or perhaps July I believe. On August 5, 2020 I issued my working paper, GPT-3: Waterloo or Rubicon? Here be Dragons, which I have since revised several times. I don’t recall off-hand what got me revved up in April of 2021, perhaps nothing in particular. But I know what got me revved in 2022. Jerry Seinfeld, Analyze This! AI meets Jerry Seinfeld, and then Grace Lindsey published Models of the Mind, which I read and wrote about, To Model the Mind: Speculative Engineering as Philosophy. Then came a dialog on “Kubla Khan” in May, Symbols and Nets: Calculating Meaning in “Kubla Khan.” That kept things moving on “Kubla Khan,” which as you my recall, is where all this started back at Johns Hopkins in the late 1960s and early 1970s.

And then the big one came in later in May, Relational Nets Over Attractors, A Primer: Part 1, Design for a Mind, which I have subsequently revised. That brought together the work on cognitive networks I had done with David Hays in the 1970s, the work we did on the brain in the 1980s, my conversations about neurodynamics with Walter Freeman in the late 1990s and early 2000s, and my reading of Peter Gärdenfors on the geometry of meaning. It’s all there in one wonderful crazy paper, a major synthesis, albeit a very speculative one. July saw the discussion of Yann LeCun’s position paper, A Path Towards Autonomous Machine Intelligence, which I found interesting and discussed over the next couple of weeks.

But, for reasons I don’t understand, the excitement had worn off by September. Come to think of it, I do see why things went downhill. I saw new possibilities of linking my work with what others were doing, but was unable to actualize any of them. I remained in a slump until ChatGPT came out at the very end of November. I signed up for an account and started playing with it.

That’s taken a considerable amount of my time over the last month. I’ve written many blog posts, most with long transcripts of my interaction with ChatGPT. I’ve been working on a paper for the last week and a half – current title, Discursive Competence in ChatGPT, Part 1: Talking with Dragons – and expect to issue it within a week.

Once again I see the possibility of linking my work to the work of others. I do think there is an audience within the technical community for the work I’m doing with ChatGPT. I think I can build on it. But I don’t want to say anything about that here and now. I suppose I don’t want to jinx things by talking in public. Bit mostly I want to finish with this post and get on with the work.

‘Till later.

A touch of yellow [What is it?]

India rising on the world stage

Roger Cohen, Russia’s War Could Make It India’s World, NYTimes, Dec. 31, 2022. The article opens:

Seated in the domed, red sandstone government building unveiled by the British Raj less than two decades before India threw off imperial rule, S. Jaishankar, the Indian foreign minister, needs no reminder of how the tides of history sweep away antiquated systems to usher in the new.

Such, he believes, is today’s transformative moment. A “world order which is still very, very deeply Western,” as he put it in an interview, is being hurried out of existence by the impact of the war in Ukraine, to be replaced by a world of “multi-alignment” where countries will choose their own “particular policies and preferences and interests.”

Certainly, that is what India has done since the war in Ukraine began on Feb. 24. It has rejected American and European pressure at the United Nations to condemn the Russian invasion, turned Moscow into its largest oil supplier and dismissed the perceived hypocrisy of the West. Far from apologetic, its tone has been unabashed and its self-interest broadly naked. [...]

In other words, with its almost 1.4 billion inhabitants, soon to overtake China as the world’s most populous country, India has a need for cheap Russian oil to sustain its 7 percent annual growth and lift millions out of poverty. That need is nonnegotiable. India gobbles up all the Russian oil it requires, even some extra for export. [...] As Western sanctions against Russia have driven up energy, food and fertilizer costs, causing acute economic difficulties in poorer countries, resentment of the United States and Europe has stirred in Asia and Africa.

Democracy or authoritarianism?

The postwar order had no place for India at the top table. But now, at a moment when Russia’s military aggression under President Vladimir V. Putin has provided a vivid illustration of how a world of strongmen and imperial rivalry would look, India may have the power to tilt the balance toward an order dominated by democratic pluralism or by repressive leaders.

Which way Mr. Modi’s form of nationalism will lean remains to be seen. It has given many Indians a new pride and bolstered the country’s international stature, even as it has weakened the country’s pluralist and secularist model.

This, that, and the other lead “to buoyant projections that India, now No. 5, will be the world’s third-largest economy by 2030, behind only the United States and China.”

“Paradoxically, the war in Ukraine has diminished trust in Western powers and concentrated people’s minds on how to hedge bets,” said Pratap Bhanu Mehta, a prominent Indian political theorist. “India feels it has the United States figured out: Yes, you will be upset but you’re in no position to do anything about it.”

That has proved a good bet up to now. “The age of India’s significant global stature has just begun,” said Preeti Dawra, the Indian-born director of global marketing at Nanyang Technological University in Singapore.

Digital access:

India wants its presidency of the group to have the world as “one family” and the need for “sustainable growth” as its core themes. It wants to push the transformation of developing countries through what Mr. Kant, the organizer, called “technological leapfrogging.” India, with its near universal connectivity, sees itself as an example.

About 1.3 billion Indians now have a digital identity. Access to all banking activities online, through digital bank accounts, has become commonplace during Mr. Modi’s eight years in power. They were once the preserve of the middle class. Poorer Indians have been empowered.

Hindus & Muslims: “In the ascendant Hindu narrative that Mr. Modi has done nothing to discourage, India belongs in the first place to its Hindu majority. The Muslim interlopers of the Mughal Empire and other periods of conquest take second place.”

Be a bridge:

With inequality worsening, food security worsening, energy security worsening, and climate change accelerating, more countries are asking what answers the post-1945 Western-dominated order can provide. India, it seems, believes it can be a broker, bridging East-West and North-South divisions. [...]

India is in a delicate position. In the face of American criticism, the country chose to take part this year in Russian military exercises that included units from China. At the same time, India is part of a four-nation coalition known as the Quad that includes the United States, Japan and Australia and works for a “free and open Indo-Pacific.”

There is more at the link.

Color play for the new year

AI DEBATE 3: The AGI Debate [hosted in Montreal]

The Pivotal Discussion in Shaping the Path of AGI's Global Discourse.

Five panels of the world's most distinguished researchers and experts on AGI :

Panel 1: Cognition and Neuroscience

Panel 2: Common Sense

Panel 3: Architecture

Panel 4: Ethics and morality

Panel 5: AI: Policy and Net Contribution

Speakers: Erik Brynjolfsson, Yejin Choi, Noam Chomsky, Jeff Clune, David Ferrucci, Artur d'Avila Garcez, Michelle Rempel Garner, Dileep George, Ben Goertzel, Sara Hooker, Anja Kaspersen, Konrad Kording, Kai-Fu Lee, Francesca Rossi, Jürgen Schmidhuber and Angela Sheffield.

Moderator and co-organizer (with Vincent Boucher) : Gary Marcus. Gary Marcus is a leading voice in artificial intelligence. Scientist, best-selling author, and entrepreneur, he was Founder and C.E.O. of Geometric Intelligence, a machine-learning company acquired by Uber in 2016. His most recent book, Rebooting AI, co-authored with Ernest Davis, is one of Forbes’s 7 Must Read Books in AI.

Vincent Boucher is President of Montreal.AI and Quebec.AI.

Website: https://agidebate.com/

Gary Marcus remarks:

Every single speaker was both articulate, and pointed. Two—Schmidhuber and Clune—notably more optimistic about the potential of current techniques. (Sparks flew when they expressed that optimism.) But not one speaker thought that current AI was anything like the holy grail of artificial general intelligence. (Clune thought we might get there, by 2030.) Virtually every speaker thought that things were about to get wild—and not necessarily in entirely good ways.

I urge you, if you care about artificial intelligence, and its future, and its impact on society, to watch the debate, in full. That’s a huge commitment, 3.5 hours, but one thing that I think all of our speakers could agree on is that artificial intelligence, in whatever form it currently is, is about to have a huge impact on society.

Marcus recommends Tiernan Ray's account of the debate for ZDNet. I have posted excerpts from it.

* * * * *

Noam Chomsky spoke first, remarking that he saw no scientific value in current AI. Why? Because machine learning systems can learn any language whatsoever and so can tell us nothing about specifically human language. In making this remark I suppose he is implicitly analogizing these AI systems to the language acquisition device (LAD) he has been theorizing about for years. The LAD can learn all and only human languages and so is quite different the engines of machine learning.

I wonder. Do we actually know that current ML engines can learn any language? Are we even sure they can learn English? To be sure, many of them produce fluent English output, but they also make many blunders. The blunders, however, tend to be about semantics, not syntax. Chomsky's theorizing has always been centered on syntax, so semantic mistakes, however glaring, may be irrelevant to him. Can one be said to know a language without knowing its semantics? But none of us knows the full semantics of any natural language.

Cf. my recent remarks, On limits to the ability of LLMs to approximate the mind’s structure, December 27, 2022.

Sabine Hossenfelder on the possibility of human extinction

Correction to what I say at 11 mins 50 seconds: A supervolcano eruption ejects more than 1000 cubic kilometers of matter (not 1000 cubic meters). Sorry about that!

What do we know about the risks of human going extinct? In today's video I collect what we know about the frequency of natural disasters and just how they would kill us, and estimates for man-made disasters.

00:00 Intro
00:30 What Is an Existential Risk?
02:00 Would Extinction be Bad?
04:18 Man-made Disasters
10:36 What's The Risk of Man-made Disasters?
11:35 Natural Disasters
13:38 What's the Risk of Natural Disasters?
16:55 Why Can't the LHC Produce Black Holes? 19:29 Protect Your Data with NordVPN

Many thanks to Jordi Busqué for helping with this video http://jordibusque.com/

What about rogue AIs?

9:01: The biggest problem with Artificial Intelligence that longtermists see is that that an AI could become intelligent enough to survive independently of us but pursue interests that conflict with our own. They call it the “misalignment problem”. In the worst case, the AIs might decide to get rid of us. And could we really blame them? I mean, most of us can’t draw a flower, let alone a human face, so what’s the point of our existence really?

This wouldn’t necessarily be an extinction event in the sense that intelligent life   would still exist, it just wouldn’t be us. Under which circumstances you might consider an AI species a continuation of our own line is rather unclear. Longtermists argue it depends on whether the AI continues our “values”, but it seems odd to me to define a species by its values, and I’m not sure our values are all that great to begin with.

In any case, I consider this scenario unlikely because it assumes that advanced AIs will soon be easy to build and reproduce which is far from reality. If you look at what’s currently happening, supercomputers are getting bigger and bigger, and the bigger they get the more difficult they are to maintain and the longer it takes to train them. If you extrapolate the current trend to the next few hundred years, we will at best have a few intelligent machines owned by companies or governments, and each will require a big crew to keep it alive. They won’t take over the world any time soon.

The biggest existential risk?

19:23: Okay, so in summary, the biggest existential risk is our own stupidity.

From yesterday's walk in the fog

My mind is now on hiatus

No more writing for me today. 

I've just sent an article off to
3 Quarks Daily. 

Time to take a very short break.

But I'll be back.