The risks of proliferating AI agents

Malcolm Murray, The Shifting Nature Of AI Risk, 3 Quarks Daily, July 22, 2024.

Second and more importantly, much of the focus in AI is currently on building AI agents. As can be seen in comments from Sam Altman and Andrew Ng, this is seen as the next frontier for AI. When intelligent AI agents are available at scale, this is likely to change the risk landscape dramatically. [...]

Introducing agentic intelligences could mean that we lose the ability to analyze AI risks and chart their path through the risk space. The analysis can become too complex since there are too many unknown variables. Note that this is not a question of “AGI” (Artificial General Intelligence), just agentic intelligences. We already have AI models that are much more capable than humans in many domains (playing games, producing images, optical recognition). However, the world is still recognizable and analyzable because these models are not agentic. The question of AGI and when we will “get it”, a common discussion topic, is a red herring and the wrong question. There is no such thing as one type of “general” intelligence. The question should be when will we have a plethora of agentic intelligences operating in the world trying to act according to preferences that may be largely unknown or incomprehensible to us.

For now, what this means is that we need to increase our efforts on AI risk assessment, to be as prepared as possible as AI risk continues to get more complex. However, it also means we should start planning for a world where we won’t be able to understand the risks. The focus in that world needs to instead be on resilience.

There's more at the link.

Sabine Hossenfelder on the possibility of human extinction

Correction to what I say at 11 mins 50 seconds: A supervolcano eruption ejects more than 1000 cubic kilometers of matter (not 1000 cubic meters). Sorry about that!

What do we know about the risks of human going extinct? In today's video I collect what we know about the frequency of natural disasters and just how they would kill us, and estimates for man-made disasters.

00:00 Intro
00:30 What Is an Existential Risk?
02:00 Would Extinction be Bad?
04:18 Man-made Disasters
10:36 What's The Risk of Man-made Disasters?
11:35 Natural Disasters
13:38 What's the Risk of Natural Disasters?
16:55 Why Can't the LHC Produce Black Holes? 19:29 Protect Your Data with NordVPN

Many thanks to Jordi Busqué for helping with this video http://jordibusque.com/

What about rogue AIs?

9:01: The biggest problem with Artificial Intelligence that longtermists see is that that an AI could become intelligent enough to survive independently of us but pursue interests that conflict with our own. They call it the “misalignment problem”. In the worst case, the AIs might decide to get rid of us. And could we really blame them? I mean, most of us can’t draw a flower, let alone a human face, so what’s the point of our existence really?

This wouldn’t necessarily be an extinction event in the sense that intelligent life   would still exist, it just wouldn’t be us. Under which circumstances you might consider an AI species a continuation of our own line is rather unclear. Longtermists argue it depends on whether the AI continues our “values”, but it seems odd to me to define a species by its values, and I’m not sure our values are all that great to begin with.

In any case, I consider this scenario unlikely because it assumes that advanced AIs will soon be easy to build and reproduce which is far from reality. If you look at what’s currently happening, supercomputers are getting bigger and bigger, and the bigger they get the more difficult they are to maintain and the longer it takes to train them. If you extrapolate the current trend to the next few hundred years, we will at best have a few intelligent machines owned by companies or governments, and each will require a big crew to keep it alive. They won’t take over the world any time soon.

The biggest existential risk?

19:23: Okay, so in summary, the biggest existential risk is our own stupidity.

Don’t read "Don't Look Up" too narrowly [Matt Yglesias] [Media Notes 65c]

The film’s creators say it’s a satire about climate change. But what do they know? Yglesias says:

If you insist on listening to the creators and seeing it as about climate, then while you might appreciate a few moments, I think you’ll mostly be annoyed and then start saying “but it’s not even funny” blah blah blah.

But that’s not the only way to read a text.

In policy terms, there’s not some sharp tradeoff between taking steps to minimize the risks of climate catastrophe and taking steps to minimize other kinds of catastrophes, and I don’t love framings that put it that way. But the use of a story about a comet collision as a metaphor for climate change — which I actually think works really well as a direct lesson about the risk of a comet hitting the planet as depicted in the film — struck me as funny. And I really encourage people to watch it with an open mind and see it as part of the cinema of existential risk and not just quibble about climate change.

I agree. It works as a story about a collision with a comet, but it works best if we read it more broadly, much more broadly.

On climate change:

The fundamental problem of climate change is that it involves asking people to make changes now for the sake of preventing harms that occur largely in the future to people living in other countries. It’s a genuine problem from hell, and it’s not actually solved by understanding the science or believing the factual information. This is exactly why ideas like McKay’s Manhattan Project [on carbon capture] are so important. While there is a lot we can do to improve the situation with more aggressive deployment of the technology we have, we also really do need more technological breakthroughs that will make lots of tradeoffs less painful and make progress easier.

Yglesias goes on to talk about the existential risk actually posed by comets, by supervolcanos, and but future pandemics.

Back to the film:

... in this case the message is much bigger than climate change. There is a range of often goofy-sounding threats to humanity that don’t track well onto our partisan cleavages or culture war battles other than that addressing them invariably involves some form of concerted action of the sort that conservatives tend to disparage. And this isn’t a coincidence. If existential threats were materializing all the time, we’d be dead and not streaming satirical films on Netflix. So the threats tend to sound “weird,” and if you talk a lot about them you’ll be “weird.” They don’t fit well into the grooves of ordinary political conflict because ordinary political conflict is about stuff that happens all the time.

So read Ord’s book “The Precipice” and find out all about it. Did you know that the Biological Weapons Convention has just four employees? I do because it’s in the book. Let’s maybe give them six?

For all that, though, I am genuinely shocked that the actual real-world emergence of SARS-Cov-2 has not caused more people to care about pandemic risk. The havoc that this pandemic has wreaked just in terms of economic harm and annoyance has been devastating.

There’s more.

First Steps

Photograph by Bill Benzon, poem by Sally Benzon, siblings all.

Words with Neil Armstrong

On the other side
of counting each night
wakened to sunrise,

here is the moment when
the shore by leave of its senses
appreciates being

the vigil: living or dead,
outstanding hands
whose sound in clapping

one by one is the toll
weathered of degrees
deep with listless light.

Somehow the moon,
the moon alone can speculate
how to crystallize

no one’s pictured face
seen in a breezy garden.
Melody calling of us

silent to the salt cast
of our reflections: You bear
seasoned bodies

whose shadows of sound
-- each and every thrall --
delights in tiding

the afterlight. Earthbound.
Forever the leap of moon
is home again. Again.

  O mercy! Love! Mercy!

Bits about Biology, Brains, and Us

1. Artificial computation on an organic substrate

From the article in XXI Century:

Researchers from Japan and the Michigan Technological University have succeeded in building a molecular computer that, more than any previous project of its kind, can replicate the inner mechanisms of the human brain, repairing itself and mimicking the massive parallelism that allows our brains to process information like no silicon-based computer can.

A relatively new technology, molecular electronics is an interdisciplinary pursuit that may very well prove the long-term solution to validate Moore’s law well into the next century. A molecular computer is made of organic molecules instead of silicon. Chips built this way are not only potentially much smaller but also, because of the way they can be networked, able to do things that no other traditional computer, regardless of its speed, can do.

The original article: Anirban Bandyopadhyay, Ranjit Pati, Satyajit Sahu1, Ferdinand Peper & Daisuke Fujita, Massively parallel computing on an organic molecular layer, Nature Physics 6, 369 - 375 (2010):

Modern computers operate at enormous speeds—capable of executing in excess of 10¹³ instructions per second—but their sequential approach to processing, by which logical operations are performed one after another, has remained unchanged since the 1950s. In contrast, although individual neurons of the human brain fire at around just 10³ times per second, the simultaneous collective action of millions of neurons enables them to complete certain tasks more efficiently than even the fastest supercomputer. Here we demonstrate an assembly of molecular switches that simultaneously interact to perform a variety of computational tasks including conventional digital logic, calculating Voronoi diagrams, and simulating natural phenomena such as heat diffusion and cancer growth. As well as representing a conceptual shift from serial-processing with static architectures, our parallel, dynamically reconfigurable approach could provide a means to solve otherwise intractable computational problems.

2. A woman's touch: Physical contact increases financial risk taking:

A woman's touch is all it takes for people to throw caution to the wind. That's the conclusion of a new study published online in Psychological Science, a journal of the Association for Psychological Science. If a female experimenter patted a participant on the back, they'd risk more money than if she just talked to them, or if a man did the patting. The researchers think this comes from the way that mothers use touch to make their babies feel secure.

I think that "explanation" needs a bit more work.

3. British Indian children have substantially better mental health than British Whites

Anna Goodman, Vikram Patel, and David A. Leon, Why do British Indian children have an apparent mental health advantage? Journal of Child Psychology and Psychiatry. Published online 26 April 2010. Abstract:

Background: Previous studies document a mental health advantage in British Indian children, particularly for externalising problems. The causes of this advantage are unknown.

Methods: Subjects were 13,836 White children and 361 Indian children aged 5–16 years from the English subsample of the British Child and Adolescent Mental Health Surveys. The primary mental health outcome was the parent Strengths and Difficulties Questionnaire (SDQ). Mental health was also assessed using the teacher and child SDQs; diagnostic interviews with parents, teachers and children; and multi-informant clinician-rated diagnoses. Multiple child, family, school and area factors were examined as possible mediators or confounders in explaining observed ethnic differences.

Results: Indian children had a large advantage for externalising problems and disorders, and little or no difference for internalising problems and disorders. This was observed across all mental health outcomes, including teacher-reported and diagnostic interview measures. Detailed psychometric analyses provided no suggestion of information bias. The Indian advantage for externalising problems was partly mediated by Indian children being more likely to live in two-parent families and less likely to have academic difficulties. Yet after adjusting for these and all other covariates, the unexplained Indian advantage only reduced by about a quarter (from 1.08 to .71 parent SDQ points) and remained highly significant (p < .001). This Indian advantage was largely confined to families of low socio-economic position.

Conclusion: The Indian mental health advantage is real and is specific to externalising problems. Family type and academic abilities mediate part of the advantage, but most is not explained by major risk factors. Likewise unexplained is the absence in Indian children of a socio-economic gradient in mental health. Further investigation of the Indian advantage may yield insights into novel ways to promote child mental health and child mental health equity in all ethnic groups.

4. Top 10 Visual Illusions of 2010

From the 6th annual Best Illusion of the Year contest at the Philharmonic Center for the Arts in Naples. Have fun looking at the videos.