Yellow tulip

Thoughts are an emergent property of brain activity. [working memory]

From the YouTube page:

Earl Miller runs the Miller Lab at MIT, where he studies how our brains carry out our executive functions, like working memory, attention, and decision-making. In particular he is interested in the role of the prefrontal cortex and how it coordinates with other brain areas to carry out these functions. During this episode, we talk broadly about how neuroscience has changed during Earl's career, and how his own thoughts have changed. One thing we focus on is the increasing appreciation of brain oscillations for our cognition. Recently on BI we've discussed oscillations quite a bit. In episode 153, Carolyn Dicey-Jennings discussed her philosophical ideas relating attention to the notion of the self, and she leans a lot on Earl's research to make that argument. In episode 160, Ole Jensen discussed his work in humans showing that low frequency oscillations exert a top-down control on incoming sensory stimuli, and this is directly in agreement with Earl's work over many years in nonhuman primates. So we continue that discussion relating low-frequency oscillations to executive control. We also discuss a new concept Earl has developed called spatial computing, which is an account of how brain oscillations can dictate where in various brain areas neural activity be on or off, and hence contribute or not to ongoing mental function. We also discuss working memory in particular, and a host of related topics.

0:00 - Intro
6:22 - Evolution of Earl's thinking
14:58 - Role of the prefrontal cortex
25:21 - Spatial computing
32:51 - Homunculus problem
35:34 - Self
37:40 - Dimensionality and thought
46:13 - Reductionism
47:38 - Working memory and capacity
1:01:45 - Capacity as a principle
1:05:44 - Silent synapses
1:10:16 - Subspaces in dynamics

I was a bit surprised to hear so much discussion of a shift in emphasis from single-neuron recording to recording activity in many neurons. Back in 1969 Karl Pribram published an article in Scientific American arguing that neural representation was based on holographic principles, which necessarily draws our attention to the activity of populations of neurons. Yes, I read about the infamous "grandmother" cell, and I read lots of work reporting the results of single neuron recordings, but I hadn't realized how long that persisted as the dominant paradigm.

Cherry blossoms

Teaching a humanoid robot to move around in the world is difficult and challenging

From the YouTube page:

Robert Playter is CEO of Boston Dynamics, a legendary robotics company that over 30 years has created some of the most elegant, dextrous, and simply amazing robots ever built, including the humanoid robot Atlas and the robot dog Spot.

This is a completely different world from large language models. It took 15 years for Boston Dynamics to get its Atlas robot to produce a natural looking walk. This discussion is worth viewing and thinking about. Figuring out how to get a robot to move is at least as intellectually challenging as getting an LLM to produce coherent and sensible prose. One might even argue that it is more challenging. At this point getting LLMs to produce coherent prose is not difficult. Multiple-column multiplication is difficult; eliminating confabulation is difficult; but mere prose production is not. But for some reason we don't know how to calibrate the difficulty of that behavior and so are prone to overvalue the significance of what the LLM is doing. But we are unlikely to view the movements of a humanoid robot and conclude that it's only a hop-skip-and-jump from playing a competent game of basketball.

On predictive control (c. 24:38):

Robert Playter: yeah those things have to run pretty quickly

Lex Fridman: what's the challenge of running things pretty quickly a thousand Hertz of acting and sensing quickly

RP: you know there's a few different layers of that you you want at the lowest level you like to run things typically at around a thousand Hertz which means that you know at each joint of the robot you're measuring position or force and then trying to control your actuator whether it's a hydraulic or electric motor trying to control the force coming out of that actuator and you want to do that really fast something like a thousand Hertz and that means you can't have too much calculation going on at that joint um but that's pretty manageable these days and it's fairly common

and then there's another layer that you're probably calculating you know maybe at 100 Hertz maybe 10 times slower which is now starting to look at the overall body motion and thinking about the the larger physics of of the uh of the robot

and then there's yet another loop that's probably happening a little bit slower which is where you start to bring you know your perception and your vision and things like that and so you need to run all of these Loops sort of simultaneously you do have to manage your your computer time so that you can squeeze in all the calculations you need in real time in a very consistent way

Look at the driver peeking at us from behind & through the daffodils.

Ellie Pavlick: The Mind of a Language Model {good stuff!}

From the YouTube page:

Ellie Pavlick runs her Language Understanding and Representation Lab at Brown University, where she studies lots of topics related to language. In AI, large language models, sometimes called foundation models, are all the rage these days, with their ability to generate convincing language, although they still make plenty of mistakes. One of the things Ellie is interested in is how these models work, what kinds of representations are being generated in them to produce the language they produce. So we discuss how she's going about studying these models. For example, probing them to see whether something symbolic-like might be implemented in the models, even though they are the deep learning neural network type, which aren't suppose to be able to work in a symbol-like manner. We also discuss whether grounding is required for language understanding - that is, whether a model that produces language well needs to connect with the real world to actually understand the text it generates. We talk about what language is for, the current limitations of large language models, how the models compare to humans, and a lot more.

0:00 - Intro
2:34 - Will LLMs make us dumb?
9:01 - Evolution of language
17:10 - Changing views on language
22:39 - Semantics, grounding, meaning
37:40 - LLMs, humans, and prediction
41:19 - How to evaluate LLMs
51:08 - Structure, semantics, and symbols in models
1:00:08 - Dimensionality
1:02:08 - Limitations of LLMs
1:07:47 - What do linguists think?
1:14:23 - What is language for?

Addendum, 4.26.23: Here's another video of Pavlick's work:

Addendum, 5.15.23: Here's yet another video of Pavlick's work:

Red tulips askew

Metaphysics and Large Language Models @ 3 Quarks Daily

I’ve got another article at 3 Quarks Daily:

World, Mind, Learnability, Large Language Models, and the Metaphysical Structure of the Cosmos

I take another crack at something I’ve looked at before, the idea that LLMs embody a metaphysics of the cosmos, whatever those are. We’re not dealing with the physical structure of the cosmos, we’re not dealing with the totality of things-as-such, but rather with the (conceptual) structures through which we apprehend them. Given the wide range of texts that have been digested and distilled into these models, their internal structure somehow is responsive to, reflects and refracts, those structures. 

Let me emphasize: Foundational LLMs implicate the metaphysical structure of the cosmos.

I set up the argument by starting with the 1977 short film, Powers of Ten, which traverses the cosmos from very large to very small in the compass of nine minutes. Then I ride one of my favorite hobby horses, Disney’s Fantasia, which also encompasses the cosmos. After that I ask: What must the world be like so that we can learn its structure?

And so forth and so on.

* * * * *

In the course of working on this piece I decided to ask GPT-4 about the film, Powers of Ten. Of course it knew about the film. I then asked to summarize what that film shows. It did so and with great confidence. And it got it wrong. First, I present the dialog I had with GPT-4, and then I present a summary of the film from Wikipedia. That summary, cursory though it is, makes it clear that GPT-4’s summary is incorrect. The Chatster was up to its old tricks, just making stuff up. I didn’t make any effort to nail down all the errors in GPT-4’s summary.

* * * * *

[April 22, 2023 – 3:15PM – Mar 23 version]

Do you know about the short 1977 film “Powers of Ten”?

Yes, I am familiar with the short film "Powers of Ten." It was directed by Ray and Charles Eames and was released in 1977. The film explores the relative sizes of things in the universe, from the scale of the observable universe all the way down to the subatomic level.

The film starts with an aerial view of a man and woman having a picnic in a park in Chicago, and then zooms out to show the Earth from a distance of ten meters, then ten times farther, and so on, until it reaches the limit of the observable universe. It then zooms back in, but this time at a smaller and smaller scale, revealing the structure of atoms and subatomic particles.

The film is a classic of science education and has been widely praised for its innovative use of scale and its ability to convey complex ideas in a clear and accessible way. It has been used in classrooms and science museums around the world, and is often cited as an example of effective science communication.

Could you talk your way through the first part of the film, starting with the couple and zooming out? What do we see at each higher power of 10?

Sure, I'd be happy to describe the first part of the film "Powers of Ten" to you.

The film begins with a shot of a couple, a man and a woman, having a picnic on a sunny day in a park in Chicago. The camera then begins to zoom out from the couple, at a rate of 10 times the distance every 10 seconds. As the camera zooms out, we see the park getting smaller and smaller, until it disappears from view.

At the first power of ten (10^1), we see the view of the park from an altitude of 10 meters above the couple. We can see the couple clearly, as well as the surrounding park and nearby buildings.

At the second power of ten (10^2), the camera has zoomed out to an altitude of 100 meters. Now we can see not just the park, but also the surrounding neighborhoods and the city of Chicago in the distance.

At the third power of ten (10^3), the camera has zoomed out to an altitude of 1,000 meters, or one kilometer. Now we can see not just the city of Chicago, but also the entire Lake Michigan shoreline and beyond.

At the fourth power of ten (10^4), the camera has zoomed out to an altitude of 10,000 meters, or 10 kilometers. Now we can see not just the city of Chicago and Lake Michigan, but also the entire state of Illinois and neighboring states.

At the fifth power of ten (10^5), the camera has zoomed out to an altitude of 100,000 meters, or 100 kilometers. Now we can see not just the Midwest region of the United States, but also the entire continent of North America.

At the sixth power of ten (10^6), the camera has zoomed out to an altitude of 1,000,000 meters, or 1,000 kilometers. Now we can see not just North America, but also the entire planet Earth.

At the seventh power of ten (10^7), the camera has zoomed out to an altitude of 10,000,000 meters, or 10,000 kilometers. Now we can see not just the Earth, but also the entire solar system, with the planets orbiting the sun.

At the eighth power of ten (10^8), the camera has zoomed out to an altitude of 100,000,000 meters, or 100,000 kilometers. Now we can see not just the solar system, but also the entire Milky Way galaxy, with its hundreds of billions of stars and countless other celestial objects.

At the ninth power of ten (10^9), the camera has zoomed out to an altitude of 1,000,000,000 meters, or 1 billion kilometers. Now we can see not just the Milky Way galaxy, but also the entire Local Group of galaxies, a cluster of more than 50 galaxies that includes the Milky Way.

At the tenth power of ten (10^10), the camera has zoomed out to an altitude of 10,000,000,000 meters, or 10 billion kilometers. Now we can see not just the Local Group of galaxies, but also the entire observable universe, with its hundreds of billions of galaxies and vast expanses of empty space.

That's a brief overview of the first part of "Powers of Ten." The film is an excellent way to explore the vastness of the universe and the different scales at which we can observe it.

From the Poetics: Beginning, middle, end

How Not To Destroy the World With AI - Stuart Russell

From the YouTube page:

About Talk:

It is reasonable to expect that artificial intelligence (AI) capabilities will eventually exceed those of humans across a range of real-world decision-making scenarios. Should this be a cause for concern, as Alan Turing and others have suggested? Will we lose control over our future? Or will AI complement and augment human intelligence in beneficial ways? It turns out that both views are correct, but they are talking about completely different forms of AI. To achieve the positive outcome, a fundamental reorientation of the field is required. Instead of building systems that optimize arbitrary objectives, we need to learn how to build systems that will, in fact, be beneficial for us. Russell will argue that this is possible as well as necessary. The new approach to AI opens up many avenues for research and brings into sharp focus several questions at the foundations of moral philosophy.

About Speaker:

Stuart Russell, OBE, is a professor of computer science at the University of California, Berkeley, and an honorary fellow of Wadham College at the University of Oxford. He is a leading researcher in artificial intelligence and the author, with Peter Norvig, of “Artificial Intelligence: A Modern Approach,” the standard text in the field. He has been active in arms control for nuclear and autonomous weapons. His latest book, “Human Compatible,” addresses the long-term impact of AI on humanity.

How do we get the machine to assist humans? (c. 36:26):

So we need to actually to get rid of the standard model. So we need a different model, right? This is the standard model. Machines are intelligent to the extent their actions can be expected to achieve their objectives.

Instead, we need the machines to be beneficial to us, right? We don't want this sort of pure intelligence that once it has the objective is off doing its thing, right? We want the systems to be beneficial, meaning that their actions can be expected to achieve our objectives.

And how do we do that? [...] That you do not build in a fixed known objective upfront. Instead, the machine knows that it doesn't know what the objective is, but it still needs a way of grounding its choices over the long run.

And the evidence about human preferences will say flows from human behavior. [...] So we call this an assistance game. So it's a, involves at least one person, at least one machine, and the machine is designed to be of assistance to the human. [...] The key point is there's a priori uncertainty about what those utility functions are. So it's gotta optimize something, but it doesn't know what it is.

And during, you know, if you solve the game, you in principle, you can just solve these games offline and then look at the solution and how it behaves. And as the solution unfolds effectively, information about the human utilities is flowing at runtime based on the human actions. And the humans can do deliberate actions to try to convey information, and that's part of the solution of the game. They can give commands, they can prohibit you from doing things, they can reward you for doing the right thing. [...]

So in some sense, you know, the entire record, the written record of humanity is, is a record of humans doing things and other people being upset about it, right? All of that information is useful for understanding what human preference structures really are algorithmically.

Yeah, we, you know, we can solve these and in fact, the, the one machine, one human game can be reduced to a partially observable MDP.

And for small versions of that we can solve it exactly. And actually look at the equilibrium of the game and, and how the agents behave. But an important point here and, the word alignment often is used in, in discussing these kinds of things.

And as Ken mentioned, it's related to inverse reinforcement learning, the learning of human preference structures by observing behavior. But alignment gives you this idea that we're gonna align the machine and the human and then off they go, right? That's never going to happen in practice.

The machines are always going to have a considerable uncertainty about human preference structures, right? Partly because there are just whole areas of the universe where there's no experience and no evidence from human behavior about how we would behave or how we would choose in those circumstances. And of course, you know, we don't know our own preferences in those areas. [...]

So when you look at these solutions, how does the robot behave? If it's playing this game, it actually defers to human requests and commands. It behaves cautiously because it doesn't wanna mess with parts of the world where it's not sure about your preferences. In the extreme case, it's willing to be switched off.

So I'm gonna have, in the interest of time, I'm gonna have to skip over the proof of that, which is prove with a little, a little game. But basically we can show very straightforwardly that as long as the robot is uncertain about how the human is going to choose, then it has a positive incentive to allow itself to be switched off, right? It gains information by leaving that choice available for the human. And it only closes off that choice when it has, well, or at least when it believes it has perfect knowledge of human preferences.

Indexical goals (51:33):

One might initially think, well, you know what they're doing. If they're learning to imitate humans, then, then maybe actually, you know, almost coincidentally that will end up with them being aligned with what humans want. All right? So perhaps we accidentally are solving the alignment problem here, by the way we're training these systems. And the answer to that is it depends. It depends on the type of goal that gets learned.

And I'll distinguish two types of goals. There's what we call common goals where things like painting the wool or mitigating climate change where if you do it, I'm happy if I do it, you are happy, we're all happy, right? These are goals where any agent doing these things would make all the agents happy.

Then there are indexical goals, which are meaning indexical to the individual who has the goal. So drinking coffee, right? I'm not happy if the robot drinks the coffee, right? What I want to have happen is if I'm drinking coffee and the robot does some inverse reinforcement, Hey, Stuart likes coffee, I'll make Stuart a cup of coffee in the morning. The robot drinking a coffee is not the same, right?

So this is what we mean by an indexable goal and becoming ruler of the universe, right? Is not the same if it's me versus the robot. Okay? And obviously if systems are learning indexical goals, that's arbitrarily bad as they get more and more capable, okay? And unfortunately, humans have a lot of indexical goals. We do not want AI systems to learn from humans in this way.

Imitation learning is not alignment.

Pink tulip

Michael Jordan: How AI Fails Us, and How Economics Can Help

Jordan argues that AI, which is mostly machine learning these days, remains dominated by the Dr. Frankenstein notion of creating an artificial human. He regards that as a mistake, and argues for a more collective approach. (Cf. this post from two years ago, Beyond "AI" – toward a new engineering discipline.)

From the YouTube page:

Artificial intelligence (AI) has focused on a paradigm in which intelligence inheres in a single agent, and in which agents should be autonomous so they can exhibit intelligence independent of human intelligence. Thus, when AI systems are deployed in social contexts, the overall design is often naive. Such a paradigm need not be dominant. In a broader framing, agents are active and cooperative, and they wish to obtain value from participation in learning-based systems. Agents may supply data and resources to the system, only if it is in their interest. Critically, intelligence inheres as much in the system as it does in individual agents. This perspective is familiar to economics researchers, and a first goal in this work is to bring economics into contact with computer science and statistics. The long-term goal is to provide a broader conceptual foundation for emerging real-world AI systems, and to upend received wisdom in the computational, economic and inferential disciplines.

Michael I. Jordan is the Pehong Chen Distinguished Professor in the departments of electrical engineering and computer science and of statistics at the University of California, Berkeley. His research interests bridge the computational, statistical, cognitive, biological and social sciences. Jordan is a member of the National Academy of Sciences, the National Academy of Engineering, and the American Academy of Arts and Sciences, and a foreign member of the Royal Society. He was a plenary lecturer at the International Congress of Mathematicians in 2018. He received the Ulf Grenander Prize from the American Mathematical Society in 2021, the IEEE John von Neumann Medal in 2020, the IJCAI Research Excellence Award in 2016, the David E. Rumelhart Prize from the Cognitive Science Society in 2015 and the ACM/AAAI Allen Newell Award in 2009.

Two slides from the presentation:

Here’s a paper where Jordan is one of the authors:

By Divya Siddarth, Daron Acemoglu, Danielle Allen, Kate Crawford, James Evans, Michael Jordan, E. Glen Weyl, How AI Fails Us, Edmond J. Safra Center for Ethics and Carr Center for Human Rights Policy, Harvard University (December 1, 2021).

Abstract:

The dominant vision of artificial intelligence imagines a future of large-scale autonomous systems outperforming humans in an increasing range of fields. This “actually existing AI” vision misconstrues intelligence as autonomous rather than social and relational. It is both unproductive and dangerous, optimizing for artificial metrics of human replication rather than for systemic augmentation, and tending to concentrate power, resources, and decision-making in an engineering elite.  Alternative visions based on participating in and augmenting human creativity and cooperation have a long history and underlie many celebrated digital technologies such as personal computers and the internet.  Researchers and funders should redirect focus from centralized autonomous general intelligence to a plurality of established and emerging approaches that extend cooperative and augmentative traditions as seen in successes such as Taiwan’s digital democracy project to collective intelligence platforms like Wikipedia. We conclude with a concrete set of recommendations and a survey of alternative traditions.

Sign, white on black

Another crazy/interesting video about AGI and the future [Goertzel]

I've been aware of Ben Goertzel since, I believe, the 1980s or 1990s, when I read and article he published in The Journal of Social and Evolutionary Systems, a journal where David Hays and I published regularly. He's a researcher in artificial intelligence who thinks that his team may well produce AGI and that AGI isn't all that far in the future.

He's working entirely outside the world of Big AI in the USofA, Google, Microsoft, OpenAI, Facebook, etc. He's skeptical about the larger claims being made about GPT systems, a skepticism I feel is warranted, but sees them as nonetheless interesting and useful. I agree with that as well.

It's interesting comparing his remarks with those of John Veraeke (immediately below). At one point while listening to him I brought myself to the edge of thinking about actually creating or encountering an artificial mind that warrants moral standing and rights. That's an uncanny feeling. We're getting closer to that. How much closer....Who knows.

From the YouTube page:

Today we’re joined by Ben Goertzel, CEO of SingularityNET. In our conversation with Ben, we explore all things AGI, including the potential scenarios that could arise with the advent of AGI and his preference for a decentralized rollout comparable to the internet or Linux. Ben shares his research in bridging neural nets, symbolic logic engines, and evolutionary programming engines to develop a common mathematical framework for AI paradigms. We also discuss the limitations of Large Language Models (LLMs) and the potential of hybridizing LLMs with other AGI approaches. Additionally, we chat about their work using LLMs for music generation and the limitations of formalizing creativity. Finally, Ben discusses his team's work with the OpenCog Hyperon framework and Simuli to achieve AGI, and the potential implications of their research in the future.

Chapters:

05:16 - AGI and Sentience
08:52 - Current Large Language Models and the Path to AGI
17:33 - Large Language Models Limited in Theory of Mind
22:07 - Exploring the Potential of Music LM Models
27:06 - AGI potential paths: Neuroscience vs. Mathematical Fusion
31:29 - OpenCog Hyperon: Rebuilding AI Infrastructure at Scale
35:44 - Advancing Towards Truth GPT and AGI Breakthrough
42:33 - The Complexities Behind Sophia's Dialogue Responses
53:08 - General Intelligence and Agency: Tightly Linked?
57:58 - The Implications of AGI Breakthrough: Decentralization Needed

An interesting video about the implications of current AI (GPT) [Vervaeke]

I'm only thirty minutes into the this, but I'm posting it, though I've got doubts. The doubts? The language and the framing, including the fact that Vervaeke declares himself to be an "Internationally Acclaimed Cognitive Scientist." Maybe he is, maybe he isn't, but a quick web search leaves me skeptical. In any event, that's the kind of appellation best left in the hands of third parties. To use it yourself is, at best, tacky.

And then we have those moments when Vervaeke lapses into the third person when referring to himself. What?

Still, Vervaeke does recognize the power and potential of the technology while at the same time seeing that it presents a deep challenge to our sense of who and what we are and that that challenge itself is a source of confusion and danger leading to a mis-evaluation and misuse of the technology. That's worth listening to.

Quick notes 

1) Too much talk of autopoesis and emergence for my taste. I tend to think of those as stand-ins for things we don't understand.

2) Makes an interesting distinction between intelligence and rationality. Intelligence seems to be a capacity while rationality is learned and can be developed. 

3) Vervaeke and his cohorts are worried that the pornography industry and the military will be the primary drivers of embodied AI. 

4) In a complex world, trade-offs are inevitable. AIs cannot avoid them. Moreover, internal coherence will be an issue.

5) "Reason is about how we bind ourselves to ourselves and to each other so we can be bound to the world."

6) "Don't try and code into them rules and values. We need to be able at some point to answer this question in deep humility and deep truth: What would it be for these machines to flourish for themselves?"

7) "I think the theological response is ultimately what is needed here." [What do I think about this? Hmmmm.]

8) Compare Vervaeke's remarks with those of Ben Goertzel.

* * * * *

All the text below is taken from the YouTube page:

AI: The Coming Thresholds and The Path We Must Take | Internationally Acclaimed Cognitive Scientist

Dr. John Vervaeke lays out a multifaceted argument discussing the potential uses, thresholds, and calamities that may occur due to the increase in artificial intelligence systems. While there is a lot of mention of GPT and other chatbots, this argument is meant to be seen as confronting the principles of AI, AGI, and any other forms of Artificial Intelligence.

First, Dr. Vervaeke lays out an overview of his argument while also contextualizing the conversation. Dr. Vervaeke then explores the scientific ramifications and potentialities. Lastly, Dr. Vervaeke concludes in the philosophical realm and ends the argument with a strong and stern message that we face a kairos, potentially the greatest that the world has ever seen.

Dr. Vervaeke is also joined in this video essay by Ryan Barton, the Executive Director of the Vervaeke Foundation, as well as Eric Foster, the Media Director at the Vervaeke Foundation. 

* * * * *

Addendum: 5.25.23: Further thoughts about the moral challenges posed by AI.