Lex Fridman talks with Ted Gibson about language, LLMs, and other things [i.e. Linguistics 101 for LLMs]

This is a long podcast and I’ve not listened to all of it. I’m excerpting part of the conversation – which I’ve taken from this transcript – which speak to two of my hobby horses: 1) the apparent lack of linguistic knowledge in the LLM community, and 2) the idea that LLMs are built on relationships between words. On the first point, I’m using Lex Fridman as a proxy for the LLM community, though he does not work on LLMs. But he is trained in computer science and machine learning, is generally familiar with LLMs, and has interviewed a number of experts in machine learning. I was just a little surprised to hear that the idea of sentence structure being tree-like seemed new to him. I thought “everyone” knew that, where by “everyone” I mean anyone in the last 50 to 60 years with a technical interest in how the mind words.

The second point is where things get interesting. In the discussion of language Gibson hammers home the distinction between form and meaning in language. And then, in discussing LLMS, he talks about them as being based on language forms, but not meaning. I think that’s right – keeping in mind that I use those terms a bit differently than they’re being used here (see this post, for example), but not in a way that’s inconsistent with what I believe Gibson is saying.

NOTE: The machine-generated transcript the transcript has the names reversed (as of today, Apr. 22), so I have corrected that. Nor have I checked it against the video for accuracy.

Linguistic knowledge and dependency theory

Let’s start with a bit from the beginning of the conversation:

LEX FRIDMAN: (00:03:23) Did you ever come across the philosophy angle of logic? If you think about the 80s with AI, the expert systems where you try to maybe sidestep the poetry of language and some of the syntax and the grammar and all that kinda’ stuff and go to the underlying meaning that language is trying to communicate and try to somehow compress that in a computer representable way? Did you ever come across that in your studies?

EDWARD GIBSON: (00:03:50) I probably did but I wasn’t as interested in it. I was trying to do the easier problems first, the ones I thought maybe were handleable, which seems like the syntax is easier, which is just the forms as opposed to the meaning. When you’re starting talking about the meaning, that’s a very hard problem and it still is a really, really hard problem. But the forms is easier. And so I thought at least figuring out the forms of human language, which sounds really hard but is actually maybe more attractable.

LEX FRIDMAN: (00:04:19) It’s interesting. You think there is a big divide, there’s a gap, there’s a distance between form and meaning, because that’s a question you have discussed a lot with LLMs because they’re damn good at form.

EDWARD GIBSON: (00:04:33) Yeah, I think that’s what they’re good at, is form. And that’s why they’re good, because they can do form, meanings are …

LEX FRIDMAN: (00:04:39) Do you think there’s … Oh, wow. It’s an open question.

EDWARD GIBSON: (00:04:42) Yeah.

LEX FRIDMAN: (00:04:43) How close form and meaning are. We’ll discuss it but to me studying form, maybe it’s a romantic notion it gives you. Form is the shadow of the bigger meaning thing underlying language. Language is how we communicate ideas. We communicate with each other using language. In understanding the structure of that communication, I think you start to understand the structure of thought and the structure of meaning behind those thoughts and communication, to me. But to you, big gap.

This is basic, very basic. It’s not that “form is the shadow of the bigger meaning” but that syntax is (based on) the form of meaning, relationships between semantic elements.

Now we’re a bit later in the conversation where Gibson is talking about syntactic dependency.

LEX FRIDMAN: (00:10:59) [...] There’s so many things I want to ask you. Okay, let me just some basics. You mentioned dependencies a few times. What do you mean by dependencies?

EDWARD GIBSON: (00:11:12) Well, what I mean is in language, there’s three components to the structure of language. One is the sounds. Cat is C, A and T in English. I’m not talking about that part. Then there’s two meaning parts, and those are the words. And you were talking about meaning earlier. Words have a form and they have a meaning associated with them. And so cat is a full form in English and it has a meaning associated with whatever a cat is. And then the combinations of words, that’s what I’ll call grammar or syntax, that’s when I have a combination like the cat or two cats, okay, where I take two different words there and put together and I get a compositional meaning from putting those two different words together. And so that’s the syntax. And in any sentence or utterance, whatever, I’m talking to you, you’re talking to me, we have a bunch of words and we’re putting them together in a sequence, it turns out they are connected, so that every word is connected to just one other word in that sentence. And so you end up with what’s called technically a tree, it’s a tree structure, where there’s a root of that utterance, of that sentence. And then there’s a bunch of dependents, like branches from that root that go down to the words. The words are the leaves in this metaphor for a tree.

LEX FRIDMAN: (00:12:34) A tree is also a mathematical construct.

EDWARD GIBSON: (00:12:37) Yeah. It’s graph theoretical thing, exactly.

LEX FRIDMAN:(00:12:38) A graph theory thing. It’s fascinating that you can break down a sentence into a tree and then every word is hanging onto another, is depending on it. [...]

LEX FRIDMAN: (00:13:05) Can I pause on that?

EDWARD GIBSON: (00:13:06) Sure.

LEX FRIDMAN: (00:13:06) Because to me just as a layman, it is surprising that you can break down sentences in mostly all languages.

Again, this is fundamental. I suppose I had something of a preview of this sort of thing when I learned Reed-Kellogg sentence diagramming in the sixth grade. It’s still kicking around – there’s lots of stuff about it on the web – but I don’t know how routinely it’s taught these days. I learned about syntactic trees in my sophomore year in college when I took a course in psycholinguistics.

LEX FRIDMAN: (00:18:22) I love the terminology of agent and patient and the other ones you used. Those are linguistic terms, correct?

EDWARD GIBSON: (00:18:29) Those are for meaning, those are meaning. And subject and object are generally used for position. Subject is just the thing that comes before the verb and the object is the one that comes after the verb. The agent is the thing doing, that’s what that means. The subject is often the person doing the action, the thing.

LEX FRIDMAN: (00:18:48) Okay, this is fascinating. How hard is it to form a tree in general? Is there a procedure to it? If you look at different languages, is it supposed to be a very natural … Is it automatable or is there some human genius involved in construction …

EDWARD GIBSON: (00:19:01) I think it’s pretty automatable at this point. People can figure out the words are. They can figure out the morphemes, technically morphemes are the minimal meaning units within a language, okay. And so when you say eats or drinks, it actually has two morphemes in English. There’s the root, which is the verb. And then there’s some ending on it which tells you that’s the third person singular.

I think that anyone working with LLMs should be conversant with the distinction between the meaning-bearing aspect of language and the positional aspect. They may not need this familiarity to work with transformers, but they should know that the distinction is basic to language mechanism. After all, the positionality of tokens is something that is central to the transformer architecture. They should know that it’s central to language itself and not just an aspect of the transformer architecture.

GPT, the magical collaboration zone, Lex Fridman and Sam Altman

I was making one more run around the web before I buckled down and got back to a major writing task, when I came across the brand-spanking-new conversation between Lex Fridman and Sam Altman. Lex is Lex, and an interesting guy, and Sam is, well, he's interesting to me, but – there was a hint of megalomania at the end of that NYTimes story from Mar. 31, 2023, that rubbed me the wrong way, and all the AI hype – he IS the CEO of OpenAI. So it seemed to me that I just had to listen in, not the whole thing – and I could legit play solitaire while listening – and so I did, skipping over stuff.

But then the conversation hit an interesting patch. So – and I'm not going to try to re-create the context – they're talking about GPT-4 at roughly 46:03:

Altman: what are the best things it can do

Fridman: what are the best things it can do and the the limits of those best things that allow you to say it sucks therefore gives you an inspiration and hope for the future

Altman: you know one thing I've been using it for more recently is sort of a like a brainstorming partner 

Fridman: Yep for that

Altman: there's a glimmer of something amazing in there
I don't think it gets you know
when people talk about it
it what it does they're like
ah it helps me code more productively
it helps me write more faster and better
it helps me you know translate from this language to another
all these like amazing things
but there's something about the like kind of creative brainstorming partner
I need to come up with a name for this thing
I need to like think about this problem in a different way
I'm not sure what to do here
uh that I think like gives a glimpse of something I hope to see more of
um one of the other things that you can see like a very small glimpse of is
when it can help on longer Horizon tasks
you know break down some multiple steps
maybe like execute some of those steps
search the internet
write code whatever put that together uh
when that works which is not very often
it's like very magical

At about 52:54:

Fridman: I use it as a reading partner for reading books
it helps me think
help me think through ideas especially when the books are classic
so it's really well written about and it actually is is I
I find it often to be significantly better than even like Wikipedia on well-covered topics
it's somehow more balanced and more nuanced or maybe it's me
but it inspires me to think deeper than a Wikipedia article does
I'm not exactly sure what that is
you mentioned like this collaboration I'm not sure where the magic is if it's in here [gestures to his head]
or if it's in there [points toward the table]
or if it's somewhere in between

It's that magic-collaborative zone that interests me. While I've spent a great deal of time working with (plain old) ChatGPT, most of that time I've been doing research on how it behaves. But every once in awhile I'll play around just to mess around. And then I've seen sparks of magic. The interaction that generated AGI and Beyond: A Whale of a Tale certainly had the magic flowing, and it showed up here and there during the Green Giant Chronicles. I suspect those two cases are somewhat idiosyncratic. Nor am I sure that I can do this at will. But there's definitely something there, and its in the interaction.

I would guess that the magic varies from person to person as well. I wonder how many uses have had these kind of magical flow interactive states? I'd thinking finding that out would be tricky because they're likely to be idiosyncratic and elusive. If I were to research it, I'd probably start out with interviews, either face-to-face or through some online medium. That might lead to a questionnaire that could be used more broadly.

It'll be interesting to see how Altman ends up characterizing this flow state – which is what I'm calling it for the moment, a man-machine flow state. It's the human, of course, that's in flow. The machine is just being the machine. 

* * * * * 

I have a final comment, of an epistemological nature. As the post indicates, I'd already had a magical interaction or two with ChatGPT before I listened to this podcast. The first time it came up in the podcast, from Altman, OK, I noted it. And went on, playing solitaire with one part of my mind and listening in on the podcast with another part. But then it came up again, this time from Fridman. Wham! That's three, my threshold number for this kind of thing. Three people independently have the same or similar experience. Maybe there's something real there.

Teaching a humanoid robot to move around in the world is difficult and challenging

From the YouTube page:

Robert Playter is CEO of Boston Dynamics, a legendary robotics company that over 30 years has created some of the most elegant, dextrous, and simply amazing robots ever built, including the humanoid robot Atlas and the robot dog Spot.

This is a completely different world from large language models. It took 15 years for Boston Dynamics to get its Atlas robot to produce a natural looking walk. This discussion is worth viewing and thinking about. Figuring out how to get a robot to move is at least as intellectually challenging as getting an LLM to produce coherent and sensible prose. One might even argue that it is more challenging. At this point getting LLMs to produce coherent prose is not difficult. Multiple-column multiplication is difficult; eliminating confabulation is difficult; but mere prose production is not. But for some reason we don't know how to calibrate the difficulty of that behavior and so are prone to overvalue the significance of what the LLM is doing. But we are unlikely to view the movements of a humanoid robot and conclude that it's only a hop-skip-and-jump from playing a competent game of basketball.

On predictive control (c. 24:38):

Robert Playter: yeah those things have to run pretty quickly

Lex Fridman: what's the challenge of running things pretty quickly a thousand Hertz of acting and sensing quickly

RP: you know there's a few different layers of that you you want at the lowest level you like to run things typically at around a thousand Hertz which means that you know at each joint of the robot you're measuring position or force and then trying to control your actuator whether it's a hydraulic or electric motor trying to control the force coming out of that actuator and you want to do that really fast something like a thousand Hertz and that means you can't have too much calculation going on at that joint um but that's pretty manageable these days and it's fairly common

and then there's another layer that you're probably calculating you know maybe at 100 Hertz maybe 10 times slower which is now starting to look at the overall body motion and thinking about the the larger physics of of the uh of the robot

and then there's yet another loop that's probably happening a little bit slower which is where you start to bring you know your perception and your vision and things like that and so you need to run all of these Loops sort of simultaneously you do have to manage your your computer time so that you can squeeze in all the calculations you need in real time in a very consistent way

Elemental Cognition is ready to deploy hybrid AI technology in practical systems

Steve Lohr, One Man's Dream of Fusing A.I. With Common Sense, NYTimes, Aug. 28, 2022

David Ferrucci is best-known as the researcher who led the team that developed IBM's Watson, which beat the best human players of Jeopardy in 2011. He left IBM a year later and formed his own company, Elemental Cognition, in 2015. Elemental cognition is taking a hybrid approach, combining aspects of machine learning and symbolic computation.

Elemental Cognition has recently developed a system that helps people plan and book round-the-world airline tickets:

The round-the-world ticket is a project for oneworld, an alliance of 13 airlines including American Airlines, British Airways, Qantas, Cathay Pacific and Japan Airlines. Its round-the-world tickets can have up to 16 different flights with stops of varying lengths over the course of a year.

Elemental Cognition supplies the technology behind a trip-planning intelligent agent on oneworld’s website. It was developed over the past year and introduced in April.

The user sees a global route map on the left and a chatbot dialogue begins on the right. A traveler starting from New York types in the desired locations — say, London, Rome and Tokyo. “OK,” replies the chatbot, “I have added London, Rome and Tokyo to the itinerary.”

Then, the customer wants to make changes — “add Paris before London,” and “replace Rome with Berlin.” That goes smoothly, too, before the system moves on to travel times and lengths of stays in each city.

Rob Gurney, chief executive of oneworld, is a former Qantas and British Airways executive familiar with the challenges of online travel planning and booking. Most chatbots are rigid systems that often repeat canned answers or make irrelevant suggestions, a frustrating “spiral of misery.”

Instead, Mr. Gurney said, the Elemental Cognition technology delivers a problem-solving dialogue on the fly. The rates of completing an itinerary online are three to four times higher than without the company’s software.

Elemental Cognition has developed an approach that all-but eliminates the hand-coding typcial of symbolic A.I.:

For example, the rules and options for a global airline ticket are spelled out in many pages of documents, which are scanned.

Dr. Ferrucci and his team use machine learning algorithms to convert them into suggested statements in a form a computer can interpret. Those statements can be facts, concepts, rules or relationships: Qantas is an airline, for example. When a person says “go to” a city, that means add a flight to that city. If a traveler adds four more destinations, that adds a certain amount to the cost of the ticket.

In training the round-the-world ticket assistant, an airline expert reviews the computer-generated statements, as a final check. The process eliminates most of the need for hand coding knowledge into a computer, a crippling handicap of the old expert systems.

There's more at the link.

* * * * *

Lex Fridman interviews David Ferrucci (2019).

0:00 - Introduction
1:06 - Biological vs computer systems
8:03 - What is intelligence?
31:49 - Knowledge frameworks
52:02 - IBM Watson winning Jeopardy
1:24:21 - Watson vs human difference in approach
1:27:52 - Q&A vs dialogue
1:35:22 - Humor
1:41:33 - Good test of intelligence
1:46:36 - AlphaZero, AlphaStar accomplishments
1:51:29 - Explainability, induction, deduction in medical diagnosis
1:59:34 - Grand challenges
2:04:03 - Consciousness
2:08:26 - Timeline for AGI
2:13:55 - Embodied AI
2:17:07 - Love and companionship
2:18:06 - Concerns about AI
2:21:56 - Discussion with AGI

Fools rush in.... We're about six insights away from AGI, says John Carmack [& I've got perpetual motion figured out]

Lex Fridman interviews John Carmack. At about 1:50 Carmack says:

I am not a madman for saying that it is likely that the code for artificial general intelligence is going to be tens of thousands of line of code not millions of lines of code. This is code that conceivably one individual could write, unlike writing a new web browser or operating system and, based on the progress that AI as machine learning had made in the recent decade, it's likely that the important things that we don't know are relatively simple. There's probably a handful of things and my bet is I think there's less than six key insights that need to be made. Each one of them can probably be written on the back of an envelope. We don't know what they are, but when they're put together in concert with GPUs at scale and the data that we all have access to, that we can make something that behaves like a human being or like a living creature and that can then be educated in whatever ways that we need to get to the point where we can have universal remote works where anything that somebody does mediated by a computer and doesn't require physical interaction, that an AGI will be able to do.

He also believes that antecedents of all the critical ideas are already in the literature, but have been lost.

On the six-or-less insights, I'm between agnostic and deeply skeptical (he doesn't know what he's talking about). But on the idea that the existing literature contains important insights that have been lost, that's likely true. My favorite example is Miriam Yevick's 1975 paper, Holographic or Fourier Logic, Pattern Recognition 7, 1975, pp. 197-213. FWIW, that article was published five years after Carmack was born.

Jim Keller: Moore's Law, Microprocessors, Abstractions, and First Principles | AI Podcast [Lex Fridman]

Program notes:

Jim Keller is a legendary microprocessor engineer, having worked at AMD, Apple, Tesla, and now Intel. He's known for his work on the AMD K7, K8, K12 and Zen microarchitectures, Apple A4, A5 processors, and co-author of the specifications for the x86-64 instruction set and HyperTransport interconnect. This conversation is part of the Artificial Intelligence podcast.

0:00 - Introduction
2:12 - Difference between a computer and a human brain
3:43 - Computer abstraction layers and parallelism
17:53 - If you run a program multiple times, do you always get the same answer?
20:43 - Building computers and teams of people
22:41 - Start from scratch every 5 years
30:05 - Moore's law is not dead
55:47 - Is superintelligence the next layer of abstraction?
1:00:02 - Is the universe a computer?
1:03:00 - Ray Kurzweil and exponential improvement in technology
1:04:33 - Elon Musk and Tesla Autopilot
1:20:51 - Lessons from working with Elon Musk
1:28:33 - Existential threats from AI
1:32:38 - Happiness and the meaning of life

I found this conversation utterly fascination – though, truth be told, I also played solitaire while listening. Much of it just whizzed by, but that's OK.  Keller designs microprocessors, and has lived through revolutions in processor design. He actually thinks about transistor size in terms of numbers of atoms. Anyone who thinks about computing needs to think about it as a physical process, even if the conversation just whizzes by.

04:00 - 21:00: Starts talking about layers of abstraction at roughly 04:00 and continues. Interesting soundbite: you can execute a program 100 times and get the same answer each time, but have 100 different execution paths.