Four-year renovation of an abandoned Japanese farmhouse [time-lapse]

Is an AI chill on the horizon?

Called it. https://t.co/IZLLKFgxhD

— Gary Marcus (@GaryMarcus) September 22, 2023

Yeah, he called it. I just wish he weren't so damned self-righteous about it.

Nor is Marcus the only one who understood clearly that the models produced by generative AI have severe limitations. They give us new territory to explore. That's exciting. But, as such, they constitute a digital wilderness. Wilderness must be 'tamed' or 'domesticated' before it can be put to use. That work as yet to be done. In fact, the rush to exploit generative AI takes resources away from research that would be useful in understanding the wilderness.

Friday Fotos: What the hey? [Hoboken all the way]

Better at chess, still sucks at planning [look at the red box]

The new GPT3.5 completion version and the claims on its chess prowess caught my students' attention--and they tested to see if it got better at, you know, planning..

tldr; Nope--RLHF didn't decimate the planning sparks of GPT3.5--it apparently had none to begin with..😅 https://t.co/JxtUqH1XCO

— Subbarao Kambhampati (కంభంపాటి సుబ్బారావు) (@rao2z) September 22, 2023

* * * * *

In related news, Subbarao Kambhampati, Can LLMs Really Reason and Plan? BLOG@CACM, September 12, 2023.

Second paragraph:

Nothing in the training and use of LLMs would seem to suggest remotely that they can do any type of principled reasoning (which, as we know, often involves computationally hard inference/search). While one can dismiss the claims by hype-fueled social media influencers, startup-founders and VC's, it is hard to ignore when there are also peer-reviewed papers in top conferences making similar claims. The "Large Language Models are Zero-Shot " is almost becoming a meme paper title! At some level, this trend is understandable as in the era of LLMs, AI has become a form of ersatz natural science–driven by observational studies of capabilities of these behemoth systems.

In conclusion:

The fact that LLMs are often good at extracting planning knowledge can indeed be gainfully leveraged. As we have argued in our recent work, LLMs can thus be a rich source of approximate models of world/domain dynamics and user preferences,, as long as the humans (and any specialized critics) in the loop verify and refine those models, and give them over to model-based solvers. This way of using LLMs has the advantage that the humans need only be present when the dynamics/preference model is being teased out and refined, and the actual planning after that can be left to planning algorithms with correctness guarantees (modulo the input model). Such a framework has striking similarities to knowledge-based AI systems of yore, with LLMs effectively replacing the "knowledge engineer." Given the rather quixotic and dogmatic shift of AI away from approaches that accept domain knowledge from human experts, something I bemoaned in Polanyi's Revenge, this new trend of using LLMs as knowledge sources can be viewed as a form of avenging Polanyi's revenge! Indeed, LLMs make it easy to get problem-specific knowledge as long as we are willing to relax correctness requirements of that knowledge. In contrast to the old knowledge engineering approaches, LLMs offer this without making it look like we are inconveniencing any specific human (we are, instead, just leveraging everything humans told each other!). So the million dollar question for reasoning tasks is: "how would you do planning if you have some doddering know-it-all ready to give you any kind of knowledge?" Traditional approaches to model-based reasoning/planning that focus on the incompleteness and incorrectness of the said models (such as model-lite planning, robust planning) can have fresh relevance.

To summarize, nothing that I have read, verified or done gives me any compelling reason to believe that LLMs do reasoning/planning as it is normally understood. What they do, armed with their web-scale training, is a form of universal approximate retrieval which, as we have argued, can sometimes be mistaken for reasoning capabilities. LLMs do excel in idea generation for any task–including those involving reasoning, and as I pointed out, this can be effectively leveraged to support reasoning/planning. In other words, LLMs already have enough amazing approximate retrieval abilities that we can gainfully leverage, that we don't need to ascribe fake reasoning/planning capabilities to them.

There's more at the link.

AI systems as infrastructure, infrastructure as open source

AI systems are fast becoming a basic infrastructure.
Historically, basic infrastructure always ends up being open source (think of the software infra of the internet, including Linux, Apache, JavaScript and browser engines, etc)
It's the only way to make it reliable, secure, and… https://t.co/p5jBkKQgyq

— Yann LeCun (@ylecun) September 21, 2023

Are we feeling queasy yet?

Chain-of-verification for LLMs reduces hallucination

- LLMs like ChatGPT & Llama are prone to hallucinate in longform generation

- Our method generates short questions that check facts in the full generation. These are answered correctly more often & are used to generate an improved response.

(2/4)🧵 pic.twitter.com/03NF9pQQQT

— Jason Weston (@jaseweston) September 21, 2023

Factored CoVe: make sure in step (3) the LLM doesn't attend to results of (1) so hallucinations aren't copied

Factor+Revise: adds cross-checks between steps

Overall, CoVe gives large gains in multiple tasks.

Read the paper for more (hopefully non-hallucinated) facts!

(4/4)🧵 pic.twitter.com/60foat7b6K

— Jason Weston (@jaseweston) September 21, 2023

But, you know, however ingenious and successful, it's a work-around. As such, it's good for interim use, but it's not a long-term solution to the problem.

A quick remark on so-called “hallucinations” in LLMs and humans

That LLMs “hallucinate” is well-known, though I prefer the term “confabulate.” They just make stuff up. They don’t do it intentionally, or with intent to deceive. They do it because they have no connection to “ground truth.” They don’t even know what such a thing is, not really, though I’m sure, if asked, ChatGPT would say something reasonable about the idea.

What LLMs have, loosely speaking, is an ontology. They ‘know’ what kinds of things exist. As Immanuel Kant pointed out in connection with the ontological argument for the existence of god, existence is not a predicate. When an LLM responds to a prompt, its response is consistent with the ontology it has internalized. If phenomena in its response happen to be true of the world, that’s incidental, though convenient for users.

What’s implicit in this conversation is an assumption that humans do not confabulate, that we can and so talk about reality. I think that’s true, sorta’, but misleading. Yes, we often anchor our statements in reality, as best we can do in the situation. And, yes, we can also deliberately side-step such anchoring. We do this when telling fictional stories without intention to deceive; in this situation we assert that the story is about imagined event. And we can also make things up with deceptive intent.

Nonetheless, I believe that our “natural” linguistic mode is just making things up, confabulation it you will. But we are surrounded by others and have to interact with them. One way to communicate effectively is to anchor our language in external events, in things others can readily observe, in reality, as we like to say. Such anchoring is not necessary for speech, but it is useful when conversing with others.

In this connection I think it’s worth noting that, for a number of years, neuroscientists have been investigating the brain structures that are most active when we’re doing nothing in particular, when we’re letting our mind wander. In that state we may attended to some external event one moment, scratch our back the next, think about that summer’s day three years ago, and so forth. We day dream. What do you think they call that conglomeration of neural structures? They call it the default mode network (DFM). If you give a verbal report of what’s passing by during a day dream, a lot of that is going to be confabulation. It’s just the way we are.

Homework assignment 1: Explicate Descartes’ worries about being deceived by a malignant being in these terms.

Homework assignment 2: Now consider what happens during sensory deprivation.

Abandoned dreams

Addicted to Disneyland? You aren't the only one. [Something's rotten in the state of Denmark]

Daryl Austin, The scientific reason why you can’t stop going to Disneyland, Los Angeles Times, Sept. 19, 2023.

Travel craving:

Though research around “travel craving” is new and relatively sparse, behavioral psychologists and cognitive scientists believe a yearning for travel can fit the clinical understanding of craving as “a strong desire to modify ongoing cognitive experiences” in ways that don’t only relate to addictions.

Neuropsychologist Paul Nussbaum, an adjunct professor at the University of Pittsburgh School of Medicine, explains that such cravings can be especially focused on getting specific outcomes from one’s vacation. “Our brain’s circuitry is wired to both desire things and to have our desires resolved temporarily with action,” he says. In other words, for people who describe their love of Disney travel this way, craving a trip to Disneyland may not only be a desire, but a yearning that continues to grow until satisfied.

It may be connected to smell:

Such yearnings are fed by any number of factors, of course, but smells are high on the list because of how they trigger memories and positive emotions. “The brain region critical to smell is located near the hippocampus, which is an important brain structure that helps us remember,” Nussbaum explains. “That is why the sensation of smell can trigger memories.” Indeed, multiple studies show how one’s sense of smell has a stronger link to memory and emotion than any of the other senses.

Disney seems to understand this as it has filled its parks with machines called Smellitzers — apparatuses carefully disguised or hidden away throughout attractions, shops and walkways, each pumping out soothing and familiar scents to passersby. In a 2017 interview, a 30-year Disney parks veteran said these machines release these familiar scents “on purpose” because the company is mindful that visitors “are using all their senses” when they’re there. But beyond simply releasing the smells, Disney also uses carefully timed systems and fans to ensure those scents reach the nostrils of guests. One Disney attraction patent notes that the ride’s Smellitzers’ “nozzles may be provided to direct scent materials into proximity to the fans so that appropriate scents may be directed to the passenger.” [...]

“Smell is a chemical sense, and it is evolutionarily ancient, so it connects more directly with the emotional parts of the brain than the other senses do,” explains David Ludden, a professor of psychology at Georgia Gwinnett College in Lawrenceville, Ga. “While some odors rise to the level of consciousness, many smells influence our behavior at an unconscious level,” he says, adding that Disneyland’s ability to evoke such unconscious emotions and connect with visitors so deeply is “a big part of Disney’s success.”

There's also peer influence and nostalgia.

More at the link.

Ancient wood-working, almost 500 kya

These ancient whittled logs could be the earliest known wooden structure https://t.co/yeBioOP3q7

— Bill Benzon, BAM! Bootstrapping Artificial Minds (@bbenzon) September 21, 2023

I'm feeling like flowers

Notes on ChatGPT’s “memory” for strings and for events

[Updated on Sept. 21 and 26, 2023]

Here I take a look at the results reported in three previous posts and begin the job of making sense of them analytically. Here are the posts:

What must be the case that ChatGPT would have memorized “To be or not to be”? – Three kinds of conceptual objects for LLMs, New Savanna, September 3, 2023.

To be or not: Snippets from a soliloquy, New Savanna, September 12, 2023.

Entry points into the memory stream: Lincoln’s Gettysburg Address, New Savanna, September 13, 2023.

I set the stage with a passage from F. C. Bartlett’s 1932 classic, Remembering. Then I consider the three cases I laid out in that first post and then go on to look at the results reported in the next two. I conclude by suggesting that we look to the psychological literature on memory and recall to begin making analytic sense of these results. Of course, we also need more observations.

F.C. Bartlett, memory, and schemas

Back in the ancient days of 1932 F. C. Bartlett published a classic study of human recall, Remembering: A Study in Experimental and Social Psychology (1932). He performed a variety of experiments, a number involving the familiar game of having people tell a story from person to person to a chain and then comparing the initial story with the final one. He made the general conclusion that memory is not passive, like a tape-recorder or a camera, but rather is active, involving schemas (I believe he may have been the one to introduce that term to psychology), which shape our recall. A story that corresponds to an existing schema will be more faithfully transmitted than one that does not.

However, I’m not interested in those experiments. I’m interested in something he reports in a later chapter, “Social Psychology and the Manner of Recall,” pp. 264-266:

As everybody knows, the examination by Europeans of a native witness in a court of law, among a relatively primitive people, is often a matter of much difficulty. The commonest alleged reason is that the essential differences between the sophisticated and the unsophisticated modes of recall set a great strain on the patience of any European official. It is interesting to consider an actual record, very much abbreviated, of a Swazi trial at law. A native was being examined for the attempted murder of a woman, and the woman herself was called as a necessary witness. The case proceeded in this way:

The Magistrate: Now tell me how you got that knock on the head.

The Woman: Well, I got up that morning at daybreak and I did... (here followed a long list of things done, and of people met, and things said). Then we went to so and so’s kraal and we... (further lists here) and had some beer, and so and so said....

The Magistrate: Never mind about that. I don’t want to know anything except how you got the knock on the head.

The Woman: All right, all right. I am coming to that. I have not got there yet. And so I said to so and so... (there followed again a great deal of conversational and other detail). And then after that we went on to so and so’s kraal.

The Magistrate: You look here; if we go on like this we shall take all day. What about that knock on the head?

The Woman: Yes; all right, all right. But I have not got there yet. So we... (on and on for a very long time relating all the initial details of the day). And then we went on to so and so’s kraal.. .and there was a dispute ... and he knocked me on the head, and I died, and that is all I know.

Practically all white administrators in undeveloped regions agree that this sort of procedure is typical of the native witness in regard to many questions of daily behaviour. Forcibly to interrupt a chain of apparently irrelevant detail is fatal. Either it pushes the witness into a state of sulky silence, or disconcerts him to the extent that he can hardly tell his story at all. Indeed, not the African native alone, but a member of any slightly educated community is likely to tell in this way a story which he has to try to recall.

What’s going on here? Keep in mind that the issue is not word-for-word recall. Rather, it is the incidents being recalled, in whatever verbal form is convenient. Why can’t the witness simply begin talking about the incident in question? And when, when asked to get on with it, must the witness return to be beginning of the day?

It's as though the memory stream of a day’s events can only be entered at the beginning of the day, and not at arbitrary points within the day. I note that we are dealing with people who do not have clocks and watches they can use to mark events during the day. Of course it’s not enough to have a watch, you must also take note of it at various times during the day. That will give you various points of entry into the memory stream.

This sort of thing is also quite familiar to me as a musician. While I have learned to read music, and have done so often, I am an improvising (jazz) musician and am quite used to playing things “by ear.” If I am practicing a melody by ear, and get lost at some point, I may not be able to restart at the point where I broke off. Rather, like the witness testifying in court, I have to go back to the beginning – in this case, the beginning of the melody rather than the beginning of the day.

To be or not, and beyond

Early in September I asked the question: “Given that [the LLM underlying ChaGPT] has been trained to predict [only] the next word, what MUST have been the case in order to ChatGPT to return the whole soliloquy when given the opening six words?” It must have encountered that soliloquy many different times in its training corpus. That’s the only way that predicting that exact sequence, word after word, not result in training loss.

However, given Bartlett’s observations about human memory, ChatGPT’s ability to rattle off a whole sequence word for word does raise a question. Is it just passively stringing one word after another, or does it recognize internal structure? How can we figure out which is the case?

I want to set those questions aside for a moment, but I will return to the question. Though interesting, such specific sequences are relatively rare. It is much more common for the training corpus to have many texts about the same event or set of events, but not expressed in the exact same words. Thus I have ChatGPT the prompt, “Johnstown flood, 1889.” Note that I specified the year because Johnstown (PA) was subsequently flooded in 1937 and 1977. But it’s the 1889 flood that made the national news, prompting national concern.

ChatGPT responded in a way I thought reasonable. Since I had grown up in Johnstown and was familiar with the flood, I didn’t bother to check the Chatster’s reply against reliable sources. But, for all I know, the Chatster was giving me some specific text word-for-word, implying that there was some specific text about the flood that had appeared many times in the training corpus. While that didn’t seem likely, I had to check. Later that same day I opened a new session and gave ChatGPT the same prompt. Again, it gave me a reasonable reply, but one that was expressed differently from the earlier one. This reply gave the sequence of events in seven numbered paragraphs. The earlier reply did not have a sequence of numbered paragraphs.

So we’ve got two cases so far: 1) a specific sequence of words that is repeated when prompted for, and 2) and flexible recall of an event using different word sequences in different sessions. There is a third case to consider: 3) and event that is in the training corpus, but in so very few times, perhaps only once, that it doesn’t register in ChatGPT’s model as a specific event. The text serves as evidence about word usage, but otherwise has no effect on the model.

Without access to the training corpus, how do you identify such things? You can’t. But you can make a plausible. I’d attended a Dizzy Gillespie concert back in the mid-1980s which I’d written about in two places which could have been in the training corpus. I prompted ChatGPT with that concert, naming the venue and city where the concert took place in addition to the artist (Diz). Apparently, it had no record of it.

Let me offer you a somewhat different example of this last case. I’m currently interested in a mathematician named Miriam Lipschutz Yevick. She published a paper back in 1975 (Holographic or fourier logic), which I think is interesting and important, but which has been forgotten. The paper is available on the web, and I have blogged about it. A few other papers are also available, as well as an obituary, all before ChatGPT’s cut-off point. I’ve asked Chatster about Yevick in several different sessions but it knows nothing about her. 

Let’s think about this a bit. GPT-3.5, the large language model underlying ChatGPT, may have been trained on (almost, a big chunk of) the entire internet, but its model does not incorporate everything that it has been trained on. It is abstracting over those texts, not memorizing them in any ordinary sense of the word. When a particular text occurs word-for-word many times and in various contexts, GPT-3.5 will learn it word-for-word; think of that as, in effect, an abstraction over those many contexts. When a particular topic, that is, a particular congeries of terms, occurs many times and in various contexts, GPT-3.5 abstracts over than congeries and meshes them together so they are mutually available. If neither of these things occurs to something that appears in a text, then that something just dissolves into the net.

Thus we’ve got three cases: 1) word-for-word recall of a text, 2) flexible recall of a specific topic, and 3) no recall of a topic that was in its training corpus. I want to return to the first case, where ChatGPT is generating a fixed text, and see what, if anything, we can learn about how it does it.

Jump and Kong discussing techbro hubris while hanging out at the mall waiting to board the Starship

Live Forever! This is one MoFo of an anxiety management routine [TechBros gotta' do what a TechBro gotsta to do]

Charlotte Alter, The Man Who Thinks He Can Live Forever, Time, September 20, 2023.

Johnson, 46, is a centimillionaire tech entrepreneur who has spent most of the last three years in pursuit of a singular goal: don’t die. During that time, he’s spent more than $4 million developing a life-extension system called Blueprint, in which he outsources every decision involving his body to a team of doctors, who use data to develop a strict health regimen to reduce what Johnson calls his “biological age.” That system includes downing 111 pills every day, wearing a baseball cap that shoots red light into his scalp, collecting his own stool samples, and sleeping with a tiny jet pack attached to his penis to monitor his nighttime erections. Johnson thinks of any act that accelerates aging—like eating a cookie, or getting less than eight hours of sleep—as an “act of violence.”

Johnson is not the only ultra-rich middle-aged man trying to vanquish the ravages of time. Jeff Bezos and Peter Thiel were both early investors in Unity Biotechnology, a company devoted to developing therapeutics to slow or reverse diseases associated with aging. Elite athletes employ therapies to keep their bodies young, from hyperbaric and cryotherapy chambers to “recovery sleepwear.” But Johnson’s quest is not just about staying rested or maintaining muscle tone. It’s about turning his whole body over to an anti-aging algorithm. He believes death is optional. He plans never to do it.

Outsourcing the management of his body means defeating what Johnson calls his “rascal mind”—the part of us that wants to eat ice cream after dinner, or have sex at 1 a.m., or drink beer with friends. The goal is to get his 46-year-old organs to look and act like 18-year-old organs. Johnson says the data compiled by his doctors suggests that Blueprint has so far given him the bones of a 30-year-old, and the heart of a 37-year-old. The experiment has “proven a competent system is better at managing me than a human can,” Johnson says, a breakthrough that he says is “reframing what it means to be human.” He describes his intense diet and exercise regime as falling somewhere between the Italian Renaissance and the invention of calculus in the pantheon of human achievement. Michelangelo had the Sistine Chapel; Johnson has his special green juice.

Ta Da!

Johnson walks into the room, wearing a green T-shirt and tiny white shorts. He has the body of an 18-year-old and the face of someone who had spent millions attempting to look like an 18-year-old. His skin is pale and glowing, which is partly because of the multiple laser treatments he’s done, and partly because he had no hair on his entire body. The hair on his head is “not dyed,” Johnson says, but he does use a “gray-hair-reversal concoction” which includes “an herbal extract” that colors the hair a darkish brown. He gestures to my Green Giant, and then toward the bathroom. “Did you warn her?” he asks Tolo. I pretend to take another sip.

Erection detection:

“I have, on average, two hours and 12 minutes each night of erection of a certain quality,” he says. “To be age 18, it would be three hours and 30 minutes.” Nighttime erections, he says, are “a biological age marker for your sexual function,” one that also has implications for cardiovascular fitness. The erection tracker looks like a little AirPods case with a turquoise strap, like a purse worn by a penis. (No penises were viewed in the reporting of this article.)

Death begone!

Johnson insists all this is about something much bigger than getting ripped and maintaining a youthful glow. “Most people assume death is inevitable. We're just basically trying to prolong the time we have before we die,” he says. Until now, he adds, “I don't think there's been any time in history where Homo sapiens could say with a straight face that death may not be inevitable.”

Living in the future:

As Johnson, Tolo and I settle in to eat our “first meal” on his massive rust-colored couch, Johnson gestures to a bookshelf full of biographies: Ben Franklin, Harry Truman, Winston Churchill, Napoleon. “I have a relationship with the 25th century more than I have a relationship with the 21st century,” he says. “I don't really care what people in our time and place think of me. I really care about what the 25th century thinks.”

There's more at the link, much more. And pictures!