Friday, January 3, 2025

Scaling of Search and Learning: A Roadmap to Strawberry fields o1, o3 (forever?)

Zhiyuan Zeng, Qinyuan Cheng, Zhangyue Yin, Bo Wang, Shimin Li, Yunhua Zhou, Qipeng Guo, Xuanjing Huang, Xipeng Qiu, Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective, arXiv:2412.14135v1

Abstract: OpenAI o1 represents a significant milestone in Artificial Inteiligence, which achieves expert-level performances on many challanging tasks that require strong reasoning this http URL has claimed that the main techinique behinds o1 is the reinforcement learining. Recent works use alternative approaches like knowledge distillation to imitate o1's reasoning style, but their effectiveness is limited by the capability ceiling of the teacher model. Therefore, this paper analyzes the roadmap to achieving o1 from the perspective of reinforcement learning, focusing on four key components: policy initialization, reward design, search, and learning. Policy initialization enables models to develop human-like reasoning behaviors, equipping them with the ability to effectively explore solution spaces for complex problems. Reward design provides dense and effective signals via reward shaping or reward modeling, which is the guidance for both search and learning. Search plays a crucial role in generating high-quality solutions during both training and testing phases, which can produce better solutions with more computation. Learning utilizes the data generated by search for improving policy, which can achieve the better performance with more parameters and more searched data. Existing open-source projects that attempt to reproduce o1 can be seem as a part or a variant of our roadmap. Collectively, these components underscore how learning and search drive o1's advancement, making meaningful contributions to the development of LLM.

Claude 3.4 describes a photo of a skate-boarding boy & we discuss technique

What’s happening in this picture?

This image captures a dynamic skateboarding scene in what appears to be an urban setting, likely a skate park or underpass. A skateboarder wearing a purple hoodie and khaki pants is photographed mid-maneuver on a curved surface. The image employs motion blur techniques to convey the speed and movement of the skater, creating a sense of energy and action.

The background features graffiti-covered walls with visible splashes of pink, blue, and other colors, adding to the urban aesthetic of the composition. The lighting appears natural, possibly filtering in from outside the structure, which creates an interesting interplay of light and shadow across the scene.

The photograph's technical execution, particularly the motion blur effect while maintaining focus on the skateboarder, suggests this was a deliberately composed action shot rather than a casual snapshot. This technique effectively communicates both the movement and the athletic nature of skateboarding as an urban sport.

Nothing deliberate about it. I was lucky to get the shot. I saw the boy skating, moved my camera to follow him, and took the shot.

I appreciate you sharing the background of how you captured this remarkable shot. It's an excellent example of "panning" - a photography technique where the camera moves along with the subject to keep them relatively sharp while blurring the background. While you describe it as luck, successfully executing this technique, even unintentionally, requires good instincts and timing.

The resulting image is quite striking. The motion blur of the background graffiti creates an engaging sense of movement, while maintaining enough clarity on the skateboarder to capture their form during the maneuver. The contrast between the sharp subject and the streaked background effectively conveys the dynamic nature of skateboarding.

This type of spontaneous photography often produces more authentic and compelling results than carefully planned shots. Would you be interested in sharing more about what inspired you to photograph at this particular location?

It’s a skate park known as FDR in South Philadelphia. My sister lived near there. I’m interested in graffiti and skate parks so went there several times to take photos.

Thank you for providing that context about FDR Skate Park. It's a significant location in Philadelphia's skateboarding culture, known for its distinctive DIY history and design. The park's location beneath I-95 provides both shelter and a unique architectural setting that has made it particularly appealing for photographers and skaters alike.

Your interest in both graffiti and skate parks makes FDR an ideal subject, as it exemplifies the intersection of these two urban art forms. The graffiti serves as more than mere decoration; it's an integral part of the park's identity and contributes to its authentic street culture atmosphere. In your photograph, this is particularly evident in how the blurred graffiti creates a dynamic backdrop that complements the skating action.

The image effectively captures the organic relationship between skateboarding and street art that defines spaces like FDR. Having multiple visits to the location likely helped you develop an understanding of how skaters interact with the space, leading to opportunities for compelling shots like this one.

Would you be interested in discussing how the park's unique architecture and layout influences your photographic approach?

My photographic approach is simple: keep my eyes open, think quick, and get the shot. It’s a bit like jazz improvisation.

Your comparison between photography and jazz improvisation is quite apt and illuminating. Both art forms require a foundation of technical knowledge combined with the ability to make split-second creative decisions based on changing conditions.

What Miriam Yevick Saw: The Nature of Intelligence and the Prospects for A.I., A Dialog with Claude 3.5 Sonnet

New working paper posted. Title above, links, summary (abstract), TOC, and introduction below.

Claude’s Summary

1. Miriam Yevick's 1975 work proposed a fundamental distinction between two types of computing: A) Holographic/parallel processing suited for pattern recognition, and B) Sequential/symbolic processing for logical operations. Crucially, she explicitly connected these computational approaches to different types of objects in the world.

2. This connects to the current debate about neural vs. symbolic AI approaches: A) The dominant view suggests brain-inspired (neural) approaches are sufficient. B) Others argue for neuro-symbolic approaches combining both paradigms, and C) Systems like AlphaZero demonstrate the value of hybrid approaches (Monte Carlo tree search for game space exploration, neural networks for position evaluation).

3. The 1988 Benzon/Hays paper formalized “Yevick's Law” showing: A) Simple objects are best represented symbolically, B) Complex objects are best represented holographically, and C) Many real-world problems require both approaches.

4. This framework helps explain: A) Why Chain of Thought prompting works well for math/programming (neural system navigating symbolic space), B) Why AlphaZero works well for chess (symbolic system navigating game tree, neural evaluation of positions), and C) The complementary relationship between these approaches.

5. These insights led to a new definition of intelligence: “The capacity to assign computational capacity to propositional (symbolic) and/or holographic (neural) processes as the nature of the problem requires.”

6. This definition has implications for super-intelligence: A) Current LLMs' breadth of knowledge doesn't constitute true super-intelligence, B) Real super-intelligence would require superior ability to switch between computational modes, and C) This suggests the need for fundamental architectural innovations, not just scaling up existing approaches.

The conversation highlighted how Yevick's prescient insights about the relationship between object types and computational approaches remain highly relevant to current AI development and our understanding of intelligence itself.

CONTENTS

Introduction: Speculative Engineering 2

Yevick’s Idea 2
Why Hasn’t Yevick’s Work Been Cited? 4
From Claude to Speculative Engineering 5
What’s in the Dialog 6
Attribution 7

From Holographic Logic 1975 to Natural Intelligence 1988 8

Yevick’s Holographic Logic 8
Natural Intelligence 13
Chain of Thought 16
Chain of Thought Is the Inverse of the Alphazero Architecture 17
Define Intelligence 18
Implications of My Proposed Definition of Intelligence 20
Summary 22

Claude Reviews Yevick’s Full 1975 Text 25
Generalizing Yevick’s Results 26
Explaining Yevick to Non-Experts 28
Uzkeki Peasants Have Trouble With Simple Geometric Objects 31
Why Yevick’s Work Has Been Neglected 33

Introduction: Speculative Engineering

In the course of the dialog with Anthropic’s Claude 3.5 about artificial intelligence I propose a novel definition:

Intelligence is the capacity to assign computational capacity to propositional (symbolic) and/or holographic (neural) processes as the nature of the problem requires.

Compare this with a version of the conventional definition:

Intelligence is the ability to learn and perform a range of strategies and techniques to solve problems and achieve goals.

Those definitions are very different. My proposal suggests specific mechanisms while the conventional definition does not. While my proposal must be considered speculative, it can be used to guide research (that’s what speculation is for). In fact it can guide two research programs: 1) a scientific program about the mechanisms of human intelligence, and 2) an engineering program about the construction of A.I. systems. The conventional definition is probably correct, in some sense. But it provides very little guidance for any research. Its use is primarily rhetorical, for use in general or philosophical discussions of artificial intelligence.

Which is preferable, a definition that is speculative and useful in guiding research, or one that is (weakly) correct but of little value in guiding research?

This is the issue that hovers over the dialog I have constructed with Claude.

I will get around to explaining why I’ve consulted Claude soon enough. But first I want to talk about where that definition of intelligence came from. The words are mine, but the basic idea is not.

Yevick’s idea

The idea belongs to a mathematician, the late Miriam Yevick. Consider this remark that Yevick published in 1978:

If we consider that both of these modes of identification enter into our mental processes, we might speculate that there is a constant movement (a shifting across boundaries) from one mode to the other: the compacting into one unit of the description of a scene, event, and so forth that has become familiar to us, and the analysis of such into its parts by description. Mastery, skill and holistic grasp of some aspect of the world are attained when this object becomes identifiable as one whole complex unit; new rational knowledge is derived when the arbitrary complex object apprehended is analytically described.

She thought of one of those computational modes as holographic and the other as logical or sequential. She regarded both as essential to human mentation. Yevick analyzed those two modes in mathematical detail in a 1975 article I used to I begin my discourse with Claude (p. 8 below). David Hays and I employed those modes in the 1988 article that I quote from later in the dialog (p. 13) and in a more informal article (about metaphor) that we published at roughly the same time. Though they use different terms, Bengio, LeCun, and Hinton recognize the same distinction in their Turing Award paper.

But Yevick’s distinction entails something that Bengio, LeCun, and Hinton do not talk about. She regards the holographic mode as best suited for dealing with one kind of object, one that is geometrically complex, and the logical mode as best suited for a different kind of object, one that is geometrically simple. Now, in the passage I’ve quoted above Yevick has generalized from the visual mode, which she argued in her 1975 paper, to mental processes in general. Hays and I do so in our 1988 article as well. As far as I know, that mathematics has yet to be done – which is one reason I am trying to bring attention to Yevick’s work.

If Yevick is correct, then the current debate we are having about how to proceed is ill-posed. As far as I can tell, the majority view is that scaling up machine learning procedures will prove sufficient to achieving human-scale intelligence (not to mention super-intelligence). Gary Marcus, Subbarao Kambhampati, and various others have been arguing that, no, we also need symbolic (logical) mechanisms. Yevick’s work comes down on this side of the debate as well, and contributes to it by specifying that the two modes are each suited to a different aspect of that world. That would naturally lead to a discussion about the nature of the world, but that is beyond the scope of this document.

In view of the importance of this debate, it does not seem unreasonable. It is not only that billions of dollars are currently being wagered–and that is what it is, gambling, but that they are being wagered on technology that may well bring about fundamental changes in the way we live. Can we not take a step back and think about what we are doing?

Friday Fotos: Glimpses of the old neighborhood

Fer Chrissake! Let kids move, plan, and act.

Jenny Anderson and Rebecca Winthrop, Giving Kids Some Autonomy Has Surprising Results, NYTimes, 1.2.25.

In a polarized nation, one point of agreement deserves more attention: Young adults say they feel woefully unprepared for life in the work force, and employers say they’re right.

In a survey by Gallup and the Walton Family Foundation of more than 4,000 members of Gen Z, 49 percent of respondents said they did not feel prepared for the future. Employers complain that young hires lack initiative, communication skills, problem-solving abilities and resilience.

There’s a reason the system isn’t serving people well, and it goes beyond the usual culprits of social media and Covid. Many recent graduates aren’t able to set targets, take initiative, figure things out and deal with setbacks — because in school and at home they were too rarely afforded any agency.

Giving kids agency doesn’t mean letting them do whatever they want. It doesn’t mean lowering expectations, turning education into entertainment or allowing children to choose their own adventure. It means requiring them to identify and pursue some of their own goals, helping them build strategies to reach those goals, assessing their progress and guiding them to course-correct when they fall short.

Learning to "drive":

No adult can force a student to be self-driven. Children develop the skill the way they learn anything else: with practice. The act of setting the goal makes it more meaningful. When we visited Carmen Arellano, a teacher at Arthur Kramer Elementary School in Dallas, she had students set learning goals for every class she taught, and when they met them, “they chase me down in the hallway to tell me,” she said. “I think it just really makes them proud of what they’ve accomplished.” [...]

As the developmental psychologist Aliza Pressman says: “Let kids do for themselves what they can already do. And guide and encourage them to do things they can almost do. And then teach and model for them the things that they can’t do.” This is how parents can help their children build agency.

Change the goal of eduction:

Maybe it’s time to define a higher ideal for education, less about ranking and sorting students on narrow measures of achievement and more about helping young people figure out how to unlock their potential and how to operate in the world. Amid the drumbeat of evolving artificial intelligence, wars, rising authoritarianism, political polarization and digital disconnection, they need to learn a lot more than how to follow instructions.

Thursday, January 2, 2025

Jersey City hinterlands, 2007

Some anecdotal evidence for segmented sleep: When I've been up during the night in December 2024

Over the years I've developed an interest in segmented sleep, as it is called. Instead of sleeping straight through the night for eight hours, or seven or nine, whatever, you have several sleep sessions interspersed with sessions where you do other stuff. This happens to me during a creative phase. I went into a down phase a couple of months ago and started coming out in the third week of November. I've had a LOT of segmented sleep since then. 

One of the things I'd do in one of my middle-of-the-night work sessions is make an entry on my Facebook page indicating that I was awake. I don't know just when I started doing that, but the first time I did it I posted a video clip of one of those "Time to make the donuts" commercials from the 1980s. I've been doing that every once in awhile since then. But other things as well.

This time I decided to keep a systematic record. Here's the record for December, 2024. I was up during every night except the 6th, 10th, and 13th. In some nights I was up more3 than once. I'll try to keep a record through January and then we'll see.

Time
Date
Content of Facebook notice

4:18 AM

12.1.24

Awake in the middle of the night. Good thoughts. All’s well.

3:22 AM

12.2.24

Middle of the night. Awake. Good thoughts.

4:02 AM

12.3.24

4:02 AM Tue 12.3.24 [I simply noted the time]

4:30 AM

12.4.24

Just another marker.

3:36 AM

12.5.24

3:38 AM & I’m just cruising by.

5:20 AM

12.7.24

Still at it.

4:33 AM

12.8.24

It’s that time again. [plus a photo of a donut]

3:18 AM

12.9.24

How’s everybody doin’?

4:16 AM

12.11.24

Peek-A-Boo!

5:41 AM

12.12.24

on the trail of CONSCIOUSNESS

4:32 AM

12.14.24

Someone’s been busy [+a photograph of a box of Krispy Kremes]

12:45 AM

12.15.24

12:45 AM

3:00 AM

12.15.24

Imagine this is about donuts, that's my state of mind at 3:00AM. [Attached to a video of “Shrimp Boats.”]

5:01 AM

12.15.24

Would someone please pass the donuts.

4:34 AM

12.16.24

Time to check in.

3:32 AM

12.17.24

Just a quick one, I promise. Comment added: Umm....er... Not so quick. 3:32 to 4:43. & I make two blog posts!

4:02 AM

12.18.24

Me to Self: You just HAD to check our email?

2:28 AM

12.19.29

2:28 AM and the night is still young.

4:09 AM

12.19.29

Nag Nag Nag! U do the donuts, I got work to do. (4:09)

4:02 AM

12.20.24

Psssst. Anyone here? {4:02 AM}

4:49 AM

12.21.24

Up every night in Dec. so far but 6, 10 & 13.

12:46 AM

12.22.24  

12:46 AM and the night is still young. [Note: I’d gone to bed at about 8:15.]

3:17 AM

12.22.24

It’s been 2 1/2 hours since last check in.

5:14 AM

12.22.24

#3

2:38 AM

12.23.24

12° at 2:40AM in Hoboken

4:33 AM

12.23.24

11° at 4:34AM in Hoboken

2:54 AM

12.24.24

I tried to resist, I held out as long as I could. 2:54 AM

Comment to above: & now it's 3:51 AM. Back to bed...& maybe to sleep as well.

3:46 AM

12.25.24

Gotta put cookies out for the Big Guy. 3:46 AM

2:10 AM

12.26.24

Resistance is futile. 2:10 AM

4:22 AM

12.26.24

Curse those Borg! 4:22 AM

4:21 AM

12.27.24

What time is it, kids? [Howdy Doody opening]

4:09 AM

12.28.24

Not Doris Day. 4:09 AM [Clip of Pixies singing “Que Sera Sera.”]

2:46 AM

12.29.24

I cannot believe...

4:49 AM

12.29.24

AGAIN? 4:49AM

4:24 AM

12.30.24

Right on time: 4:24 AM

1:21 AM

12.31.24

Peek-a-boo!

Rodney Brooks has his tech predictions up

As you may know, roboticist Rodney Brooks has been publishing and systematically updating tech predictions every January 1st since 2018. He makes predictions in the following categories: (1) self driving cars, (2) robotics, AI , and machine learning, and (3) human space travel. Here’s the link for 2025. I’ve got some Rodney Brooks stuff here on the Savanna, including, but not limited to, excerpts from earlier prediction.

In general

In these excerpts he talks about where we are and where we aren’t in a general way (coloring in the original):

I want to be clear, as there has been for almost seventy years now, there has been significant progress in Artificial Intelligence over the last decade. There are new tools and they are being applied widely in science and technology, and are changing the way we think about ourselves, and how to make further progress.

That being said, we are not on the verge of replacing and eliminating humans in either white collar jobs or blue collar jobs. Their tasks may shift in both styles of jobs, but the jobs are not going away. We are not on the verge of a revolution in medicine and the role of human doctors. We are not on the verge of the elimination of coding as a job. We are not on the verge of replacing humans with humanoid robots to do jobs that involve physical interactions in the world. We are not on the verge of replacing human automobile and truck drivers world wide. We are not on the verge of replacing scientists with AI programs.

Breathless predictions such as these have happened for seven decades in a row, and each time people have thought the end is in sight and that it is all over for humans, that we have figured out the secrets intelligence and it will all just scale. The only difference this time is that these expectations have leaked out into the world at large. [...]

Today I get asked about humanoid robots taking away people’s jobs. In March 2023 I was at a cocktail party and there was a humanoid robot behind the bar making jokes with people and shakily (in a bad way) mixing drinks. A waiter was standing about 20 feet away silently staring at the robot with mouth hanging open. I went over and told her it was tele-operated. “Thank God” she said. (And I didn’t need to explain what “tele-operated” meant). Humanoids are not going to be taking away jobs anytime soon (and by that I mean not for decades).

You, you people!, are all making fundamental errors in understanding the technologies and where their boundaries lie. Many of them will be useful technologies but their imagined capabilities are just not going to come about in the time frames the majority of the technology and prognosticator class, deeply driven by FOBAWTPALSL, think.

But this time it is different you say. This time it is really going to happen. You just don’t understand how powerful AI is now, you say. All the early predictions were clearly wrong and premature as the AI programs were clearly not as good as now and we had much less computation back then. This time it is all different and it is for sure now.

Humanoid robots

I think we are a long way off from being able to for-real deploy humanoid robots which have even minimal performance to be useable and even further off from ones that have enough ROI for people want to use them for anything beyond marketing the forward thinking outlook of the buyer.

Despite this, many people have predicted that the cost of humanoid robots will drop exponentially as their numbers grow, and so they will get dirt cheap. I have seen people refer to the cost of integrated circuits having dropped so much over the last few decades as proof. Not so.

They are committing the sin of exponentialism in an obviously dumb way. As I explained above the first integrated circuits were far from working at the limits of physics of representing information. But today’s robots use mechanical components and motors that are not too far at all from physics based limits, about mass, force, and energy. You can’t just halve the size of a motor and have a robot lift the same sized payload. Perhaps you can halve it once to get rid of inefficiencies in current designs. Perhaps. But you certainly can’t do it twice. Physical robots are not ripe for exponential cost reduction by burning wastes in current designs. And it won’t happen just because we start (perhaps) mass producing humanoid robots (oh, but the way, I already did this a decade ago–see my parting shot below). We know that from a century of mass producing automobiles. They did not get exponentially cheaper, except in the computing systems. Engines still have mass and still need the same amount of energy to accelerate good old fashioned mass.

Human spaceflight

Brooks has a couple of paragraphs on the SpaceX Starship. This is the next to the last of those (my highlighting):

This is the vehicle that the CEO of SpaceX recently said would be launched to Mars and attempt a soft landing there. He also said that if successful the humans would fly to Mars on it in 2030. These are enormously ambitious goals just from a maturity of technology standpoint. The real show stopper however may be human physiology as evidence accumulates that humans would not survive three years (the minimum duration of a Mars mission, due to orbital mechanics) in space with current shielding practices and current lack of gravity on board designs. Those two challenges may take decades, or even centuries to overcome (recall that Leonardo Da Vinci had designs for flying machines that took centuries to be developed…).

If and when we produce AGI-level AIs and robots, perhaps they’ll take over the space travel mission, as I’ve recently suggested in a conversation with Claude 3.5. I’ve also published an excerpt from that conversation over at 3 Quarks Daily. At the moment I can imagine a future in which humans regularly spend time in near earth orbit, and perhaps we’ll have a few on the Moon, but their presence there will be more ritual than practical. Maybe robots and AIs will have a permanent presence on Mars, and perhaps other planets as well. And perhaps we’ll have Jeff Bezos’s rotating cities as well. If one of them is near Mars, people can travel back and forth between it and Mars. But that’s a long way off.

Wednesday, January 1, 2025

Top 25 Construction Projects of 2025

Barbie [Media Notes 151]

I just watched Barbie on Netflix. Visually gorgeous, stunning, witty. I liked that A LOT. Otherwise:

What The Fucketty Fuck?!

I know about the stereotypes and the unrealistic body image and Ken, the car, and all the accessories. And about Mattel making mucho $$$$$s. Guess what? the movie made mucho $$$$$s too. Wikipedia: “...the highest-grossing live-action comedy film of all time...”

What The Fucketty Fuck?!

And, Yay! Margo Robbie. Yay! Kate McKinnon. Yay! Gretta Gerwig. Where’d Ryan Gosling get those pecs? From a box of Wheaties?

Forget Bambie Meets Godzilla.

I want to see Barbie Meets Godzilla.

HOOAH! FUCK YEAH!!!!