We will use everyday language where possible and mark technical terms in purple. Key is that modern LLMs have a component of making answers (in blue, called "inference") and a component of improving answers (red, called "training"). pic.twitter.com/iUmGaNMKUz
— Kording Lab 🦖 (@KordingLab) August 25, 2026
The real work is done by a core machine, the neural network which is used to generate one word at a time based on its inputs. The probability of each word depends on prior words. And words are chosen one by one according to those probabilities. pic.twitter.com/7zWO87TThs
— Kording Lab 🦖 (@KordingLab) August 25, 2026
The result of this are what is sometimes called large-scale "stochastic parrots". Very impressive text but often somewhat superficial. But these are the old LLMs (think 2022). Innovation was super fast since.
— Kording Lab 🦖 (@KordingLab) August 25, 2026
A second idea (B) for making answers, tool use. Call programs like databases, internet search, calculators etc. pic.twitter.com/zyPU00vmV3
— Kording Lab 🦖 (@KordingLab) August 25, 2026
And all of this, is better with better “training“ goals. Wherever truth is defined, push towards this truth (D). And where it is not defined, use LLMs to check answers ("self evaluation", (E)) pic.twitter.com/vLOy2UCsYb
— Kording Lab 🦖 (@KordingLab) August 25, 2026
A great deal of the strengths and weaknesses of today's LLMs can be understood from these component ideas.
— Kording Lab 🦖 (@KordingLab) August 25, 2026
Kording Lab... "And there you have it. The Old Recipe got extended with ‘inner speech’ + tools + agents + better evaluation into today's interactively ‘thinking’ machines." ... "A great deal of the strengths and weaknesses of today's LLMs can be understood from these component ideas."
ReplyDelete"The Old Recipe got extended with" 500 agents evolving code...
Just do evolution... 500 sub agents, one only agent to check subagents test / approve / show faults / repeat / repeat / repeat...
andai 19 hours ago | root | parent | next [–]
"AGI ≈ Artificial Stupidity × Infinite Persistence
Ralph is all you need!
https://ghuntley.com/ralph/
reply
https://news.ycombinator.com/reply?id=49414213&goto=item%3Fid%3D49407507%2349414213
https://venturebeat.com/technology/how-ralph-wiggum-went-from-the-simpsons-to-the-biggest-name-in-ai-right-now?ref=ghuntley.com
BY GEOFFREY HUNTLEY IN AI — 14 JUL 2025
"Ralph Wiggum as a "software engineer"
...
'geoff @GeoffreyHuntley
"From my iMessage (shared with permission) Cost of a $50k USD contract, delivered, MVP, tested + reviewed with @ampcode. $297 USD."
...
"What's kind of wild to me is that Ralph has been able to build this language and is also able to program in this language without that language being in the LLM's training data set.
...
"Each time Ralph does something bad, Ralph gets tuned - like a guitar.
...
"Second task is to use up to 500 subagents to study existing source code in examples/ then compare it against the compiler specifications. From that create/update a fix_plan.md which is a bullet point list sorted in priority of the items which have yet to be implemeneted. Think extra hard and use the oracle to plan. Consider searching for TODO, minimal implementations and placeholders. Study fix_plan.md to determine starting point for research and keep it up to date with items considered complete/incomplete.
...
https://ghuntley.com/ralph/
SD