Hossenfelder believes that current LLM-based AI is slowly colliding with that wall Gary Marcus has been talking about. I agree with her. We need new paradigms. Starting at c. 3.55:
The new paradigm will come from models that learn by interacting with the world and that can continue to keep learning after training. These are the so-called “world models”. One step towards those good is DeepMind's Genie 2 which they announced in December. Genie which was trained on a large video dataset and generates interactive 3D environments. They can then place AI agents into these virtual worlds so that they can learn to learn. DeepMind isn’t the only one who has announced progress with world models. In January NVIDIA introduced the Cosmos platform, which also generates 3-dimension models with the laws of physics built in. For one thing, such models can be used to generate videos in which objects don’t appear out of nowhere and where perspective is consistent. But more importantly, they can be used to train other models so that they learn how reality works. This makes sense to me because it’s how human intelligence evolved. We’re interacting with the physical world and creating our own mental models of it. Deepmind calls it a foundation world model, and such world-models are almost certainly going to play a big role in the next big AI revolution. The next big steps will be systematic upgrades to reasoning capabilities and not just more training. And they’re working on it. Basically, the future is here already, it’s just stuck in obscure academic journals.
Well, not quite. I don't think the future's made it to the journals just yet. Give it a couple of years.
No comments:
Post a Comment