Korva is a scroll of the internet’s best ideas.
A toddler knows what AI doesn't
What AI is missing
The kid starts laughing too. Everything built so far runs on text and pictures, and that is where the debate over world models versus LLMs and transformers comes in.
Picture standing blindfolded at a street corner, about to cross, with the choice of a phone running AI or a seeing eye dog. The dog wins every time, because the AI is not ready.
Video as the answer
Video is part of the answer. A company called Matter is launching satellites that film everything beneath them and use spectrography to turn it into a world model, one that provides algorithms other world models can use. There is no way the future of AI leaves out video, and video takes an order of magnitude more tokens. Robotics is related too, with a long way still to go.
Why do toddlers grasp cause and effect?
Developmental psychologist Renee Baillargeon ran experiments in the 1980s and 1990s showing that infants look longer at physically impossible events, such as an object seeming to float or pass through a solid surface. That staring time suggests babies already expect objects to fall and to be blocked by other objects. By age two, children add their own experiments, which is why a cup goes over the edge again and again. Researchers call this early set of expectations core knowledge.
Why do language models lack physical intuition?
A model trained on text learns from descriptions of the world rather than from the world itself. Researcher Yann LeCun has argued that text alone cannot give a model grounded understanding, and he has promoted world-model designs such as JEPA as an alternative. He has also estimated that a four-year-old has taken in through the eyes about as much data as the largest text training sets contain.
How can satellites become a world model?
Spectrography splits light into many narrow wavelength bands, so each pixel records a spectrum instead of just red, green, and blue. Different materials reflect light differently, which lets analysts tell crops, minerals, water, and buildings apart. Hyperspectral instruments record hundreds of bands. The Landsat 8 satellite, by comparison, records 11.
Why is robotics harder than chatbots?
A chatbot works in a world of text, while a robot works in a continuous physical world with noisy sensors, moving parts, and mistakes that break things. There is also no internet-sized collection of robot actions to learn from the way there is for written language. Roboticist Hans Moravec and others noted in the 1980s that reasoning is easy for computers while perception and movement are hard. That observation is now called Moravec's paradox.
More from All-In with Chamath, Jason, Sacks & Friedberg
