Korva is a scroll of the internet’s best ideas.
No internet for robots
Videos, simulations, and teleoperation are all being tried. The internet was full of junk even before AI slop, and some of it was worth more than the rest: certain Reddit threads, Wikipedia over many forums, GitHub over other sites. Model companies weighted the data accordingly. Robotics companies are going after the high-value data first to bootstrap a model. Once a strong base exists, small sets of examples added in post-training teach it new tricks, the same move that works with language models.
Vertical integration
Waymo and Tesla were built vertically integrated. Tesla's fleet gathered data that trained the models that drive Teslas, and Waymo did the same. Sunday Robotics, which is building household robots, applies this with gloves designed alongside the robot hand, so data from a person wearing the glove transfers cleanly to the robot. That produces a high-quality pretraining set, then reinforcement learning and post-training on top, and emergent behavior.
A basement demo
The first demo took place in a basement at a Stanford lab, with a janky cardboard glove. A later visit, in the basement of the company's own office building, showed the robots folding arbitrary laundry while people measured each piece to check the folds against a rigorous baseline.
How do model makers weight their training data?
OpenAI's 2020 GPT-3 paper described sampling its training sources at different rates. Wikipedia text was seen about three times during training, while the large Common Crawl web scrape was seen less than once. Smaller, curated sources were repeated because they were considered higher quality. The paper listed Common Crawl, WebText2, two book collections, and Wikipedia as its sources.
How does a data glove teach a robot?
The glove records the positions of a person's fingers as they do a task. If the glove's geometry matches the robot's hand, those motions map directly onto the robot's joints without a lossy translation. The model then learns from what the human hand did. This approach is called imitation learning.
How do robots learn in simulation?
A simulator can run thousands of virtual robots faster than real time, at no wear cost. The weakness is the gap between simulation and reality, since physics in a simulator is only approximate. Randomizing things like friction and lighting during training helps policies survive the gap. In 2019, OpenAI trained a robot hand to solve a Rubik's Cube using this approach, learning the skill in simulation.
More from Invest Like the Best with Patrick O'Shaughnessy
