Korva is a scroll of the internet’s best ideas.
Don't make every AI Einstein
Einstein was strongly motivated by the bias that things should look the same in different reference frames, and it was very formative in his thinking. He also had a bias toward God not playing dice. If every language model were Einstein, progress on quantum mechanics might halt.
There is no single correct heuristic for science. What it needs is multiple independent research programs, each with its own heuristics. Models trained in similar ways tend to collapse into the same patterns, so the entropy can be added back deliberately: one agent tries to prove a statement while another tries to disprove it, and agents can be given different biases and tested to see which ones are productive on a given problem.
The design question is how to describe the different approaches. Proving versus disproving is the easy split. The harder one is listing all the tactics that could be taken on a proof and making sure each gets enough breadth.
How did Einstein's bias shape relativity?
Einstein held that the laws of physics must take the same form for observers in every reference frame. In 1905 he applied that to observers moving at constant speed, which led to special relativity. By 1915 he extended it to accelerating observers, which produced general relativity. Both theories came directly from that single guiding requirement.
How was Einstein's dice objection tested?
He wrote to Max Born in 1926 that God does not play dice, objecting to chance in quantum mechanics. In 1964, John Bell derived inequalities that any local hidden-variable theory must satisfy. Experiments by Alain Aspect in the early 1980s, and later by others, found the inequalities violated. Aspect, John Clauser, and Anton Zeilinger shared the 2022 Nobel Prize in Physics for this line of work.
Why do diverse ensembles beat copies?
Leo Breiman's random forests, published in 2001, average many decision trees that are each trained on a random sample of the data. The averaging reduces error because the trees' mistakes are not identical. Identical copies would make identical mistakes, and averaging them would change nothing.
More from Dwarkesh Podcast
