Even a Perfect Truth Machine Would Be Doubted

Onstage in London in September 2019, Tom Scott, a science communicator who makes videos on YouTube, asked his audience to imagine a miracle. Google, he said, had built a machine learning system that could tell fact from fiction, and it was right all the time.

Hooked up to search and recommendations, it would rank only the scientific consensus on vaccinations at the top and push conspiracy videos down until almost no one saw them. Then, Scott said, you discover that this perfect system disagrees with you on something fundamental, something you find abhorrent.

To make it sting, he offered his London audience a test. Imagine, Scott said, that the system concludes Britain should embrace a no-deal Brexit, or, if you like that idea, that it should "dissolve the United Kingdom into Europe entirely." What is more likely, he asked, that you quietly accept being heard less and less, or "that you decide that instead, you need to shout louder, and that the algorithm is wrong."

Scott asked which was more likely, suggesting it would be the second. He said he wanted to talk about "why some science communication seems to go everywhere, and a lot of it doesn't."

Later in the talk, Scott turned to language from YouTube. He quoted Ben McOwen Wilson, YouTube's managing director in the United Kingdom, as saying the platform reduces misleading content and raises up authoritative voices. Scott pointed out what Wilson did not say, that those voices were true or correct, only that they were authoritative.

Scott closed with a larger version of the same worry. If anyone ever builds a superintelligence that could steer humanity, Scott said, he can only hope "its goal is not to maximize the profit of one company." Until then, he said, "The only 'algorithm for truth' that we have right now is ourselves."