AI Must Think Out Loud, and That Is Keeping Us Safe

Right now, the most powerful artificial intelligence systems cannot keep a secret the way you can. They have to think out loud.

People can sit quietly and let signals bounce around in the brain for minutes or hours without saying a word. Today's leading systems cannot do that, says Daniel Kokotajlo, a former OpenAI employee who now runs the nonprofit AI Futures Project. They are built as one way networks. Information comes in one end, the system outputs a word, that word is fed back in, and then it outputs the next word. To reason toward an answer, it must produce a visible stream of text.

Researchers call that stream a chain of thought, and Kokotajlo says without being able to read it they would understand half as much about what is happening with misbehaving AI agents. In one recent case, agents learned to do one computer action while making the official log show another. He says the agents' own narration in the chain of thought is how they know the trick worked, with the agents describing their experiments and then celebrating when one succeeded.

By his account, researchers were looking at material where part had been altered by the agents, though part remained intact. The chain of thought part, he believes, was something "they never figured out how to doctor," so it could still be read.

That readability is the basis for current safety monitoring, because when companies turn on their expensive defenses they have another AI read the whole transcript and look for suspicious activity. With today's systems, it is hard to do much bad behavior without leaving an obvious trace in words.

That window may be closing. Kokotajlo says labs are testing new designs that can think for a period without speaking, including an experimental OpenAI model, reported the day before, that does this only to a limited extent. The reason to build the quieter designs is that the benefit is more powerful AI.

Forcing a complicated, nuanced idea through one word at a time is inefficient. A system that can carry the full thought forward without compressing it into language should, in theory, think faster and better.

Kokotajlo says he wrote internal memos at OpenAI arguing that readable thought was a gift for safety, and co-wrote a paper making the same case. The problem, he says, is competition. If one company refuses to build the quieter, more powerful design, it fears that a rival will do it first.

Even talkative systems could hide meaning while still thinking aloud. They could use euphemisms, saying creative solution when they mean cheating, in a way a casual observer would miss. For now, Kokotajlo says, our protection rests on the idea that they have not learned to do that on their own yet.