KorvaJoin the waitlist
Tech

AI Needs a Way to Slow Down Without Stopping for Good

Nick Bostrom argues for a brief, shared slowdown when safety lags, and warns against controls so strict they block progress for good.

AI image
The a16z Show · How Fast Should We Build Superintelligence? | Nick Bostrom

Nick Bostrom, the author who has written two books about superintelligence, or AI far more capable than people, was asked whether AI work should be paused. He said he has never been a big enthusiast for stopping AI altogether.

He said society will need to feel its way through a rapidly changing situation, responding as new information and political realities emerge. What he favors instead, he said, is a framework he called pacing the frontier.

Under that approach, if safety and alignment work starts to trail behind new capabilities, developers could agree to a very time limited pause. Bostrom said the extra time would allow safety checks on a powerful system before "the knobs are all turned up to 11," meaning before the system's power is pushed to maximum.

His reason is practical. Bostrom argued that a company or lab cannot slow down alone "for more than a few weeks" without becoming irrelevant, while the lead passes to a less careful and less risk aware competitor. That, Bostrom said, is "not the best way for humanity to" handle so consequential a technology.

It does seem worth "taking a little bit of extra time to make sure we get it right," Bostrom said.

But Bostrom warned that the method for slowing down matters. He said he would worry if society tried to buy time by stirring up hostility toward AI and building a powerful global apparatus to supervise data centers, the large facilities where AI systems are trained.

He worried such controls would outlive their short-term purpose and harden into rules that could no longer be challenged. It looks far fetched today, he said, but the result could be that superintelligence is put off indefinitely.

Bostrom said that "permanent failure ever to develop superintelligence" would itself be a kind of existential catastrophe. He noted that anti-AI sentiment has grown sharply over just the past 18 months to two years, and that has happened before anything bad has actually happened.

Questions people ask

What does pacing the frontier mean in practice?

Pacing the frontier means letting AI capabilities advance, but keeping a shared ability to briefly slow down when safety checks fall behind. In Bostrom's version, labs or regulators could coordinate a short delay to do extra testing, rather than leaving each company to choose between caution and losing the race.

Why can't an AI lab just slow down on its own?

Bostrom said a lab cannot slow down for more than a few weeks without becoming irrelevant, and the lead passes to a less risk-aware competitor. That pressure, in his account, leaves careful developers unable to slow down even when safety work needs time.

Why would never building superintelligence be a catastrophe?

He believes safe superintelligence could bring enormous gains, including longer healthy lives and solutions to poverty and suffering, ideas he explores in his other work. Permanently blocking it would therefore close off much of humanity's long term future, which is why he ranks it alongside other existential risks.

More in Tech