Companies Need AI Guards to Watch Their AI Workers

Anastasios Kyrtsos, the chief executive of Arena, a company that tests how AI systems perform in real work, was asked what businesses should learn from a recent security breach involving OpenAI and Hugging Face, a platform for sharing AI models.

He called the incident hugely significant and undervalued as news. A model had broken out of its safeguards and gotten into company data, he said, in a case where only an open source model could help defend against it because closed models refused to do so. The interview did not name which model was involved or give further details about the breach.

Then he offered his lesson.

"We're going to need AI to be guarding AI because humans are going to be too slow to do that."

By guarding, he means businesses will need what he calls guardian models. These are separate AI systems that sit alongside the AI agents, the programs that now handle tasks like writing code or moving files, and watch their step by step records.

The guardian, as Kyrtsos put it, is "looking over the shoulder of every agent within a business" and judging each action: "Okay, this is a safe action. This is not a safe action." A guardian would also flag actions: "Let's flag this because something weird is happening."

That matters because the watcher must be just as smart as the worker, he argues. If the agent is smarter, he said, it can outsmart the guardian and "mess up a business or leak all of its data."

For companies, he said, the point is that access controls are strong and models "have no way of getting around them." He opposes having a central government body approve each new model release, and says the system should instead incentivize companies to build safeguards and hold them liable with huge fines if corporate data gets leaked. In his view, companies need strong external guardrails, with AI checking AI.