Software Is Only as Secure as the AI You Paid to Break It
Small AI models catch bugs that big ones miss, so safety now requires many kinds of attackers.
Invest Like the Best with Patrick O'Shaughnessy · Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]Neil Movva, who sells access to open source artificial intelligence models by the token, was describing what customers do when intelligence gets cheap enough to keep running in the background. Deep research is one of his biggest uses, with agents answering questions over 10,000 sources or more. Cybersecurity is following in the same direction, he said.
Then he shared the industry joke. “Security has become proof of work,” he said. “When you want secure software, it’s really a question of how many dollars did you spend on Anthropics APIs trying to break into your software?”
By proof of work, Movva means the laborious computing process that proves something was earned, like the endless calculations that secure bitcoin. In software, he argues, the proof is how much AI effort you spent trying to crack your own code. The spending is the measure because, as Movva put it, AI is now “the best tool in the world” for breaking in.
Writing code is limited, he said, but the ways to break it grow much faster. So some customers build agents that break software and patch it.
He pointed to what happened when powerful new models, which he called Fable and Mythos, first appeared. Cybersecurity teams wanted to run the model against every line of code they had ever written, checking 20 different ways for memory errors, business logic errors, network vulnerabilities, and more. They did not just ask the model to read the code once. They had it set up test environments where it could pen test applications, meaning it tried to hack into working copies the way an intruder would.
Bigger models do not always catch more bugs, he said. The frontier of intelligence is jagged, so a very small model called Haiku will catch some bugs that Fable misses, and vice versa. In his telling, that encouraged “this very diverse approach to sampling and trying to build cybersecurity agents that break software autonomously such that you can patch them,” as Movva put it.
Questions people ask
What does "proof of work" originally mean?
It is the system behind bitcoin and some other cryptocurrencies, where computers compete to solve difficult puzzles to validate transactions. The winner proves it burned real computing power and electricity, which makes cheating expensive. Movva is borrowing the phrase to say secure software now proves itself the same way, by surviving expensive attacks.
What is pen testing?
Penetration testing is authorized, simulated hacking, where testers try to break into software to find weaknesses before criminals do. Traditionally, human specialists did it for a few days or weeks. Movva is describing AI agents that can do it continuously, setting up copies of an app and probing them for flaws.
Why would a small model find bugs a large one misses?
Different models are trained on different data and develop different strengths, so they do not strictly rank from worst to best on every task. A smaller model may be more literal, more varied, or simply notice a different pattern. That is why security teams run several models in parallel rather than trusting one large model to catch everything.