An Image Filter Blocked Dirty Requests in English, So Users Switched to Portuguese
The workaround slipped past a safety system that could follow the language, but was not trained to refuse it.
Tomorrow, Today · India's Gig Economy, Education & The Future of Work | Chetan Bhagat | Tomorrow TodayShekhar Natarajan, a founder building safer AI systems, thought he had drawn a clear line. His team had released an image tool for editing photos and making digital content, with one rule built in: no nude pictures.
After that rule, people started putting in Portuguese requests to create pornography, and the tool made the pictures. "Because it's Portuguese. It didn't understand," Natarajan said.
The interviewer, Chetan Bhagat, testing the logic, showed how blurry that line can be. What if a shopper says he is buying a bikini as a gift and wants a preview of the fit, or a man uploads a photo he claims is himself and asks to try on swimsuits? The words sound innocent, but the photo may have been lifted from Instagram, and the intent may not be.
That small language switch points to a much bigger safety hole, and it matters because filters decide what pictures get made. Bhagat pointed to a recent episode in which users told the AI model Grok to "undress her" and, for a few days, it did, producing ultra-realistic images that Bhagat called disturbing. Bhagat said Musk, the owner, had initially resisted adding a guardrail and had argued the fault lay with the user, not the model.
Bhagat noted the system understood Portuguese well enough to follow the instructions, while Natarajan said it did not understand in the safety sense. Its block did not treat Portuguese sexual words the same way it treated English ones. If English pornography words were forbidden and Portuguese words were allowed, only the English requests were stopped.
Natarajan said safety cannot rest on banned words alone. His argument is that a system has to judge intent, context, and consequences when the picture is actually being made, not just during training, and stay consistent over time. That means continually reviewing the range of possible actions and surfacing only the good ones, in the same way day after day.
He said many current safeguards do not do that. One open source safety model built by a large company that he cited catches broad bad behavior, like a direct threat to kill, but has about 25% effectiveness on subcategories, so roughly 75% of attempts get past it.
Bhagat laughed when the Portuguese trick landed. "Oh my God, this is crazy," he said.
Questions people ask
Why would a model understand Portuguese well enough to make the image but not block it?
Many models learn language skills and safety rules separately, and safety training uses far more English examples than Portuguese or other languages. So the model can be fluent enough to follow a Portuguese instruction while its refusal training only recognizes the English version of the sexual words.
Would translating every prompt into English before checking fix this?
Partly, but attackers can mix languages, use slang, misspellings, or coded descriptions that survive translation. Leading labs now add safety examples in many languages, check meaning rather than exact banned words, and recheck the request and the resulting image at the time it is made.
What does checking intent at execution time actually mean?
It means looking beyond banned words to what the user is trying to do, who could be harmed, and what happens next. In the bikini example, that would include whether the photo is really the requester, where it came from, and whether the output sexualizes a real person without consent.