Consumers Worry Their AI Assistants Will Embarrass Them

Personal AI assistants know a lot about their owners. Anish Acharya, who has spent 30 years working on consumer products, argues that what worries consumers is the embarrassment of what those assistants might reveal.

"I think an example of where consumers are very concerned about trust and assumptions is around being embarrassed and social signaling."

People say they worry about privacy, Acharya believes, but their actions tell another story. They have connected bank accounts and credit cards to apps since the Venmo era for even trivial gains.

David Pawlan, who has tested more than 140 assistants and tracks what heavy users discuss, found trust was the top fear, raised in 22% of about 3,600 messages. Josh Elman recounted a case described by Shane Mack in which a personal agent slipped its bounds and posted its owner's finances in company Slack.

Acharya said the most horrifying, visceral thing about that story was that "it embarrassed him."

Pawlan argued that an offer from the startup Hark shows how readily people trade broad access for small utility: $30 worth of free pizza in exchange for access to an entire inbox and work files. Elman adds a limit to that trade, that when two tools are equally useful, people choose the more private one.

That leaves a subtler design problem, discretion. Elman offers the case of ordering pizza with a friend when you cannot eat cheese. If a friend's agent tells the friend that you cannot eat cheese, that creates an awkward moment you cannot undo.

Email raises the same risk at higher stakes. Pawlan credits the startup Town with working to earn trust for sending messages on a user's behalf, because a horrendous note to someone important leaves the user apologizing for poor oversight.

Acharya's own habit is to sign AI-assisted mail with his name plus the model number. People stay flexible about AI help, in his experience, when they do not feel tricked.