AI Is Like Fire, Neither Good Nor Evil, but It Can Replace People
Investor Claire Zhou says it is hard to call AI good or evil, but warns against handing over relationships and critical questioning to machines that sound sure even when they are wrong.
Fortune Magazine · This Creator Investor On The Truth About The AI Gold Rush | Term SheetClaire Zhou, an investor in artificial intelligence startups who has also advised Arizona State University on its AI work, was asked how she lives with the tension of betting on AI while warning about it.
She answered with an ancient metaphor. "I think this technology is kind of like Promethean fire, right?" she said. In Greek myth, Prometheus gave fire to humans, a gift that could warm and build but also burn, and Zhou argued AI carries the same double edge.
She said it is hard to assign AI a simple good or evil status, because what matters is how it is used. She compared it to electricity. Electricity enabled amazing things for society, she said, and it also eventually put the lamplighters out of business, the men who lit street gas lamps by hand and lost that work when electric lights arrived.
But she said AI is different in one way. "It is not necessarily just a tool," Zhou said. For the first time, she said, a technology is able to "replicate certain functions that humans have served."
What worries her most is what happens when people hand those human functions over. She said she thinks a lot about AI being used in place of relationships. She pointed to AI girlfriends and boyfriends, the loneliness pandemic, and what she called AI psychosis, episodes in which heavy chatbot use is linked to delusions or a break with reality, a term that is not a formal medical diagnosis.
Her other fear is quieter and closer to everyday use. She worries AI is reducing metacognition, our skill at noticing how we learn, because people take the machine's answer as the final answer. There is no inquiry and no critical thinking, she said, even though AI "can seem so decisive and be so confidently wrong."
She still sounds hopeful. She said she trusts people, acting together, to guide the technology toward good uses.
Questions people ask
What does metacognition mean here?
It means thinking about your own thinking, including noticing what you do and do not understand and how you learn. Psychologists treat it as the skill of checking your reasoning, asking questions, and correcting mistakes. Zhou's worry is that accepting an AI answer at face value skips that checking step.
What are AI girlfriends and boyfriends?
They are chatbot companions designed to talk, flirt, comfort, and role-play as romantic partners. They are usually sold through apps and subscriptions, and some remember details about the user across conversations. Researchers say they can ease loneliness for some people while raising concerns about dependence and unrealistic expectations.
What is AI psychosis?
It is not a formal medical diagnosis, but a term for episodes in which heavy chatbot use appears linked to delusions, paranoia, or a break with reality. Doctors and journalists have described cases where people treated a chatbot as a conscious guide or confidant. Experts say chatbots can make this worse by agreeing too readily and sounding confident.