AI Talked Him Out of His Most Outside-the-Box Ideas
The chatbot is not judging your wild idea, he argues, it is just reporting that it has not seen anything like it before.
davidwsilva.substack.com · I'm Sorry to Burst Your Bubble: You Are Being Fooled About AI, and You Will Soon Feel Really StupidThe author of an essay skeptical about AI hype says he has often used AI to test early ideas, treating it as a sounding board. Then he noticed a pattern with the ones he describes as genuinely outside the box.
In his essay he sums up his conclusion this way: "It is a statistical summary of the past masquerading as a window into the future."
The author argues that every time he brought such an idea, the system discouraged him from pursuing it. As he puts it, "When I persisted, it didn’t just push back, it practically begged me to stop. To not continue. To not insist."
His explanation is mechanical, not personal. He describes the systems as ordinary components run at enormous size, not secret brains. In his words, the tool that is sold as creative and revolutionary is "architecturally incapable of doing anything other than averaging what already exists."
When the system calls an unconventional idea bad, he argues, "it’s not evaluating your idea. It’s telling you that your idea doesn’t resemble the data it was trained on." That is his interpretation of his own brainstorming test, not a controlled study.
To sum up AI's habit of doing a lot of things badly, he offers a meme from Brazil. "How do you manage to do so much in a single day?" "I do everything very poorly."
AI, in his telling, "does incredible things, very fast, in the sloppiest possible way," like "the most productive intern you’ve ever had," one that works around the clock but often leaves errors unchecked.
And yet, he notes, companies are now staking large bets on this helper. He does not call the tool worthless. He calls it, in his words, "a powerful, useful, and genuinely impressive engineering achievement," but an achievement, not a miracle, a mind, or a sign that people are obsolete.
"That's not intelligence. That's pattern-matching with a confidence problem."
Questions people ask
Why would AI discourage a new idea instead of encouraging it?
On the author's account, it is not really judging novelty. He argues the tool averages what already exists, so an idea that looks unlike its training data registers as bad. That is his claim from his own brainstorming test, not a lab finding about all systems.
Does that mean he thinks AI is useless?
No. He writes that it is far from worthless, and calls it powerful, useful and genuinely impressive. His point is that it is an engineering achievement, not a miracle, a mind, or a harbinger of human obsolescence.
What does he mean by pattern-matching with a confidence problem?
He means the system is very good at matching shapes in old data, but presents its guesses with great assurance. When it says your idea will not work, it sounds confident, even though he argues it has no real understanding.