You’ve probably done it. You ask an AI chatbot about a topic you know cold—maybe it’s physics, or the realities of the entertainment industry. It spits out a paragraph so fundamentally wrong, so completely divorced from the actual facts, that you laugh out loud. You realize in that moment the machine is utterly full of shit.
Then, ten minutes later, you ask it to summarize a legal document or research a historical event you know nothing about. And you believe every single word it says.
The machine isn’t hallucinating; you’re just forgetting.
We are currently watching a slow-motion societal lobotomy, and nobody wants to admit it. For years, the tech world has been obsessed with a pointless debate: do Large Language Models (LLMs) actually “understand” the world, or are they just stochastic parrots mindlessly predicting the next word? It’s a red herring. Whether the machine understands us is irrelevant. The real danger is that we are desperate to project understanding onto a system that fundamentally lacks any internal model of truth.
Let’s call it what it is: Slop Amnesia. You see the AI fail spectacularly in your area of expertise. You acknowledge its flaws. But as soon as you step outside your lane, your confirmation bias takes the wheel. You forget the failures and revert to treating the output as gospel.
The utility of these tools is undeniable. They can write code, draft emails, and summarize data with terrifying speed. But utility is not comprehension. An AI that confidently invents a fake legal precedent is annoying. A human who acts on that precedent without checking it is dangerous.
We aren’t building artificial intelligence; we’re outsourcing human judgment.
This isn’t a technical limitation we can patch in the next software update. It’s a collective psychological blind spot. We want the easy answer. We want the frictionless solution. And because the AI sounds so confident, so conversational, so perfectly human, we willingly suspend our disbelief.
The critics who complain that AI doesn’t have a true sense of self are missing the point. It doesn’t need a sense of self. It just needs you to forget that it doesn’t have one. Every time you trust an AI output without verifying it, you aren’t using a tool—you are actively eroding your own critical thinking skills.
The realization that these systems are full of holes will eventually hit everyone. But it will happen one by one, usually after a costly mistake, and nobody else can make that realization happen for you.
The machine doesn’t need to be perfect to fool you; it just needs you to be lazy.
If you rely on AI for research, writing, or decisions, your own confirmation bias is the weakest link in the chain. Stop arguing about whether the AI understands you. Start asking why you are so eager to trust a machine that you already know is lying to your face.
FAQ
Q: Isn't the AI getting better and more accurate every day?
A: Models are improving, but they still lack an internal model of truth. They are designed to sound convincing, not to be correct. If you assume accuracy based on past improvements, you're already falling for the amnesia effect.
Q: How do I use AI without falling into this trap?
A: Treat every AI output like a rough draft from an eager but inexperienced intern. Use it for friction and ideation, but verify every factual claim, citation, and logical leap yourself before acting on it.
Q: If we know AI is flawed, why is society still trusting it blindly?
A: Because it's easy. Critical thinking is mentally exhausting. AI offers a frictionless illusion of competence, and our brains are wired to take the cognitive shortcut, even when we know the machine is full of shit.