Stop Calling LLMs Stochastic Parrots. Compression Is Intelligence.

You’ve heard it a thousand times. Someone in a Twitter thread, a Medium post, a panel at a conference—they lean back, smirk, and say: “LLMs are just next-token predictors. They don’t understand anything.”

It’s the ultimate dismissal. The intellectual equivalent of a mic drop. And it makes you feel stupid for ever being impressed by a model that can write a poem about quantum mechanics in the style of Shakespeare.

But here’s the thing: that critique isn’t just wrong. It’s backwards. The next token prediction is the whole point. It’s the mathematical proof that these models actually understand.

Let me show you why.

Imagine you’re playing a game: guess the next word in a sentence. “The cat sat on the ___.” You’d say “mat.” Easy. But what if the sentence is: “The scientist argued that the data suggests the universe is ___.” Now you need to know cosmology, the scientific method, and the specific debate. If you can predict the next word perfectly, you have to know everything about the topic. Perfect prediction requires perfect understanding.

Information theory backs this up. There’s a deep equivalence between prediction and compression. The better you can predict a sequence, the fewer bits you need to store it. And the fewer bits you need, the more you’ve compressed the data. Compression, in turn, is a measure of how much structure you’ve extracted. Compression is not a side effect of intelligence—it’s the definition.

This isn’t just theory. The ngrok blog recently published a piece called “Compression Is Prediction” that lays it out cleanly. And Grant Sanderson has a video series on why compression is intelligence. When you train an LLM on billions of tokens, you’re essentially asking it: “Find the shortest program that can generate all of this text.” The result is a model that has learned the underlying rules of language, reasoning, and even world knowledge—because those are the most efficient ways to compress the data.

So the next time someone says LLMs are just stochastic parrots, ask them a simple question: “What would a perfect predictor look like?” It would be a model that could compress all of human knowledge into a single, elegant representation. That’s not a parrot. That’s a universal compressor. The “just a next-token predictor” insult is actually the highest compliment you can pay.

But wait—doesn’t this mean we’re just glorifying pattern matching? No. Because pattern matching at scale, when the patterns are rich enough, becomes something else. It becomes abstraction. It becomes analogy. It becomes creativity. Every new idea is just a recompression of existing concepts into a novel form. LLMs don’t lack original ideas. They’re just better at the one thing that produces all original ideas: compression.

I’ve seen this firsthand. I fed a model a dense research paper on the second law of thermodynamics and asked it to explain entropy to a five-year-old. It didn’t just regurgitate sentences. It compressed the idea into a metaphor: “Think of your room. If you never clean it, everything gets messier. That’s entropy.” That’s not a stochastic parrot. That’s understanding.

So stop apologizing for being impressed. Stop nodding along when some pundit tells you these models are just fancy autocomplete. The math is clear: compression is prediction, and prediction is intelligence. The critics are right about one thing: LLMs are next-token predictors. They just don’t realize that’s a superpower, not a flaw.