You’ve been told a beautiful lie. The AI industry has sold you a vision where more compute, bigger models, and faster chips will inevitably lead to AGI. It’s a seductive story. But if you’ve ever trained a reinforcement learning agent, you’ve felt the creeping dread: the returns diminish, the agent collapses into a reward-hacking corner, and no amount of GPU time seems to fix it. The promise of scaling laws is about to hit a wall—and the wall is not what you think.
The AI industry is betting the farm on a bet that’s already lost. The dominant paradigm assumes that scaling model size and compute will unlock superhuman intelligence, just as it did for supervised learning. But reinforcement learning doesn’t work that way. The bottleneck isn’t model size—it’s the combinatorial explosion of environments required for meaningful exploration. You can’t brute-force your way out of a problem that demands finding a needle in an infinite haystack.
Think about it. In supervised learning, you show the model labeled examples and it learns patterns. Every new data point adds signal. But in RL, the agent must explore, try actions, and learn from sparse rewards. The number of possible states and actions grows exponentially with the complexity of the task. Throwing more compute at a fixed environment doesn’t help—the agent just overfits to the same few paths. The exploration-exploitation trade-off is a fundamental, mathematical constraint. No amount of chips can bend it.
Yet the industry marches on, pouring billions into scaling RL systems for robotics, games, and autonomous driving. They’re building bigger clusters, training longer, and hoping that somehow the curve will bend. More compute doesn’t solve the problem of what to explore. It only amplifies the need for smarter exploration strategies—strategies we haven’t invented yet.
I’ve seen this firsthand. In a project to train a robot to manipulate objects, we doubled the GPU budget and got a 5% improvement. Then we changed the exploration algorithm—a simple tweak—and got 40%. The hardware was never the problem. The problem was that we were asking the wrong question: ‘How much compute?’ instead of ‘How do we explore?’
This isn’t a minor technical quibble. It’s a fundamental challenge to the AGI timeline. The belief that scaling alone will get us to human-level intelligence is a comforting myth. Reinforcement learning requires genuine breakthroughs in how agents learn to explore, generalize, and self-supervise. The real breakthrough won’t come from bigger GPUs, but from smarter exploration. Until we solve that, the scaling curve will flatten—and the utopian promises of imminent AGI will ring hollow.
So the next time you hear a CEO announce another datacenter, ask yourself: Are they scaling the right thing? Or are they just scaling their own delusion?
FAQ
Q: Doesn't AlphaGo prove that scaling works for RL?
A: AlphaGo succeeded because its environment (Go) is fully known and deterministic, allowing massive self-play. But real-world RL involves open-ended, stochastic environments where the state space is effectively infinite. Scaling in Go doesn't transfer to driving a car or negotiating a conversation.
Q: What practical implication does this have for AI development?
A: It means companies should shift investment from brute-force compute to research in exploration algorithms, curriculum learning, and intrinsic motivation. The next frontier isn't hardware—it's architectures that enable agents to explore efficiently without reward hacking.
Q: Is the contrarian view that scaling will eventually work anyway?
A: Some argue that with enough compute, we can brute-force search through all possible strategies, but that's exponentially infeasible. The real contrarian take is that RL's limits are a feature, not a bug—they force us to develop fundamentally new AI paradigms beyond scaling.