You’ve probably felt it. That eerie, unsettling moment when ChatGPT seems to actually understand what you’re asking. It feels like magic. It feels like intelligence. But what happens when you strip away the sprawling complexity of human language and just ask an AI to play the simplest game in the world?
Recently, a developer decided to find out. They trained a Transformer model—the exact same architecture powering the AI revolution—to play tic-tac-toe. No rules were hardcoded. No strategic heuristics were fed into the system. The AI was simply fed thousands of games as flat sequences of tokens, and left to figure it out.
We don’t have artificial intelligence; we have autocomplete on steroids.
The results are fascinating, not because the AI became an unbeatable grandmaster, but because of how it got there. The Transformer didn’t learn the concept of a ‘winning row’ or a ‘fork trap.’ It didn’t deduce the geometry of the board. It just learned that a specific sequence of tokens is statistically likely to follow another. When the AI places an ‘X’ in the center, it isn’t plotting a trap. It’s just predicting the next most probable move based on its training data.
This is the dirty secret of the AI industry. We are so blinded by massive language models writing Shakespeare and passing the bar exam that we forget what they actually are: sequence predictors. When you apply a language model to a deterministic, rule-based game like tic-tac-toe, the illusion of comprehension cracks.
A transformer doesn’t know what a winning row is. It just knows that ‘X in the center’ is usually followed by ‘O in the corner’.
Most analysts completely miss why tic-tac-toe is the ultimate testbed for Transformer limitations. It’s not about the game itself; it’s about the state space. When you have a 3×3 grid, the total number of possible board configurations is incredibly small. There’s nowhere for the model to hide. In a massive language model, the sheer volume of data obscures the fact that the AI is just pattern-matching. But in tic-tac-toe, you can watch the attention mechanism struggle with symmetries and forced moves in real-time.
You can literally see the model fail to generalize. If you rotate the board, the AI might suddenly make a rookie mistake, because the token sequence looks different, even though the strategic state is identical. It exposes the fragile, brittle nature of pattern-matching versus true strategic reasoning.
Complexity is a mask. When you shrink the state space down to a 3×3 grid, the magic trick disappears.
For developers building autonomous agents, this isn’t just an academic curiosity—it’s a warning. If you’re relying on a Transformer to enforce strict rules, manage state transitions, or execute flawless logic, you are building on sand. The model will hallucinate a move that breaks the game because it ‘feels’ like the right sequence, not because it follows the rules.
We need to stop anthropomorphizing these systems. The wonder of seeing a machine play a game without being told the rules should be tempered by the chilling reality of how it achieves it. It doesn’t think. It doesn’t strategize. It just predicts.
And if we aren’t careful, we’ll bet our entire digital infrastructure on a machine that doesn’t even know it’s playing a game.
FAQ
Q: If the AI can play the game perfectly, doesn't that mean it learned the rules?
A: No. It learned the statistical likelihood of moves. It's mimicking the shape of a winning game, not calculating strategy. It can't tell you why a move is good, only that it frequently appears in successful sequences.
Q: What's the practical implication for developers building AI agents?
A: It proves that LLMs fail at deterministic logic unless heavily guided. You cannot rely on them for strict rule adherence or state management without external, hardcoded verification tools.
Q: Are you saying LLMs are useless for logic and reasoning?
A: Not useless, but fundamentally mismatched. Using a language model for strict logic is like using a paintbrush to tighten a bolt. It might work eventually, but you're using the wrong tool for the job.