You’re sitting at your desk, staring at a screen. You’ve just been handed the keys to an AI system smarter than every human who has ever lived. You can make it more capable, or you can make it safer. You cannot do both. What do you do?
That’s not a thought experiment from some Oxford philosophy seminar. That’s the opening move of a browser-based strategy game built by a former banker with zero game development experience, and it will ruin your afternoon in the best possible way.
The uncomfortable truth nobody in AI safety wants to admit: you cannot align a system that is designed to be smarter than you, because every constraint you impose makes it less useful, and every freedom you grant makes it more dangerous.
The game, posted quietly on Hacker News by its creator — a redundant banking professional who spent six weeks teaching himself game development from scratch — does something that a thousand academic papers, policy briefs, and Twitter threads have failed to do. It makes you feel the problem in your gut.
You don’t read about the alignment dilemma. You live it. Every click is a trade-off. Every decision branches into consequences you didn’t anticipate. You start confident. You end rattled.
Here’s what happens when you play it: you begin by trying to optimize. You give the system a bit more autonomy because the results are impressive. Then something goes sideways. You pull back, add guardrails, and suddenly the system is a glorified calculator. So you loosen the leash again. The cycle repeats. You’re not solving a puzzle — you’re trapped in a loop that mirrors exactly what real AI labs are experiencing right now.
Alignment isn’t a technical problem waiting for a clever engineer to solve it. It’s a strategic game theory nightmare where every optimal move is politically and ethically repulsive.
Think about what that means in practice. The people building the most powerful systems on Earth — OpenAI, Anthropic, DeepMind — are not working on a solvable engineering challenge. They’re playing a game where the rules are designed to make you lose. The more capable the model, the harder it is to control. The more controlled the model, the less it matters. There is no middle ground. There is only a spectrum of acceptable losses.
The game makes this visceral because it removes the abstraction. You’re not reading about “existential risk” in a footnote. You’re clicking a button and watching the fallout. You’re choosing between a system that can cure diseases but might decide humans are the problem, and a system that’s perfectly safe but can’t do anything worth building it for.
The real alignment debate isn’t about whether we can control superintelligence. It’s about whether we should be building it at all — and almost no one with skin in the game is willing to say that out loud.
That’s the twist the game delivers without ever lecturing you. The winning strategy, the one that actually preserves human flourishing, might be to walk away from the table. To not build the thing. To accept that some games are rigged and the smartest move is not to play.
Try saying that at an AI conference. Try telling a venture fund that just poured two billion dollars into a frontier model lab that the optimal strategy is to stop. The room will go quiet. Someone will change the subject to “responsible scaling.” The conversation will drift back to comfortable technical territory — RLHF, constitutional AI, interpretability research — because those are problems that feel solvable. They have papers. They have frameworks. They have the comforting illusion of progress.
But the game doesn’t let you hide behind frameworks. It forces you to make the call, again and again, until you realize the call is the problem.
A former banker with no credentials in AI safety built something in six weeks that the entire alignment community has struggled to communicate in years. That’s not because he’s smarter. It’s because he understood something fundamental: people don’t change their minds from data. They change their minds from experience. And the experience of making these choices, of feeling the weight of every trade-off, of watching your best intentions produce your worst outcomes — that’s what shifts something.
You don’t need another white paper on AI alignment. You need to sit down, play this game for twenty minutes, and feel the walls close in around you.
The creator didn’t set out to make a political statement. He set out to make a game. But the game became a statement anyway, because the alignment problem is not something you can interact with honestly and walk away unchanged. The choices are too stark. The trade-offs are too real. The conclusion — that maybe the smartest thing humanity can do is stop reaching for the god machine — is too uncomfortable to ignore once you’ve felt it for yourself.
Go play it. Then ask yourself a question that no amount of technical sophistication will answer: if the winning move is not to build it, why is everyone still playing?
FAQ
Q: Isn't this just fearmongering about AI? Real labs have safety teams and frameworks.
A: Safety teams and frameworks are exactly what the game exposes as insufficient. They address the symptoms of the alignment problem while leaving the core paradox untouched: you cannot safely build a system smarter than yourself, because control and capability are inversely related. Frameworks make people feel better. They don't change the math.
Q: What does this mean for developers and policymakers actually working on AI?
A: It means the current framing of alignment as an engineering problem is a category error. If alignment is fundamentally a game theory trap, then the practical implication is that regulation should focus not on making AI safer to build, but on questioning whether certain capability thresholds should be built at all. That's a conversation almost no one is having seriously.
Q: So the answer is just don't build AI? That's absurd and unrealistic.
A: It sounds absurd because the entire industry is built on the assumption that more capability is always better. But the game's core insight is that this assumption is the trap. Saying 'don't build superintelligence' is only unrealistic because the economic incentives make it unthinkable — not because it's logically wrong. The uncomfortable part is recognizing the difference.