AI Benchmarks Are Lying to You. Here’s the Truth.
The ARC-AGI leaderboard shows models leapfrogging each other, but real-world performance regresses within weeks. The uncomfortable truth: benchmarks are being gamed through training on the test puzzles. If you’re making decisions based on these scores, you’re being misled. Stop trusting the leaderboards. Test your own use cases.