AI Benchmarks Are Dead. Welcome to ‘Pinky-Promise’ Evaluation.
The latest AI coding benchmarks claim to test models on ‘real-world enterprise codebases’ to prevent cheating. But by hiding the data, they’ve made the tests completely unverifiable. If you’re choosing AI tools based on these scores, you’re flying blind.