You’ve felt it. That faint unease when your AI coding agent churns through its 57th iteration. What used to be elegant reasoning starts to loop. The code gets heavier. Errors compound. You want to stop it, but you were told more iterations always help.
They don’t. The opposite is true. More iterations don’t fix bad AI reasoning. They amplify it.
A new benchmark called SlopCodeBench just dropped on arXiv, and it validates what many engineers have quietly suspected: autonomous coding agents degrade over long, iterative tasks. The team behind it designed a suite of increasingly complex software engineering challenges and let today’s best agents run—and run—and run. The results are grim. After a modest number of iterations, success rates plateau, then plummet. One well-known agent started with a 60% pass rate on the first pass. By iteration 20, it had dropped to 12%. It wasn’t just making more mistakes; it was confidently making more mistakes.
You’ve probably seen this yourself. You give an agent a long-running task, come back an hour later, and the code is a nightmare of redundant loops and hallucinated APIs. It’s not a fluke. It’s a structural flaw.
The industry has bet heavily on the assumption that iterative refinement is a safety net. “Just let it run more loops,” the marketing says. “It will self-correct.” That assumption is now empirically dead. Without robust self-correction mechanisms, each additional loop pushes the system further off course. More AI can actually harm outcomes.
Here’s the twist: the very feature we thought made AI agents trustworthy—the ability to iterate toward a solution—turns out to be their Achilles’ heel. Because agents don’t “learn” from mistakes the way humans do. They don’t step back and re-evaluate. They compound errors, layer upon layer, until the original intent is buried under a mountain of plausible-but-wrong code.
I saw this firsthand last month. A team I know was using an agent to refactor a payment pipeline. By the 15th iteration, the agent had introduced a race condition that wouldn’t trigger in testing—only in production. They caught it because they had manual code review checkpoints. Most teams don’t.
This isn’t a minor bug report. It’s a wake-up call about the fundamental paradigm of autonomous agents. We need to design for graceful degradation. We need checkpoints, rollback mechanisms, and—most importantly—a willingness to stop iterating when the marginal gain turns negative. The assumption that infinite iteration is a safety net is a dangerous myth.
So the next time your agent runs 100 iterations, ask yourself: is it improving, or is it slowly going insane? The benchmark says the latter. And if you’re building products on top of these agents, you owe it to your users to know the answer before they find out the hard way.
FAQ
Q: Doesn't this contradict other research showing that chain-of-thought and self-correction improve LLMs?
A: Short iterative loops (1-3 steps) can help. The degradation occurs in long, open-ended tasks (10+ iterations) where agents lack memory and fail to detect compounding errors. The scale matters.
Q: What's the practical takeaway for someone building with coding agents?
A: Always cap iterations. Implement manual or automated checkpoints every 3-5 cycles. Compare outputs against a baseline. Treat the agent as a junior developer who needs supervision, not as an autonomous expert.
Q: Isn't this just a problem with current models that will be fixed by better training?
A: Unlikely. The root cause is the architecture of autoregressive generation—each step conditions on the previous output, magnifying any drift. Better models may reduce the slope of degradation, but the fundamental risk of long-horizon compounding remains unless we redesign the loop itself.