You probably don’t think about COBOL. You should. Every time you swipe your card, file your taxes, or check your bank balance, there’s a decent chance you’re interacting with code written before man landed on the moon. And now, some of the largest institutions on Earth are handing that code to AI and asking it to translate it into Java.
What could go wrong?
Everything. That’s what.
A new study out of arxiv confirmed what every grizzled mainframe veteran already knew: when you feed COBOL into an AI migration tool, it faithfully reproduces every bug, every hack, every undocumented workaround that’s been accumulating like sediment since the Carter administration. But that’s not even the worst part. The AI also introduces entirely new bugs of its own, because — and this is the part nobody in the boardroom wants to hear — AI is non-deterministic. It doesn’t translate. It guesses. Repeatedly. At scale.
The hardest part of legacy modernization was never the translation. It was understanding why someone in 1987 decided to divide by zero and catch the exception as a feature.
Let me explain what’s actually happening here, because the research paper buries the lede in academic hedging.
The study tested AI migration on COBOL programs ranging from 300 to 4,000 lines. That’s adorable. The IRS alone has roughly 160 COBOL programs averaging 230,000 lines each. Real enterprise legacy systems span dozens of files, interleave with assembler, and contain logic that no living human fully understands. One commenter on the study pointed out that their production code uses assembler “in its innards” — good luck feeding that into your favorite LLM.
But here’s the twist that nobody’s talking about.
Everyone focuses on whether AI copies bugs or creates new ones. That’s the wrong frame. The real blind spot is this: the COBOL programs themselves were never “correct” in the first place. They are approximations of business rules that have been patched, hot-fixed, and jury-rigged for decades. There was never a single clean version of the truth. There was only whatever worked on the day someone needed it to work, documented in a spiral notebook that was lost in a flood in 1994.
AI migration isn’t translation. It’s a game of telephone with a system that never had a correct answer to begin with.
Think about what that means. You’re asking a probabilistic model to reproduce the behavior of a deterministic system that was never actually deterministic in its intent. You’re asking a machine that hallucinates to faithfully copy the logic of a machine that was already lying.
At JAX 2026, someone from the Sparkasse Group presented their AI-driven COBOL migration like it was a victory lap. “Everything was working smoothly,” they said. I’ve heard that phrase before. It’s what people say right before the audit reveals that the new system has been silently miscalculating interest on 40,000 accounts for six months.
Another commenter on the study cut straight to the point: “Maybe fix the bugs before swapping the average COBOL dev for the average Java dev.” It’s funny because it’s not funny. The average COBOL developer is 58 years old and retiring. The average Java developer has never seen a mainframe transaction system in their life. You’re not just migrating code. You’re migrating institutional knowledge that exists only in the muscle memory of people who are leaving the building.
And that’s the real catastrophe lurking here. It’s not the AI. The AI is doing exactly what AI does — pattern-matching across statistical space and producing plausible-looking output that’s correct 80% of the time and catastrophically wrong 20% of the time. The catastrophe is the assumption that migration is a translation problem. It’s not. It’s an archaeology problem. It’s a forensic investigation into decades of business decisions that were never written down because the person who made them thought they’d be there forever.
You don’t migrate legacy code. You excavate it. And if you skip the excavation, you’re not modernizing — you’re building a skyscraper on a foundation of undocumented sand.
The only realistic path forward, as one practitioner noted, is incremental, step-by-step migration using tools like Cursor with heavy human oversight at every stage. Not “throw 230,000 lines at GPT and hope.” Not “automate the boring parts.” Slow, painful, line-by-line archaeology where every business rule is excavated, documented, tested, and verified before the old code is allowed to die.
That’s not the pitch anyone wants to hear. The pitch they want is: “AI will eat your technical debt for breakfast.” The pitch they need is: “Your technical debt isn’t even written down, and no tool — AI or otherwise — can read the mind of a programmer who died in 2011.”
So here’s where I stand, clearly and without apology: if your AI migration strategy doesn’t include a human auditing step that is at least as expensive and time-consuming as the migration itself, you are not modernizing. You are gambling. And the house always wins.
The systems running your taxes, your mortgage, your retirement fund — they deserve better than a game of telephone with a hallucinating machine. They deserve the slow, unglamorous, deeply human work of understanding what they actually do before anyone — human or AI — is allowed to rewrite them.
The future of legacy modernization isn’t AI doing the migration. It’s humans doing the understanding, and AI being a very fast, very fallible assistant that never gets the last word.
FAQ
Q: But won't AI get better at this? The models are improving fast.
A: Better models don't solve the fundamental problem. You can't train a model to reproduce logic that was never documented. The bottleneck isn't AI capability — it's the absence of a ground truth to train against. No amount of scaling fixes missing institutional knowledge.
Q: So what should enterprises actually do with their legacy systems?
A: Incremental, human-supervised migration. Move piece by piece, document every business rule, test against the old system's behavior, and never let AI make an unreviewed decision. It's slower and more expensive than the pitch deck promises. That's the price of not blowing up your core systems.
Q: Is the research even meaningful? 4,000 lines of COBOL isn't real enterprise scale.
A: Exactly. The study tested toy programs and still found serious issues. Real enterprise COBOL systems are 50-100x larger, interleave with assembler, and contain decades of undocumented patches. If AI struggles with 4,000 clean lines, imagine 230,000 messy ones. The research undersells the problem.