What do the IRS, banks, airlines, and government infrastructure around the world have in common? They’re all running on systems propped up by antique computer code that nobody dares touch, lest everything come crashing down.
OpenAI thinks it may have a solution. On October 19th, the company touted “substantial improvements” in its latest AI models’ ability to migrate aging software written in older programming languages into modern code, which it claims could help organizations with “maintaining and modernizing older systems.”
That’s a potentially massive deal. COBOL, one of the languages highlighted by OpenAI, remains stubbornly embedded in vast portions of the world’s critical infrastructure. And despite decades of efforts to banish it to history, the language has limped on, simply refusing to die.
“Nobody in their right mind would use COBOL for a new development project.”
It is hard to overstate COBOL’s ubiquity. Short for “common business-oriented language,” COBOL came into being in 1959. Nearly seven decades later, it is still everywhere. According to IBM, COBOL is the foundation for more than 40 percent of online banking systems, supports 80 percent of in-person credit card transactions, and handles 95 percent of ATM transactions. It also underpins many government systems, from Social Security to the IRS to services run by most states. One rough estimate puts the annual cost of COBOL’s inefficiencies to the US GDP at around $105 billion.
But replacing COBOL is fiendishly difficult. The same goes for other legacy programming languages, like Fortran, still in use today for everything from scientific computing and weather forecasting to atmospheric and ocean studies and codes critical to nuclear security. These systems are often too important to risk breaking, and many of the programmers who understand them have long since retired or died.
For all their impressive advances in modern software development, AI coding tools have struggled to make much headway with these legacy systems. But recent developments suggest that may be changing.
In its latest blog post, OpenAI reported promising results migrating COBOL code to more modern programming languages, including C++, Java, Kotlin, Python, and Rust, without changing what the original software does. That last bit is crucial for organizations looking to ditch code like COBOL, and OpenAI acknowledges that “a migration is considered successful only if all tests pass.”
In OpenAI’s own tests, GPT-6.1 Sol achieved 87 percent behavior preservation accuracy at low reasoning effort — how many computational steps the model spends analyzing a problem before giving an answer — climbing to 98 percent at higher reasoning. By comparison, at low reasoning effort, GPT-6 Astra scored around 86 percent, GPT-6 Sol just 21 percent, and GPT-5.6 Sol a mere 5 percent.
“Rewriting code is not the same as replacing a system.”
“These evaluation results demonstrate progress toward more accurate, affordable code migration for teams modernizing the software their businesses depend on,” OpenAI wrote.
The company also reported improvements on its models’ ability to build database software in various programming languages, including COBOL.
Anthropic also sees an opportunity in legacy systems. In February, it published a step-by-step “code modernization playbook” and pitched its coding agent, Claude Code, as a way to migrate COBOL code into modern languages like Java or Python. It offered no performance benchmarks showing how well Claude Code could perform such migrations, but declared that “the economics of COBOL modernization have shifted.” Shares in IBM, which derives part of its revenue helping companies maintain legacy systems, nosedived following Anthropic’s announcement.
Experts who spoke to The Verge agreed that AI tools have made strides in tackling legacy code in recent years. But they cautioned that advances trumpeted by OpenAI this week are a very long way from making it possible to safely replace critical legacy systems built around it.
OpenAI spokesperson Oscar Haines acknowledged in an email to The Verge that legacy code “is still far from a solved problem” and said the company is listening to feedback from developers and enterprises and exploring ways to build better tests and continue improving its models’ understanding of coding languages. Anthropic did not provide comment in time for publication.
“The evaluations shown by OpenAI don’t address the key difficulty: Breaking up a complex legacy system into smaller units that can be replaced individually.”
“Rewriting code is not the same as replacing a system,” said Vadim Zaytsev, an associate professor of software evolution at the University of Twente in the Netherlands. Systems do have a lot of code — Zaytsev said the largest system he’d seen had 250 million lines of COBOL, “definitely not the world’s largest” — but that’s only part of the problem. Code may be written in multiple languages and there’s data, potentially stored in various formats, to consider, he said. There are also technical integrations with other systems and the expectations of the people and organizations to bear in mind.
All that complexity means “AI is not playing a major role at the moment in replacing or migrating legacy COBOL systems,” Zaytsev said.
OpenAI’s evaluations don’t capture this complex reality either, Arie van Deursen, a software engineering professor at Delft University of Technology in the Netherlands, told The Verge. He described the company’s blog as “superficial,” noting it offered few details about the migration task, while its language benchmark tested models on a relatively simple and well-defined task involving systems similar to those they would have encountered in training. He also stressed the huge gap between maintaining an aging system and modernizing it. “Maintaining is of course a lot easier than modernizing,” van Deursen said, calling the tasks OpenAI used to assess its models “lightyears” away from actual modernization. “The evaluations shown by OpenAI don’t address the key difficulty: Breaking up a complex legacy system into smaller units that can be replaced individually.”
The biggest challenge isn’t necessarily rewriting code. “The main challenge is indeed not understanding how the system works,” said Alastair Donaldson, a professor of programming languages at Imperial College London. Such a lack of understanding is a “very common” problem for legacy systems using languages like COBOL, he said, adding that very few people know how to program in them anymore. And while it is everywhere, Donaldson said “nobody in their right mind would use COBOL for a new development project.”
“In many cases even minor differences in system behaviour before and after migration are unacceptable.”
That makes robust testing crucial, Donaldson said. One approach could be to run the same workflows through the old and new systems to check they behave identically. AI could also document its code translation, ideally allowing humans familiar with the legacy language to review the work. Formal verification, which uses mathematical techniques to computationally verify that code behaves correctly, is another option, said Jacopo Mauro, a mathematics and computer science professor at the University of Southern Denmark.
Such methods are already used to verify AI-generated mathematical results, but applying them to legacy code would require a “huge amount of work,” Mauro said. Using those methods would mean code could be checked automatically, Mauro said, adding that “having this will be better than having a human check that the translation is correct.” Until then, he said, “human verification remains essential.”
And when the systems involved are as critical as those running on COBOL, experts say you need to be absolutely certain the replacement works. “In many cases even minor differences in system behaviour before and after migration are unacceptable,” Zaytsev said. While AI can definitely deliver savings and speed things up, the complexity of legacy systems means partial automation isn’t enough.
“Suppose you have a magic AI wand that handles 99 percent of your code,” Zaytsev said. “What do you do with the remaining 1 percent? That 1 percent can be a million lines of code, spread all over the place and subtly failing in more ways you can imagine within your lifetime.”