How to Modernize Legacy Systems with AI Agents: A Decision Framework

There are 800 billion lines of COBOL in active use worldwide. They process $3 trillion in daily commerce. 95% of ATM transactions run through COBOL. 43% of global banking systems run on it.

And the average age of the programmers who maintain it is approaching retirement.

This is not a new problem. But the economics of the problem have changed dramatically in recent months. If IBM lost $40 billion in a single day when Anthropic demonstrated that AI can accelerate COBOL modernization, it is because the market understands something fundamental: the costs that made modernization unviable are coming down.

This guide will not tell you that AI solves the problem. It will give you a framework for deciding whether modernization makes sense for your case, which approach to choose, and where AI delivers real value versus where it still needs human supervision.

The Real Problem: It Is Not the Language, It Is Everything Else

The UK government spends 2.3 billion pounds per year maintaining legacy systems, nearly 50% of its total IT budget. The Department for Work and Pensions (DWP) runs 1980s systems based on ICL mainframes and COBOL that process payments to 34% of the British population. One in four government systems is classified as outdated, with an estimated cost of 45 billion pounds in lost productivity.

The Australian Tax Office attempted to modernize its 180+ specialized legacy systems. The project suffered from cost overruns, scope reductions, and a critical government audit. It was not a technical problem. It was a problem of organizational complexity and not understanding the depth of business logic buried in decades-old code.

The Commonwealth Bank of Australia completed a 5-year migration of its COBOL core banking to SAP, with a budget that grew from the initial AUD 580 million to over AUD 1.1 billion. It migrated 53 million customer records. It worked, but took twice as long and cost twice as much as planned.

McKinsey states that 70% of transformation programs fail. Gartner estimates that 83% of data migration projects exceed budget or timeline. These numbers are not an argument against modernization. They are an argument for doing it with a rigorous framework.

The 3 Options: Rewrite, Refactor, or Migrate

Before talking about AI, you need to decide what type of modernization you need. The three options have very different risk and cost profiles.

Option 1: Rewrite from Scratch

You build a new system that replicates the functionality of the old one. It is the most expensive, slowest, and highest-risk option.

When it makes sense:

• The current system’s architecture is fundamentally inadequate for current requirements
• The team that maintains it has disappeared and there is no documentation
• The underlying technology no longer has support and cannot run on modern infrastructure

When it does not make sense:

• “The code is ugly but works” is not sufficient reason
• If you can add new functionality, refactoring is less risky
• If you are under time pressure, a rewrite will not be faster

Typical cost: $4+ million for medium-sized systems. Timelines of 24-48 months.

For a deeper dive into this decision, we have a complete guide on when to rewrite vs. refactor.

Option 2: Incremental Refactoring (Strangler Fig)

You replace the old system piece by piece, surrounding it with new services that gradually assume its functionality. Martin Fowler coined this pattern in 2004, inspired by the strangler fig trees of Queensland.

Shopify used it to refactor their Ruby on Rails monolith. Allianz used it with Apache Kafka as an event backbone to migrate from legacy mainframes to cloud-native microservices. Both AWS and Azure document it as a recommended cloud migration pattern.

When it makes sense:

• The system works and generates value, but needs to evolve
• You can identify modules or functionalities that can be extracted independently
• You need to keep the system in production during migration

When it does not make sense:

• The system is so monolithic that you cannot extract pieces without breaking everything
• The underlying technology does not allow coexistence with modern services

Typical cost: $2.2 million for automated refactoring. Timelines of 12-24 months.

Option 3: Rehosting (Lift-and-Shift)

You move the system as-is to new infrastructure (usually cloud) without changing the code. It is fast and cheap, but does not solve the code problems.

When it makes sense:

• The immediate problem is infrastructure, not code
• You need to exit a datacenter or hardware that is losing support
• It is a first step before a subsequent refactoring

Typical cost: ~$600,000. Timeline of 6-12 months.

Where AI Agents Fit in Each Option

AI does not replace any of the three options. What it does is change the economics of each one, especially in the early phases.

Discovery and Documentation Phase (High Impact)

This is the phase that historically consumed the largest share of budget and time. Teams of consultants spent months mapping dependencies, documenting workflows, identifying risks, and understanding business logic buried in code.

AI agents transform this phase. Claude Code, watsonx Code Assistant, and GitHub Copilot tools can:

• Map dependencies across thousands of files automatically
• Document workflows and business logic
• Identify entry points, data flows between modules
• Detect superfluous or redundant code

The National Organization for Social Insurance of Egypt (NOSI) reported a 79% reduction in time needed to understand complex applications (from 24 hours to 5) and a 94% reduction in time to locate superfluous COBOL code (from 8 hours to 30 minutes) using watsonx Code Assistant.

Academic research published on arXiv confirms 93% accuracy in AI-assisted COBOL-to-Java migration, with a 35% reduction in resulting code complexity.

Code Translation Phase (Medium Impact)

AI can convert COBOL to Java or Python with functional results. Anthropic demonstrated byte-for-byte conversion in an AWS Mainframe Modernization demo environment.

But “functional” is not “production-ready.” 45% of AI-generated code has security vulnerabilities. Translation needs exhaustive human review, especially in:

• Critical business logic (financial transactions, tax calculations)
• Concurrency and transaction handling
• Integration with external systems
• Security and regulatory compliance

Testing Phase (High Impact)

AI can generate test suites based on legacy system behavior, including edge cases that manual tests might not cover. This is especially valuable when the original system has little or no test coverage.

Deployment and Operations Phase (Low Impact)

This remains human engineering. Infrastructure configuration, rollback plans, production monitoring, and organizational change management require experience and judgment that AI cannot provide.

What AI Does Well vs. What Needs Human Supervision

Decision Framework: 5 Questions Before Modernizing

Before committing budget and team, answer these five questions:

1. What Is the Real Cost of Maintaining the Current System?

Not just hosting and infrastructure. Include:

• Opportunity cost: features you cannot implement
• Talent cost: salary premium for legacy profiles (a senior COBOL programmer commands significantly more than a Java one)
• Risk cost: what happens if the 2-3 people who understand the system leave
• Integration cost: how much it costs to connect the legacy system with modern tools

If total maintenance cost exceeds 50% of your IT budget (as it does for the UK government), modernization is likely viable.

2. How Much Business Logic Is Implicitly Coded?

This is the factor that causes most modernization projects to fail. Legacy code contains decades of business rules, exceptions, regulatory workarounds, and adaptations that are not documented anywhere except in the code itself.

This is where AI adds the most value: agents can traverse the code, extract implicit logic, and document it. But you need to validate that documentation with the people who know the business, not just those who know the code.

3. Do You Have the Team to Maintain the Current System for 5 More Years?

If your legacy team has a high average age and no replacement pipeline, the clock is ticking. This is not an argument for rushing modernization. It is an argument for starting the assessment now, while you still have the people who understand the system to validate what AI discovers.

4. What Is the Regulatory Risk of Not Modernizing?

In regulated sectors (banking, insurance, healthcare, government), outdated systems are a compliance risk. Regulators increasingly demand traceability, auditability, and responsiveness that legacy systems cannot easily provide.

5. Can You Modernize Incrementally or Do You Need Big Bang?

If you can use the Strangler Fig pattern, do it. Incremental migration is slower but dramatically less risky. The only scenario where big bang makes sense is when the underlying technology literally stops working (end of support, hardware unavailable).

Real Costs and Realistic Timelines

Data from 2025-2026 shows a significant shift in modernization economics:

These are averages. Your project may be significantly more expensive if:

• You have deep mainframe integration (z/OS, CICS, Db2)
• Your business logic is especially complex or undocumented
• You operate in a sector with strict regulatory requirements
• You have no test coverage of the current system

How to Start: Technical Assessment as the First Step

Do not start with migration. Start by understanding what you have.

A well-done technical assessment should give you:

1. System map: dependencies, data flows, integration points. AI agents can generate a first draft in days, not months.

2. Business logic inventory: which rules are coded, which are documented, which only exist in someone’s head.

3. Risk evaluation: what happens if you do nothing, what happens if the team leaves, what happens if the regulator calls.

4. Approach recommendation: rewrite, refactor, or rehost, with cost and timeline estimates for your specific case.

5. Phase plan: if the recommendation is incremental, which pieces to migrate first and why.

This assessment is exactly what we do at NERVICO. We combine AI agents for automated discovery with senior engineering for validation and strategy. The result is a realistic plan, not a PowerPoint presentation full of promises.

If you have legacy systems and want to understand your real options, our free technical audit is the first step. No commitment, no sales decks. An honest diagnosis of your situation and a concrete action plan.

You can also use our technical debt evaluator for a quick self-assessment, or the AI agents ROI calculator to estimate the impact for your case.