AI Is Probabilistic, Your Core Systems Can’t Be

Enterprises are racing to adopt AI, but the systems underneath it has not kept pace. Bain’s research finds that 72% of CIOs see legacy technology debt as their biggest barrier to transformation. With more than 12 billion lines of COBOL still powering mainframe ecosystems, the core cannot simply be switched off. In this interview with CIO&Leader, Devipad Tripathy, SVP and Global Head of Digital Engineering & Enterprise Transformation at Xebia, argues that the real obstacle is trust. AI is probabilistic, he says, and modernizing business-critical systems needs deterministic quality gates, rigorous reverse engineering and human-in-the-loop checks. He explains why earlier modernization programs stalled, why the “verification tax” on AI-generated code keeps rising, and why sovereign, open-weight models matter in regulated sectors. He also describes how Xebia’s Ace platform analyzed 1.2 million lines of COBOL in a week, a tenfold productivity gain. And he sets out a practical, phased plan for CIOs to modernize without disrupting business as usual.

Devipad Tripathy, SVP and Global Head of Digital Engineering & Enterprise Transformation at Xebia

CIO&Leader: Bain’s research finds that 72% of CIOs see legacy technology debt as their biggest barrier to transformation, yet AI spending continues to rise. So why are enterprises adopting AI faster than they can modernise the systems underneath it?

Devipad Tripathy: We are seeing significant gains. Let’s be practical about it: yes, AI is delivering gains and helping us. It is helping us visualise or implement things we were sceptical about two or three years back. It is also helping us find new ways of doing things we were not able to do two to three years back. So yes, there is a shift in the thought process of CIOs and leaders.

However, when it comes to enterprise systems, which we call legacy, they have been there in their ecosystem, stable and running, for the last 30, 40, 50 years. They are on legacy technologies as well, which creates a skills-related problem; we do not have the skilled people available to take them forward. AI has also not matured to the point where it can directly say, “Okay, I will go ahead with transforming or modernising my existing ecosystem.”

That is because people fail to understand one very important aspect while adopting AI: AI is probabilistic. To implement something or take on a challenge, you have to make it deterministic so you know your decisions are data-driven and will help you modernise existing ecosystems.

They are adopting AI and adding layers on top of existing legacy ecosystems to automate a few things and improve efficiency. But most are not taking that step directly. Many of them have taken guided or well-defined steps, following a staged or phased approach to modernising their ecosystems. Changing the core is a risky business, and that’s what is limiting the whole thought process.

CIO&Leader: When AI is layered on top of decades-old systems, what breaks first: data quality, integration, performance, or security?

Devipad Tripathy: The biggest problem people are facing is trust: can they trust AI adoption to modernise their existing ecosystem?

Let me give you an example. There are more than 12 billion lines of COBOL code in the world right now. These COBOL programs power our mainframe and legacy ecosystems; they are huge, handle complex processes, and are stable. So the challenge is not that they are facing challenges of that sort. Instead, the challenge is to add new features and scale existing ecosystems as the digital landscape and customer experience change. Organisations want to delve into new areas. “Can I change my whole ecosystem on day one itself?” That’s the problem they are facing.

So what they are trying to do is break the bigger problem into steps and phases: “These are the low-hanging fruits. I’ll start experimenting in this area. Based on the success and the learnings, I’ll take it to the next area.” They are taking a phased, wave-based approach to adopting AI and modernising their existing ecosystems with its power. That’s what is really happening right now.

This is a trend. And believe me, when you digitally modernise your existing ecosystem, you even adopt the concept of a digital twin. I have modernised part of my existing ecosystem and built a new one. Is it an apples-to-apples comparison? So you create a digital twin and compare the outputs of the legacy and new ecosystems for three or six months, depending on the criticality of the business function you have transformed. It is not a step that can be taken right away, because it is business-critical. A lot depends on these systems, and I have to do it in a way that doesn’t impact my business-as-usual.

CIO&Leader: Business-critical systems in banks or hospitals cannot simply be shut down. How do you modernise them while keeping operations running?

Devipad Tripathy: You have to analyse the existing ecosystem. Then you have to identify the low-hanging fruit from an experimentation point of view. And that’s what people are doing: “This is a less critical ecosystem, part of the bigger ecosystem. Let me first experiment there. Based on that learning, I’ll start getting into the core of the whole enterprise ecosystem,” and do it in a phased manner.

There are also situations where it’s a highly coupled environment, so decoupling is necessary. Is this a less dependent ecosystem within the enterprise? Or how do I even make it less dependent if it is tightly coupled? I won’t say “Modernise it.” First, I’ll have to decouple it if it is a tightly coupled ecosystem. I’ll decouple it from the existing one and take a few steps to keep the dependency to a minimum. Then I’ll begin the modernisation journey and gradually replace the entire codebase. It cannot be a big-bang approach in any of these ecosystems.

It also depends on the industry. As you mentioned, BFSI and healthcare are business-critical applications. The industry that takes risks in this area is retail. Retail is ready to take those risks. Retail is revamping its existing ecosystems at a much faster pace compared to industries like BFSI, healthcare, and energy and utilities. Every one of them is analysing and understanding its ecosystem, so right now there’s a lot of focus there. And believe me, AI is also being used to understand their existing ecosystems.

CIO&Leader: Why has legacy modernisation so often failed or stalled, and what changes now with AI in the picture?

Devipad Tripathy: We have not been trying to modernise these ecosystems only recently. We have been trying to modernise them for the last 20 to 30 years, even before the advent of Gen AI and what we have seen in the last four years. During that time, solutions were specifically designed to modernise COBOL-based, mainframe-based, and AS/400-based ecosystems and migrate those landscapes to the latest technologies so those applications can be hosted.

However, the challenge they face is: “Have I been able to replicate what was there?” Once that is addressed, adding new features becomes a second topic. How do I benchmark whether what existed has been replicated?

Reverse engineering plays a very critical role in this. Reverse engineering is nothing but understanding your existing ecosystem: what it carries, its nuances, the hidden pieces of code, the underlying logic, the business rules, and the validations. All of that needs to be understood first so that you can replicate it and do forward engineering on top of it. Forward engineering is easy. Understanding what you need to implement in forward engineering comes from reverse engineering, which is the biggest bottleneck. If you can understand what is there, it’s very easy to implement it in the forward engineering life cycle. Modernisation journeys have failed simply because they were unable to understand their existing ecosystems.

With the advent of Gen AI and the power of LLMs, both frontier models and the open-weight models that have been coming in, that job has become easier. But I’ll come back to the same point: AI is probabilistic, and it comes at a cost. Can I trust the output given during the reverse engineering process?

AI is now being used across enterprise ecosystems. But people have started asking, “I have generated something using AI, but is it good enough? How do I even say this is going to work?” So the review process has become very cumbersome. Code has become easy to generate using Gen AI; however, I have to review it. We have started calling it the verification tax. The verification tax has begun to increase because we need traceability to demonstrate that whatever AI generates is good enough.

That is what is really making things more difficult for enterprises: “Can I trust that process? Is that process mature enough to take into the forward engineering life cycle?” That is the reason I brought up the deterministic aspect I mentioned five minutes ago. How can I make probabilistic AI deterministic with the right quality gates in place, so I can understand the existing ecosystem, replicate it, and improve the modernisation journey? That’s where the crux of the problem lies. And this is where we excel, because we have understood those nuances through our 25-plus years of engineering experience. We said, “This is what the problem really is; let’s try to address it.”

CIO&Leader: How does ACE reverse-engineer legacy code, and how do you ensure nothing critical is lost or misread?

Devipad Tripathy: That is the crux of the whole Ace ecosystem. The Ace ecosystem has evolved over three years. We are a pure engineering company with 25-plus years of experience, and we know the nuances involved. We jumped into this life cycle of using and adopting AI early, so we understood the nuances of using AI that’s why we are the only company talking about the deterministic aspect. You can search across the globe: no one says, “I am using AI, but it is deterministic.” We are the only ones who can guarantee deterministic gates across the whole process, on top of probabilistic AI, which helps us deliver a risk-free migration or modernisation.

When I talk about the reverse engineering process, I talk about the billions of lines of code that are out there. Understanding an existing enterprise legacy system is the crux of the problem. Multiple modules, structures, and codebases can contribute to the delivery of your enterprise’s core platform. Now, when you do it purely with AI, AI has context issues. It takes the context and begins to understand your ecosystem; however, it starts to lose that context if the codebases are large or complex, with many dependencies and integration issues.

That is where the problem lies: identifying the right business functions, business rules, and validations; how the whole business process works; and which areas are truly critical and must be part of the ecosystem. These codebases have been around for 30-40 years, and multiple people have worked on them, so identifying the real bottlenecks is an issue.

We use AI and have our own ways of bringing in the deterministic aspect. We have built solutions like Code Insights that help us understand the codebase and properly document it. We even integrate with best-in-class tools like CAST. CAST has been around for 25-plus years. It is a code engineering insights tool that supports many languages. So if customers are comfortable with CAST, we bring that in as well. We mix and match both processes: we use AI and validate it with CAST and Code Insights, our homegrown tool, to ensure a proper end-to-end understanding of you’re codebase.

From there, we pull out and create the technical specifications, the business specifications, and everything else that can be extracted from the codebase. Because, believe me, code is the source of truth, with less documentation. So from the code, you have to understand everything.

Once we have done that, we validate it against what is happening in production. Have we mapped each business process, the business rules and the business understanding with the business folks who already exist in the enterprise? That validation is done. When we see that this is now the source of truth we have extracted and that it is accurate, we perform forward engineering using Ace. So, Ace handles your reverse engineering as well as your forward engineering, and it integrates best-in-class tools to bring in a deterministic aspect, rather than relying solely on AI and creating a mess down the line. Because AI is also maturing, you see frontier model providers come up with a new model now and then. Why do you think they come up with new models? Because they are also optimising at the back end: there were problems; they are working through them, adding parameters and doing a lot of other things. So you cannot just unthinkingly rely on AI to do your work properly.

CIO&Leader: Enterprises often use different vendors for analysis, transformation, and deployment. So what does handling the whole journey under one roof change in practice?

Devipad Tripathy: Let me give an example of what is normal, common practice across the industry. In a software engineering life cycle, different personas handle different stages. Business analysts cover requirements, business understanding, and domain aspects. Architects design the whole ecosystem, which is a different role altogether. UX designers, developers, and DevOps engineers then build, deploy, and support the solution. Finally, maintenance and support teams handle post-production go-live and maintain those applications in production. Different personas are involved, based on the skills required at each stage.

Similarly, if I am trying to do something that requires multiple skills, or has multiple complexities that need to be addressed across the life cycle, I’ll bring in those that are best in class. Let me talk about Ace. When we were building Ace, we said it is an engineering platform. Many tools and frameworks already exist, so do I need to redo something that already exists? Why not make it part of the whole process? If something is good enough, make it part of the process; if something is not there, I’ll build it.

Similarly, people bring in multiple vendors based on the problem statement: “If I have someone with very good expertise in, let’s say, the COBOL and mainframe area, why not utilise their expertise, their tools and frameworks in the process, to make it faster with the right kind of quality gates, so that my whole journey is risk-free?” I mitigate those risks early, and that decision is taken early in the life cycle.

However, now, with AI coming in and the whole process maturing, we have been able to do that. We can do it end-to-end now because we have that expertise and have resolved the nuances across different areas. But people need to have confidence that we can do it. Only then will enterprises reach out to you, saying, “They have the end-to-end process. Why go to multiple vendors rather than one vendor who is going to take it on end to end?” It de-risks them. That’s why the operating models are changing right now. We are introducing outcome-based models: “We are going to give you the outcome.” We talk about the outcome rather than T&M (time and materials), fixed price, and so on. We have started talking about outcomes so that the organisation’s risk is also limited, and we have the confidence to offer you a risk-free modernisation journey or software engineering life cycle.

CIO&Leader: Where does human expertise still matter in an AI-led transformation, and where can automation safely take over?

Devipad Tripathy: It really depends on the business area and the business problem you are trying to solve. Right now, we’re hearing about agents, the agentic ecosystem, and agentic AI, and a lot is happening in this space. We have not yet reached a stage where you can directly rely on AI to do your work. To move to the agentic way, or the AI way, of automating or enhancing your business processes, you also have to make many structural changes within your organisation: the underlying data ecosystem and the governance that need to be in place. What are the risks if I do it automatically? What mitigations need to be in place across the process? What boundaries do I need to set for an AI automation or agentic ecosystem so it doesn’t cross them and start doing unnecessary things? We have seen and heard about those things, so trust, compliance, governance, and data play a very important role; they are foundational concepts.

Now, who knows these things better? One part is the understanding that already exists in code bases, documents, and so on. Then there is tacit knowledge, which lies only with the people who have been working in that process and has never been documented. That’s where human decisions come into play: is this decision right or wrong? Automation has been around for the last 15 to 20 years, but we still come back and ask whether it accounts for all the tacit knowledge humans possess.

Iit has to be the right mix. If a process is small and less business-critical, it can be automated end-to-end with lower risk and clear mitigations already in place. However, if it is a business-critical application, you need HITL (human-in-the-loop) gates, where people verify before a process moves to the next step of execution.

What people in the industry are doing right now is documenting this tacit knowledge and these human actions so that the learning ecosystem across the whole AI-based journey keeps getting updated. As a result, less human involvement is required down the line because your systems are also learning in the process.

The human in the loop is critical because we haven’t reached maturity yet. For business-critical applications where decisions have an impact, such as in the healthcare industry, we haven’t reached the point where I can say that a robot or piece of equipment, because it is AI-driven, can take action during an operation or surgery. Because it is critical; it impacts a life. Or it impacts a business process, whatever it may be, in an industry like BFSI. That’s where the dependency on people comes in. They have to come in, verify, and make the ecosystem learn: ‘This decision has been taken because of these five parameters,’ so that over a period of time, the system also learns from the data that this is the right path to take down the line, when full automation comes into the ecosystem. The decision should be based on the process and the area of work you are working on.

CIO&Leader: Many enterprises worry about sending sensitive code and data to external AI tools. How do you keep it within the company’s secure boundary?

Devipad Tripathy: That’s why we have started talking about sovereign AI and private AI. We are trying to understand the nuances of these entire ecosystems, as well as their costs. I mentioned the context issues and the learnings earlier.

If I use frontier models like OpenAI, Gemini, or Claude, we are in a constant process of sending and retrieving information. There are ways to secure it, and these frontier models provide extensive governance, trust, and compliance processes on top of that. Even with hyperscalers, AI gateways have come in to keep compliance, governance, PII, and everything else in check.

Still, you want that system to learn in line with your enterprise. There are COTS solutions and SaaS-based solutions. But when an enterprise puts them into practice within its ecosystem, it has to customise them, because every enterprise has its own way of operating. There is no such thing as a one-size-fits-all solution. The same problem exists with AI. With frontier models, we face these issues because they address everything from a multimodal perspective. But if I am in BFSI, retail, or healthcare, I want domain-specific capabilities within my ecosystem.

That’s the reason for open-weight, open models hosted on an NVIDIA ecosystem or a sovereign data centre, so that you can control all of this. You can teach them in line with your enterprise processes; you have full control over them, and there’s less vendor lock-in. That is the approach a lot of our customers are trying and experimenting with, and it is making them more efficient. Your data resides within your ecosystem; it’s not going out. You have your enterprise standards, guidelines, and governance in place to control those stacks and technologies, and you can fine-tune the models to meet your enterprise needs. That is playing a critical role. Enterprises are exploring these solutions.

From our point of view, we understood this very early, and we have made Ace, our engineering solution, sovereign on the NVIDIA stack as well, using open-weight models. So if a customer, say in BFSI or healthcare, doesn’t want to use Ace as a SaaS-based model, they can put it within their own ecosystem, within their data centre, with all the governance, security, and compliance in place, and do their work within their ecosystem.

CIO&Leader: How much complexity, cost, and time can automated reverse engineering realistically save? And what should CIOs use to measure whether modernisation is paying off?

Devipad Tripathy: It’s always about the scope. If you are scoping properly and breaking your problem statement into multiple small blocks, it is very easy to cater to this whole ecosystem.

With reverse engineering, believe me, based on the experience we have had (I think there are a few articles published on our website on the exercises we have done for multiple customers), it has become very easy now, with the process, with AI, and the deterministic aspects we have brought in. 1.2 million lines of COBOL code were analysed within a week, and the entire documentation was generated. This was a dream. It means we are seeing a 10x increase in productivity. That’s the kind of thing we are looking at.

Understanding an existing ecosystem used to take months, in some cases years. There have been modernisation journeys that are still ongoing after six or seven years because they are uncovering problems at a later stage. If you can uncover your problems and nuances at an early stage, the next part, forward engineering, becomes easy. Yes, we are seeing a lot of gains, more than 10x in most cases.

CIO&Leader: BFSI, healthcare, automotive, and retail all face pressure to modernise core systems. What should each sector learn, and where should leaders begin?

Devipad Tripathy: When we talk about the modernisation journey, it is very important to understand your existing ecosystem. Understanding your existing ecosystem is the first thing. Then you have to classify it as critical, less critical, or medium criticality. It’s typical T-shirt sizing: these are the applications in the landscape; let’s break the bigger problem into smaller chunks and start the modernisation journey with the least impacted ones. Apply the learnings.

As I said, every organisation is customised. It’s not that there is a thumb rule you can follow and replicate. Yes, there are thumb rules: the best practices, standards, guidelines, and processes. But your enterprise is still very customised to your business needs.

Break the problem into smaller chunks. Identify the low-hanging fruit to help you learn and adapt before you go to a big-bang scale down the line, so you know what you are trying to address and the journey you are going to take over a period of, let’s say, six months or one year. Breaking down the problem is the biggest thing people are missing right now.

Have the right tools, standards, and governance in place. When modernising the ecosystem, ask: how do I make it agentic? How do I make it future-ready, so that I do not need to revisit the whole ecosystem five years down the line? Otherwise, I won’t get the ROI on the investment I’m planning.

It’s all about understanding your ecosystem first, breaking it into smaller chunks, addressing the smaller, less critical chunks first, applying the learnings from these modernisation journeys, and taking the next steps to make it more efficient. The more learning you incorporate, the faster, more capable, and higher-quality your subsequent modernisation journeys will be, so you can make decisions, add future features down the line, and make it more digital and customer-facing.

It’s a journey you have to take. Learnings are very important. Deterministic aspects are very important. Governance is critical to understanding the nuances properly and making data-driven decisions throughout the process. That’s what I typically talk about when I meet CIOs and CTOs and plan their enterprise transformation journeys.

Share on