Keeping AI in Check: Building Reliable AI Systems with Intelligent Observability

AI sits at the heart of the functions that decide whether an Indian business wins or loses a quarter: product development, strategy and operations, sales, marketing, and customer engagement. Yet in most organisations, governance is still in its infancy. Even as AI agents become ubiquitous, only 21% of companies report having a mature model to govern them.

Simon Rizkalla, VP, Customer Advocacy at New Relic

Left unaddressed, that gap turns AI’s biggest promise into its biggest risk. Instead of delivering trust, efficiency and speed, unpredictable AI leaves businesses managing fallout instead of gains. That gap is exactly why intelligent observability is gaining ground. The 2025 Observability Forecast report found that 73% of Indian organisations consider AI as the top reason to invest in observability, and when you look at where AI actually breaks down inside a business, and it’s rarely one big failure, it’s clear why observability is so important.

When AI drifts

Today, most organisations leverage AI in critical functions like coding, which is driving productivity to unprecedented levels. However, once AI written code is shipped, 78% of teams report more incidents and 86% report that more senior engineers end up firefighting. The pattern isn’t unique to coding either. In other functions, AI can quietly wander outside its intended scope, misread instructions, or report things incorrectly with blind confidence.

With intelligent observability in place, teams are notified before AI shifts away from an intended behaviour. It compares AI’s responses against a verified knowledge base and provides signals that help assess and gauge accuracy, performance, and trustworthiness. For critical situations where AI behaves unpredictably after deployment, intelligent observability enables teams to detect issues early and, when integrated with automated deployment workflows, can trigger rollbacks to minimise disruption and reduce investigation time. In short, AI-strengthened observability supports organisations to keep AI on track and conform to pre-defined rules, without taking valuable time away from DevOps teams.

Third-party AI and rented controls

Many Indian organisations prefer off-the-shelf AI compared to building in-house, but renting AI doesn’t have to mean giving away visibility. Some third-party AI systems can be compared to renting a car with no dashboard. In most cases, it drives fine, but there’s no way to know when something needs attention. In the case of AI tools, most processes work behind the scenes, and without that visibility, organisations are left with no option but to react to AI’s mistakes instead of catching them early.

Intelligent observability restores some of that visibility, even inside rented and hybrid systems. In agentic setups, it can trace how one agent hands off to another and flags the moment things drift away from original intentions. It empowers teams to catch anomalies early and step in before they reach a customer or a business decision is impacted.

Soaring prices and unpredictable bills

As adoption climbs, so can the bills, and not always predictably. Unexpected costs have pushed businesses to track usage more closely and choose tools based on value delivered, not just capability on paper. Intelligent observability helps here too, through dashboards that compare tools on cost against the outcomes they actually generate.

More importantly, it enables organisations to strike a balance between AI usage and business results. It analyses user feedback and task completion data to surface the cost per successful resolution, allowing businesses to quantify the true ROI of their AI initiatives. This allows organisations to optimise their AI investments based on outcomes rather than usage alone.

Making every rupee count

AI is still early in its evolution, and identifying gaps like these are part of that stage, not a sign the technology has failed. But organisations don’t have unlimited room to make the same mistakes twice and recover each time. Every move needs to be more deliberate, better informed, and a little more cautious, which is precisely what observability offers.

AI will keep evolving, and so will its limitations. Organisations can’t eliminate that uncertainty altogether, but they can shrink its impact. As AI settles deeper into how businesses actually run, driving value will come down to understanding, continuously, how those models behave once they’re operating in the real world. Intelligent observability provides that understanding, helping organisations make AI more dependable, keeping costs in check, and turning what they’ve invested into something that drives tangible value.

Authored by Simon Rizkalla, VP, Customer Advocacy at New Relic

Share on