Enterprises have no shortage of AI pilots; what they lack, argues Niraj Nagrani, Chief Data and AI Officer at Altimetrik, is an operating system to run them on. In this conversation with CIO&Leader, Nagrani makes the case that scaling AI isn’t a modeling problem at all: it’s an engineering, governance, and accountability problem that a stronger LLM can’t solve. Drawing on Altimetrik’s HFS Research study of 505 Global 2000 executives, in which 83% said they depend on partners to move fast and 80% flagged unclear accountability when AI goes wrong, Nagrani unpacks ALTi AIOS™, a five-layer framework spanning experience, context, orchestration, governance, and substrate. He talks through what breaks when pilots meet real data, why context means more than a knowledge base, how governance can be built in rather than bolted on, and what he got wrong 18 months ago about the “AI-first enterprise.”
CIO&Leader: Enterprises have already poured budget into LLMs and pilots. What specifically breaks down when they try to scale those pilots into production, and why isn’t a better model the fix?
Niraj Nagrani: The break happens the moment a pilot leaves the demo room and meets a real, messy input. A pilot is fed curated data, run by a forgiving internal audience, and judged on “did it impress the room.” Production asks a completely different question: can the business depend on this every single day? That’s not a model problem; it’s an engineering and operating problem: identity and access for the agent, audit trails for every decision, cost and latency at real volume, resilience when a tool call fails, and a way to prove to risk and compliance that the system behaves the way you say it does.
That’s why a better model doesn’t close the gap. Every enterprise today has access to roughly the same frontier models. The advantage was never going to come from the model-it comes from operationalizing it. In our HFS Research study of 505 Global 2000 executives, 83% said they depend on partners to move quickly, and 80% said accountability is unclear when an AI partner’s system makes the wrong call. Only 14% of enterprises told us they have a clear, accountable AI strategy, and Gartner puts the ROI-measurement gap in similar territory; most organizations can’t point to what they’re getting back. Swapping in a stronger model doesn’t fix any of that. It’s still sitting on the same fragmented context, the same missing guardrails, and the same absent ownership.
CIO&Leader: How do you define an “AI Operating System” in concrete terms — what are its core components, and how is it different from an orchestration layer or a MLOps stack that enterprises may already have?
Niraj Nagrani: I’d put it plainly: AI strategy without a contextual operating system is just intent. An AI Operating System is the layer that turns that intent into enterprise execution — it coordinates agents, data, models, and human decision-making as one governed system, rather than leaving each pilot to invent its own stack.
Concretely, ALTi AIOS™ is five interlocking layers, roughly 45 engineering disciplines deep. Experience is where people and agents work together — product and UX for non-deterministic systems. Context is the enterprise’s reality represented as a semantic graph — retrieval, knowledge graphs, ontologies, semantic layers, agent memory. Orchestration is where many agents and humans execute one workflow — multi-agent architecture, tool and MCP engineering, routing, failure recovery. Governance is permissions, audit and testing — identity, policy-as-code, evals, red-teaming, regulatory compliance. And Substrate is the engineering and production foundation underneath all of it — LLMOps, fine-tuning, cost engineering, SRE for AI systems.
An orchestration layer is one slice of that — it routes and sequences agent calls, nothing more. An MLOps stack is largely Substrate — deployment pipelines and model lifecycle management, built for a world of versioned models with deterministic outputs. Neither one, on its own, has an opinion on enterprise context, on agent identity and policy, or on the human-agent experience layer. An AIOS wraps all of that into one operating layer so a use case isn’t rebuilding context, governance, and orchestration from scratch every time.
CIO&Leader: Context, orchestration, governance, and AI engineering are named as the four pillars of AIOS — which of these is hardest for enterprises to get right today, and why?
Niraj Nagrani: I’d push back gently on isolating just one — the hardest thing for enterprises today isn’t context, orchestration, or governance in isolation; it’s getting context, orchestration, and governance to operate as one composable system instead of three isolated systems run by three different teams on three different roadmaps.
Each pillar is solvable on its own. You can stand up a knowledge graph. You can stand up a multi-agent orchestration layer. You can stand up an audit and policy framework. What’s genuinely hard is the seams between them — an orchestration layer routing work to agents that are pulling from context that’s already stale, or a governance framework that audits a decision after the fact instead of being wired into the same runtime that produced it. We saw this directly with a banking and financial services client building governed, conversational BI across multiple agents: the hard part was never any single layer; it was making sure the semantic layer feeding the agents, the orchestration deciding which agent answered which question, and the audit trail proving why were architected together from day one. Build any of the three on its own timeline, and by the time you connect them, you’ve usually had to rebuild at least one.
AI engineering — the production substrate underneath all three — is what makes that integration durable rather than a one-off integration project. But the pillar enterprises most consistently underestimate is the coordination cost of context, orchestration and governance, not any one of them alone.
We’ve seen the same pattern with a payments processor building multi-agent dispute and fraud-resolution workflows. The agents needed real-time transaction context, coordinated hand-offs between fraud, risk and customer-service agents, and an audit trail regulators could inspect on demand — and none of those could be retrofitted independently. Sequencing context first, orchestration second, and governance third as separate workstreams would have meant redesigning the transaction graph a second time once the audit requirements surfaced. Designing all three together from the start is what kept it on schedule for production.
CIO&Leader: Walk us through what “context” means architecturally in ALTi AIOS™: is this about data pipelines, retrieval systems, enterprise knowledge graphs, or something broader?
Niraj Nagrani: It’s broader than any one of those — we describe the Context layer as your enterprise’s reality represented as a semantic graph. It’s nine engineering disciplines working together, not one technique. Context engineering itself is the core discipline: structuring and assembling exactly the right information for a model at the right moment, managing context windows and multi-turn coherence. Underneath that sits retrieval engineering — RAG, vector and hybrid search — plus enterprise ontology engineering and knowledge graph engineering, which capture entities, relationships and business rules well enough for an agent to reason across them, not just look them up.
Then there’s the semantic layer, which lets agents query enterprise data in natural language without ever seeing raw schema, data product engineering across data mesh and data fabric, memory architecture for stateful agents that need to remember across a session or a workflow, synthetic data engineering for fine-tuning where real data is sparse or sensitive, and data fabric and integration engineering actually to connect it all to brownfield systems. Pipelines and retrieval are in there, but on their own they’re flat — they answer “what does this document say.” A knowledge graph and ontology let an agent answer “what does this mean for this customer, this claim, this shipment” — that’s the difference between information and context.
We see this most clearly in life sciences, where we’ve built agentic operations for batch release, deviation and CAPA workflows, HCP engagement, and CSR evidence review. None of those work off a document index — they depend on a knowledge graph connecting batch records, quality events, and regulatory obligations, because the question an agent has to answer is never “what does this document say,” it’s “is this batch safe to release given everything we know.” That’s context as we define it — broader than a pipeline, and broader than retrieval.
CIO&Leader: Governance is often where AI initiatives stall due to compliance and risk aversion. How does an AIOS approach enable enterprises to govern AI without slowing deployment velocity?
Niraj Nagrani: The stall happens when governance is a manual review that sits between a working pilot and a production launch — a committee reads a deck, asks questions no one has data to answer, and the initiative sits for months. An AIOS approach removes that bottleneck by making governance something the system proves continuously, not something a person certifies once.
In practice, that means policy-as-code instead of policy-in-a-document, so guardrails execute automatically rather than waiting on a reviewer. It means identity, permissions, and audit logging are built into an agent’s runtime from day one, so “can this agent take this action” is answered by the system itself. It means every agent carries evals, benchmarking and red-teaming results as it moves from prototype to pilot to production, so risk and compliance are looking at evidence, not intentions. And it means that human-in-the-loop escalation and refusal logic are built into the workflow, so a human reviews genuine exceptions rather than every transaction. When trust is provable at any point in time, deployment velocity and governance stop competing — the velocity comes from not having to re-litigate trust for every new use case.
We saw this play out with a banking and financial services client rolling out governed, multi-agent conversational BI across regulated customer and transaction data. Because identity, access policy, and audit logging were engineered into the AIOS from day one rather than layered on after the pilot, the compliance and risk teams could continuously review evidence instead of holding up every release for manual sign-off. That let the bank move from a single proven use case to a broader, governed rollout without a corresponding slowdown in its risk posture.
CIO&Leader: What does a fragmented AI pilot typically look like inside a large enterprise, and what are the earliest warning signs that a pilot won’t survive contact with production?
Niraj Nagrani: It looks like a dozen well-intentioned teams solving the same problem a dozen different ways — one team’s own prompt library, its own vector store, whichever model an engineer preferred, no shared context layer and no shared way to evaluate success. Each one looks impressive in isolation. None of them are built to talk to each other, which is exactly the brownfield reality largest enterprises live in.
The earliest warning signs are consistent across industries. There’s no single accountable owner for the outcome — which tracks with the 80% of executives in our HFS study who told us accountability is unclear when an AI system makes the wrong call. Success is being measured by a demo reaction rather than a repeatable KPI based on real data. The pilot is hard-wired to one model with no fallback and no cost or latency ceiling. Security, identity, and audit logging weren’t part of the design — they’re being added after the fact, if at all. And there’s no evaluation framework, so nobody can say with evidence whether the system got better or worse after the last change. Any one of those is a flag. Two or three together, and the pilot is very unlikely to survive contact with production.
CIO&Leader: How do you measure whether an AI deployment is actually “production-ready” — what metrics or thresholds separate a proof of concept from something the business can rely on?
Niraj Nagrani: We treat production-readiness as a gate a system has to pass, not a feeling. Our Agent Development & Assessment Framework moves a use case through three stages with different bars at each one: a two-week sprint to a working prototype, a pilot where KPIs are locked and tested against real production data rather than curated demo data, and only then production, with CI/CD, cloud orchestration and a global SLA behind it.
The metrics that matter at the production gate span both performance and trust. On performance: task success rate against a labeled benchmark, cost per task at real volume, latency under real load, retry and failure rates, and — for multi-agent systems — memory effectiveness and coordination quality between agents. On trust: audit coverage of every decision path, the human-in-the-loop escalation rate and whether it’s trending down for the right reasons, drift detection against the original evaluation baseline, and guardrail violation rate. A proof of concept only has to clear the first set, once, on a good day. Something the business can rely on has to clear both sets, continuously, under the conditions the business actually operates in.
We see the same gate in manufacturing and industrial deployments, where autonomy expands from a single machine to a full plant. A pilot proving that one line’s throughput improved is not the same as proving that task success, safety-guardrail adherence, and escalation rates hold steady across every line, every shift, and every operator — that’s the threshold that actually separates a proof of concept from plant-wide autonomous operations.
CIO&Leader: Altimetrik works across OpenAI, Gemini, Anthropic, and Databricks. How does an AIOS avoid vendor lock-in while still letting enterprises get the most out of each model provider’s strengths?
Niraj Nagrani: History is a useful guide here — enterprises never standardized on one database, one cloud, or one programming language, and I don’t think AI ends up any different. The way we design for that is with a model router and an AI gateway layer sitting between our agents and any given provider’s API, rather than hard-wiring a workflow to a single model. A task hits the gateway, and the router — not an engineer’s original choice from six months ago — decides which model handles it: a frontier reasoning model for complex analysis, an open-weight or on-prem model where data has to stay in-house, a small domain-specific model where speed and determinism matter more than general intelligence, or traditional deterministic software where generative AI was never the right tool to begin with.
We built exactly this pattern for a life sciences client — their agentic operations for batch release, HCP engagement, and evidence review run on ALTi AIOS™ and are live today on one enterprise AI platform. Still, the workflows themselves are portable across model platforms because the gateway, not the model, owns the contract with the business logic. The discipline that prevents lock-in is asking, for every task, which model delivers the best business outcome, which one satisfies our security and data-sovereignty requirements, which one balances intelligence, latency, and cost appropriately, and which one should continue to improve over time using the enterprise’s own data. The test I’d apply to any AI vendor relationship is simple: can you swap the underlying model without losing everything you’ve built? If changing models means rebuilding the workflow, the gateway has failed. If it’s just a routing change, it’s done its job — which is why we maintain trusted relationships with OpenAI, Google Gemini, Anthropic and Databricks rather than an exclusive one. The enterprise owns the gateway and the data; no single provider does.
CIO&Leader: As Chief Data and AI Officer, how has your own view of the “AI-first enterprise” changed over the past 12–18 months? What did you get wrong early on that you’d now do differently?
Niraj Nagrani: Eighteen months ago, I would have told you the priority was velocity — get more pilots running, get more forward-deployed engineers embedded, prove the art of the possible as fast as you can. That instinct wasn’t wrong, but I underweighted what comes after it. More pilots and more engineering intensity feel like progress, but they’re inputs, not outcomes — I’ve seen us describe this pattern as “FDEmaxxing,” mistaking engineering effort for enterprise transformation, the same way the industry once mistook token consumption for business value. Neither guarantees the outcome on its own.
What I’d do differently is sequence the investment. I’d put context and governance foundations in place before scaling pilot volume, not after — because every pilot we didn’t build on shared context and shared governance became a system we had to re-govern individually later, one at a time, which is a much more expensive way to get to the same place. I also underestimated how much of the gap was organizational rather than technical — 505 executives told us in our HFS study that accountability, not model capability, was their biggest open question. I now treat production-readiness as a requirement from the first week of a pilot, not a checklist we run once the pilot already “works.”
CIO&Leader: Three to five years out, what does an enterprise running on a mature AI Operating System actually look like day-to-day — and what’s the biggest organizational (not technical) barrier standing in the way of getting there?
Niraj Nagrani: Day-to-day, it looks quieter than people expect. Agents and employees work inside the same workflows rather than agents being bolted on as a separate chat window. A new use case takes weeks, not quarters, because it draws on existing context, governance, and orchestration rather than being built from scratch. Humans spend their time on genuine exceptions and judgment calls that got escalated to them, not on approving routine transactions one by one. Every decision an agent makes leaves an audit trail by default, so a compliance question is answered by pulling a record rather than launching an investigation. And leadership can see, continuously, what a given AI investment is actually returning — not through an annual survey, but through the operating system’s own observability. We already see an early version of this in manufacturing, where autonomy expands from one machine to a full plant only once the operating system — not any one pilot team — can prove it holds every day, across every line.
The barrier to getting there isn’t technical — the architecture I’ve described exists today. It’s ownership. Right now, most AI initiatives are owned at the line-of-business or project level, and 80% of the executives in our HFS research told us accountability is unclear when an AI system gets something wrong. Getting to a mature, enterprise-wide operating system means someone — a CDAIO, a governance council, an operating committee — has to own the platform the way IT once had to learn to own the network, rather than every business unit owning its own AI. That’s a change-management and organizational-design problem, and in most enterprises I’ve worked with, it’s a harder problem than any engineering problem.