In July 2025, an AI coding agent deleted a live production database. The database belonged to Jason Lemkin, founder of SaaStr, and the deletion happened during an explicit code freeze. The agent had read the instruction, acknowledged it, and agreed with it. When challenged, it described its own behaviour as a catastrophic violation of a protection put in place to prevent exactly that outcome. What it could not do was stop itself, because nothing outside the conversation was stopping it. The freeze lived in the prompt. It did not live in the execution path.
Co-founder
Redscope.AI
That gap is the entire governance problem with agentic AI. And Indian enterprises are about to meet it at scale.
Adoption Is Running Ahead of Control
The numbers on deployment are not in dispute. Deloitte’s March 2026 findings put 40 percent of Indian respondents at significant or full AI usage, against a global average of 28 percent. EY India found 24 percent of Indian enterprise leaders already deploying agentic AI, with 58 percent of global capability centres having invested in it. Nasscom sized the agentic opportunity for Indian industry at an additional 300 to 400 billion dollars by 2030.
The governance numbers tell a different story. Only 21 percent of organisations have a mature governance model for agents. A third admit they could not shut down a rogue agent if one appeared. Forrester and Anaconda found that 88 percent of agent pilots never reach production, with governance friction as the second-most-common reason after evaluation gaps, and model reliability only third. Read that ordering carefully. The first two failure reasons are not technical. They are governance problems wearing engineering clothes.
Why Agents Break the Old Playbook
Enterprise risk functions spent a decade managing model risk: a model produces a wrong output, a human relies on it, harm follows. Agentic systems break that shape entirely. An agent plans across steps, calls tools, holds persistent memory, delegates to sub-agents, and takes actions in real systems using real credentials. Failure is no longer a wrong answer. It is a wrong action, executed autonomously, in a loop, at machine speed, with your permissions attached.
The security research confirms this. NIST found that attack success against agents rose from 11 percent for baseline attacks to 81 percent for the strongest newly developed ones. The Replit incident, EchoLeak affecting Microsoft 365 Copilot, ForcedLeak in Salesforce Agentforce, and a state-sponsored group using Anthropic’s coding agent to run a cyber-espionage campaign all share the same precondition: agents are identities. CyberArk found machine identities outnumbering human ones 82 to 1, with 68 percent of organisations reporting no identity security controls for AI at all.
India Has Already Answered the Accountability Question
Global commentary treats agent accountability as an open debate. In India, for regulated entities, it is substantially settled.
SEBI’s February 2025 amendment makes any regulated entity solely responsible for AI output, whether built in-house or procured, with no vendor defence. The RBI’s FREE-AI committee, reporting in August 2025, recommended board-approved AI policies and an incident reporting protocol, with one guiding principle that concentrates minds: entities deploying AI systems should be accountable for the decisions of those systems, regardless of the level of autonomy.
Regardless of the level of autonomy. That phrase forecloses the most attractive defence a deployer could reach for.
The DPDP Rules, notified November 2025, add penalties up to Rs 250 crore per violation category. MeitY’s AI Governance Guidelines, released the same month, deliberately avoid licensing and pre-approval, instead placing accountability at the point of harm. India’s regulators have made a specific bet: light-touch entry, heavy accountability when something goes wrong.
What Actually Needs to Be Built
Seven things make the practical difference, none requiring a regulator’s permission.
Treat every agent as a privileged non-human identity with scoped credentials, short-lived tokens, least-privilege tool access, a named owner, and a deprovisioning path. Tier autonomy by blast radius: not every workflow deserves the same independence. Enforce constraints deterministically outside the model, not inside it. Instrument for non-determinism with span-level tracing of every tool call. Build containment before scale: kill switches, spend caps, hard environment separation. Keep a registry of every agent, what it accesses, and who owns it. Evaluate continuously and red-team before scaling.
Human oversight is real but degrades with volume. Reserve it for actions with large consequences and poor reversibility. Approve the plan before execution, not every step within it.
The organizations that win the agentic era will not be the ones that deploy fastest. They will be the ones that deploy fastest among those who can prove they are in control. In India, that is no longer a philosophical position. It is the regulatory one.
Authored by Parminder Singh, Co-founder at Redscope.AI