Somewhere in your company this week, a product manager connected an AI agent to your customer inbox. There was a permissions screen. They clicked through it. No lawyer reviewed it, no risk assessment preceded it, and nobody told them what they had just agreed to.
What they agreed to is this: if the agent tells a customer something false, promises a refund you do not offer, or says something it should not, the consequence lands on them. Not on the agent, and not on the vendor. The supply of people willing to carry that kind of exposure is finite, set by the org chart, and it does not scale with the model. That, not intelligence, is the real ceiling on how far agents go.
Moreover, an another interesting study by EY’s 2026 cited “AI Risk and Governance Survey found that 91% of senior AI executives said their organizations were already using agentic AI in pilots or deployment, yet 49% of those using agents said their existing governance frameworks had not been updated to specifically address agentic AI risks. More strikingly, 85% said at least some of their AI agents were already executing actions without real-time human involvement. The question is therefore moving from whether companies will deploy agents to who remains accountable when those agents act.
It is not the model
Stalled agent projects usually get blamed on model quality, data readiness or change management. Gartner forecast last year that “over 40% of agentic AI projects will be canceled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls.” Model capability did not make that list, and none of the three failure modes is something a smarter foundation model would fix. Its 2026 hype cycle found that only 17% of organizations have deployed AI agents, yet more than 60% expect to deploy within two years. That gap is not waiting for a better model.
The permission you did not know you signed
Organizations’ have always granted authority through consequence. A manager signs off a budget because they answer for it if it is wasted. A sales director closes deals because they can be fired for bad ones. Authority and accountability travel together, and that pairing is what lets a company delegate without supervising every decision.
An agent breaks the pairing. It can act, decide, send and pay, but it cannot be fired, sued or embarrassed. When it errs, the consequence does not vanish. It relocates onto a named human who often did not make the decision, did not know it was being made, and would not have approved it. Every permission granted to an agent is an act of private underwriting, usually by someone who has no idea they signed.
This is why capability scales and deployment does not. It is also why the second agent is often harder to ship than the first, even when it is simpler: the first one used up the obvious volunteer. The bottleneck moved from the model to the org chart, and nobody relabelled it.
The market has already priced it
The people who price risk for a living got here first. Verisk’s ISO teams developed general liability endorsements that let carriers exclude generative AI exposures, effective January 2026. They are showing up quietly on renewal policies without the fanfare a change of this magnitude would normally receive. Many firms are carrying exposure they assume is covered.
Meanwhile a new market is forming to fill the hole. In February, Chaucer and Armilla launched a structure in which AI-specific liability is covered by a standalone policy backed by Lloyd’s of London, including erroneous outputs, model underperformance and AI agent actions, with dedicated AI limits of $25 million or more per organisation. Munich Re’s aiSure is already putting a price on AI accountability, where businesses can insure models against underperformance beyond an agreed threshold, transferring part of the financial risk of AI failure to an insurer.
Courts and regulators are closing the exit. California’s AB 316, in effect since January 1, bars the legal defense that AI autonomously caused the harm. In March 2026, the UK’s CMA told businesses that consumer law applies whether a customer deals with a human or an AI agent, and that the business is responsible even when a third party designed the agent. And on May 28, 2026, a Munich court found Google directly liable for false statements generated by its AI Overviews.
Put those together and the shape is clear. Accountability can no longer be pushed onto the machine, so it must sit with a person or be bought from an insurer. Authority is becoming something you purchase rather than something you grant. And if it is purchasable, the ceiling on autonomy is set by an underwriting market, not by your org chart.
The test
The fair objection is that plenty of automation already acts without anyone signing each decision. Trading systems, credit scoring and fraud engines move money every second. The difference is that those systems are narrow, statistically bounded and backed by decades of loss data. Agents are general, and generality is exactly what makes them hard to underwrite.
So the firms that move furthest will not be the ones with the best models. They will be the ones that made accountability transferable, by buying it or by creating a role that formally holds it.
Until then, put one question to every agent on your roadmap. When it gets something wrong, who takes the blame? If you can name that person and they know it, you have a deployment. If you cannot, you have a demo.
Authored by Parminder Singh, Co-founder at Redscope.AI