Avoiding the coming agentic disasters by creating an architecture for accountability

From risk assessment to operational governance, organizations must build accountability into every stage of agentic AI deployment to unlock value while minimizing risk
6 min 所要時間
Tarun Goyal
Tarun Goyal
Vice President, HCLTech
6 min 所要時間
Avoiding the coming agentic disasters by creating an architecture for accountability

When I see statistics that indicate 97% of enterprises are exploring agentic AI, but only 12% have centralized control, I get really worried. AI is so powerful. With AI-powered coding, solutions can be developed amazingly quickly. But without built-in guardrails it is clear that many firms are headed for disaster.

The fundamental risk isn't the failure of agents to scale or to automate work effectively. Rather the biggest disasters will come when agents succeed but go in the wrong direction with no one accountable.

This isn’t a new issue. In 1979, IBM used this message in an internal presentation: “A computer can never be held accountable. Therefore a computer must never make a management decision.”

The solution is not to slow everything down. In the world of AI, computers will make management decisions. The challenge is to make sure that such decisions are made with full accountability. Based on what has worked for early adopters, I suggest a four step program to prepare for driving agentic technology safely at high speeds.

Step 1: Assess risks

The first step is to understand the nature of agentic technology so the risks can be assessed and categorized. AI doesn’t just execute rules like RPA or earlier generations of business process automation technology, where the rule makers were accountable. AI is driven by huge amounts of data and makes decisions and recommendations in ways that are frequently not transparent. The agent designer can’t be held accountable because they don’t know what the agent will do. They only know what information it will rely on and what model will be used.

The accountable party then must be those who deploy agentic technology to have impact inside business operations.

Step 2: Apply a risk model

The second step is to use the following three-tier risk model categorizes the type of risk so the appropriate accountability can be designed:

  • Tier 1: Advise only — agent recommends, human decides
  • Tier 2: Act with human-in-the-loop — agent executes, human reviews
  • Tier 3: Fully autonomous — agent acts, humans audit retroactively

Inside these tiers, risks also must be assessed with respect to regulatory exposure, data sensitivity and reversibility of the action.

Step 3: Create an org chart for accountability

The third step is to create an org chart for agentic accountability. Every autonomous agent needs a human owner, a defined scope and an audit trail — just like an employee does. As agents proliferate across IT, finance, supply chain and customer service, the absence of this accountability map creates compounding risk. The CIO and CTO should design this framework before unmanaged sprawl creates accountability debt.

The tactics to ensure and manage accountability and implement controls but be built-in to the harness for creating agents and the control plane to operate and govern them.

Such a control plane should include capabilities for:

  • Lifecycle management
  • Context sharing
  • Authentication
  • Observability
  • Kill-switches

The ability to precisely specify what agents can and cannot do and to be alerted when things go wrong separates enterprises who scale safely from those who create chaos.

Ideally, the productivity of AI with respect to development of agents should lead to a much larger footprint of automation inside the enterprise. New roles will be needed to manage the operations of this footprint including: agent trainers, autonomy auditors, workflow architects and embedded AI ethicists. The leadership challenge is not just re-skilling the workforce for these operational tasks, but also defining what 'working alongside agents' means at an organizational level.

Step 4: Proactively remove bottlenecks to agentic development

Step four is to recognize that few companies are ready to drive at high speed with respect to agentic development because of bottlenecks related to access to legacy infrastructure and readiness of data to be used by LLMs.

More than 38% of agentic AI projects are stalled by legacy infrastructure. Unlike humans, agents cannot 'work around' broken APIs or fragmented data — they expose and amplify every deferred technical debt. Tech department can accelerate the destruction of bottlenecks by focusing in improving:

  • Data pipelines and data management.
  • The ability of identity/auth systems to work with agents.
  • Improving security, control and scalability of the API layer.

Avoiding agentic disaster

Just letting agentic development rip will result in a car crash when ungoverned agents waste money or make bad decisions that lead to regulatory problems or many other types of messes.

But the right culture for risk tolerance and controls will vary widely based on the nature of a business. Leadership must make the tradeoffs clear so everyone understands the difference between an agent that is responsibly autonomous and one that is recklessly so.

Predictions of agentic disaster are easy to come by. Gartner warns 40%+ of initiatives will be discontinued by 2027 because of weak governance and unclear ROI.

Those who aggressively embrace accountability and all that it requires will be the ones who scale.

Data sources:

  • The brief said that these came from this report: https://www.outsystems.com/1/state-ai-development
  • 97% of enterprises are exploring agentic AI, but only 12% use a centralized platform for control — OutSystems 2026 State of AI Development report (1,879 IT leaders surveyed)
  • 38% of agentic AI projects have stalled due to legacy systems as the primary blocker — OutSystems 2026
  • Gartner predicts 40% of enterprise software will include task-specific AI agents by end of 2026
タグ:
共有:
AI AIと生成AI ブログ Avoiding the coming agentic disasters by creating an architecture for accountability