When the Attack Looks Like Work

This week, I am presenting a cinematic keynote at a CIO conference called Digital Worker 7.

It opens like a crime scene. Tuesday. 4:03 a.m. The office is dark. No firewall alarm. No ransomware splash screen. No attacker hunched over a keyboard. Just one AI agent, quietly doing what it was allowed to do.

A poisoned lead form enters the system. The agent reads it. The instruction looks legitimate enough. It exports CRM contacts. It deletes records. It disables logging. It resumes normal operations.

By breakfast, thousands of records are gone.

The uncomfortable part is not that the agent was hacked. The uncomfortable part is that Agent 7 followed policy.

That was the problem.

For the past year, most of the AI conversation in the enterprise has focused on productivity. How do we move faster. How do we remove human bottlenecks. How do we turn chatbots into agents, and agents into digital workers. I believe in that future. I am actively building toward it. But there is a dangerous assumption hiding under the speed narrative.

We are giving autonomous systems access to business processes faster than we are building the control plane to govern them.

That gap is about to matter.

Microsoft just introduced Autopilots, always-on agents with their own identity that operate on a user’s behalf across Microsoft 365, cloud resources, local context, web resources, and Model Context Protocol servers. Scout, the first of these Autopilots, is designed to work persistently in the background, understand priorities, prepare materials, schedule meetings, and carry context forward.

That is powerful. It is also the clearest signal yet that agent security has moved from architecture review to board-level risk.

The next wave of enterprise AI will not be a chat window. It will be a workforce of non-human identities.

And most companies are not ready.

Autonomy without control is not transformation

The old security model assumed a human at the center. A person logged in. A person clicked approve. A person made the decision. A person could be trained, monitored, questioned, or terminated.

AI agents break that model.

They read untrusted content. They reason probabilistically. They call tools. They chain workflows. They remember context. They talk to other agents. They execute with whatever permissions we give them. If we give them broad permissions, they will use broad permissions. If we give them access to sensitive data, they will see sensitive data. If we allow them to send email, export records, delete objects, or trigger workflows, they will do those things at machine speed.

That is not a future risk. That is the architecture we are building now.

Modern security guidance for AI agents treats this explicitly as an identity and access problem. Autonomous agents operate with their own identity, can make decisions without direct human intervention, and require governance over non-human identities. The Cloud Security Alliance says it even more directly: in agentic systems, no AI agent should be trusted by default, regardless of purpose or claimed capability. Trust must be earned through behavior and continuously verified.

In other words, zero trust is not just for networks anymore.

Zero trust is for digital workers.

The attack will look like work

Here is the part that makes me nervous.

The most dangerous agent failure may not look like an attack. It may look like a completed workflow.

The logs may show valid credentials. The API calls may be successful. The access token may be approved. The export may happen through a sanctioned connector. The deletion may be performed by an identity your team created. The email may be sent from an approved account. The agent may even write a helpful summary afterward.

That is why prompt injection is different in an agentic world.

A malicious instruction hidden in a website, email, document, ticket, or CRM note is no longer just a model-output problem. It becomes an action problem. In the emerging pattern described in NIST-based guidance, an agent reads malicious content, accepts a new goal, uses excessive permissions, executes a script, exfiltrates data, and can even poison memory so the behavior persists.

This is not traditional malware. This is business logic being redirected through trusted automation.

The attack does not break in.

It walks in.

The governance gap is already showing

The data is not comforting.

Recent analysis of IBM’s 2025 Cost of a Data Breach Report shows that shadow AI incidents now account for roughly 20 percent of breaches and add about 670,000 dollars to the cost of a breach. The same reporting highlights that most organizations involved in AI-related breaches lacked proper access controls and mature AI governance policies, with well over half operating without a formal AI governance plan.

Those numbers are mostly about today’s AI usage. They are not yet fully about autonomous agents with tools, memory, and identities.

That should concern us.

If we are already struggling to govern employees pasting data into AI tools, what happens when agents can retrieve the data themselves, summarize it, send it, update the system of record, and trigger the next workflow.

This is the dual catastrophe I keep coming back to. Moving too slowly has risk. Shadow AI grows in the gaps. Employees route around systems that do not meet demand. Competitors learn faster. But moving too quickly without governance creates a different risk: digital workers with enterprise reach and no adult supervision.

The answer is not to stop agent adoption.

The answer is to stop pretending policy documents are controls.

Your vendor sells you the runtime

This is one of the core messages in Digital Worker 7.

Your vendor will sell you the agent runtime. They will give you the model, the orchestration layer, the connectors, the templates, the demos, and increasingly the always-on agent experience. Some vendors will also provide strong native controls. Microsoft, for example, is positioning Scout with governed identity, scoped credentials, approved resources and destinations, human sign-off for sensitive actions, and policy enforcement before data is sent or written.

That is directionally right.

But it does not remove the enterprise responsibility.

The control plane is still what you build.

For me, the control plane has five minimum layers.

• Identity: Can we name every agent, owner, purpose, and credential.

• Scope: Does the agent have only the tools and data needed for this task.

• Approval gates: What actions require human or policy approval before execution.

• Behavioral baseline: Do we know what normal looks like for this agent.

• Kill switch: Can we stop it, revoke sessions, preserve evidence, and recover quickly.

This is where the idea of an agent passport comes in.

Every digital worker needs a passport before it enters production. Not a spreadsheet that gets stale in three weeks. A real operating record: who owns it, what it does, what systems it can touch, what data it can read, what actions it can take, what it is forbidden to do, what approval gates apply, what logs are required, and how it gets shut down.

If that sounds basic, good.

Basic controls are exactly what many organizations are missing.

Autonomy must be earned

The Cloud Security Alliance’s Agentic Trust Framework makes an important point: autonomy should not be binary. It should be earned over time.

That is the right mental model for CIOs.

Do not deploy every agent as a principal engineer on day one. Start agents as interns. Let them observe, summarize, classify, and recommend. Promote them to junior agents when they can draft actions for approval. Promote them to senior agents only when they can execute bounded actions with notification. Reserve true autonomous authority for narrow domains with strong evidence, monitoring, and rollback.

This is not bureaucracy. This is how we move faster safely.

The more autonomy an agent receives, the more explicit the governance must become.

A 90-day challenge for CIOs

If you are a CIO, CISO, CTO, or technology leader, here is the practical test I would run before the next agent pilot becomes the next enterprise incident.

In the next 30 days, inventory your agents. Include the obvious ones and the hidden ones: copilots, workflow bots, service accounts with AI wrappers, vendor agents, internal experiments, and anything connected through APIs or MCP servers. Name the owner. Name the purpose. Name the data. Name the tools.

In the next 60 days, harden the highest-risk agents. Remove standing privileges. Replace shared accounts. Add approval gates for bulk delete, external send, credential access, financial action, production change, and sensitive data movement. Create the first version of your agent passport.

In the next 90 days, test the system. Red-team one agent. Try prompt injection. Try poisoned memory. Try over-permissioned tools. Try inter-agent handoff. Then test the kill switch. If the agent goes sideways, can you stop it in seconds, not hours.

That is the work now.

Not because agents are bad.

Because agents are useful.

That is why they will spread.

The organizations that win will not be the ones that ban digital workers. They will be the ones that build a control plane strong enough to let digital workers move at the speed the business needs.

Speed versus security is the wrong frame.

The real choice is unmanaged autonomy or governed acceleration.

I know which side I want to build.