01Borrowed authority
The assistant inherits access nobody granted it.
It was wired up with a service credential so it would 'just work'. Now it reads every row a human never could, and no ticket records the moment that started.
Where it breaks
Autonomy fails quietly, and the logs stay green.
Agents act with credentials nobody scoped, documents rewrite their instructions, and personal records ride out through an integration that looks routine. These are the four patterns we keep finding.
Where it breaks
Nothing alarms when autonomy goes wrong.
There is no stack trace for a decision. The system runs green while doing precisely the wrong thing, and you find out from a customer, a regulator or a bank statement.
02Instruction hijack
A document tells the agent what to do next.
The page it retrieved carries a sentence written for the model, not the reader. The agent obeys. Your prompt never mentioned any of it.
03Quiet data exit
Personal records leave through a helper call.
One integration widens its payload during an upgrade. Identity fields ride along to a vendor you never mapped, and the export looks routine in the logs.
04Machines in motion
Nothing broke. The robot simply improvised.
The task was under-specified, so the fleet resolved the ambiguity itself. Every individual choice was defensible; the combined outcome was not.
The missing control
Everything you own today watches, none of it decides.
Four familiar categories, one shared blind spot: none of them stand between intent and execution while the system is live.
Dashboards
Recorded
They narrate the incident after the money left. Useful for the write-up, powerless at the moment of action.
Prompt rules
Suggested
Text inside a prompt is persuasion. A long context, a clever input or a bad day is enough to talk the model out of it.
Annual audits
Snapshot
A binder describing last quarter's system. Your data flows changed on Tuesday and the evidence never caught up.
Model reviewers
Approximate
Asking a model to police a model doubles both the uncertainty and the bill, and still yields no reproducible verdict.