TL;DR

In July, a set of AI agents running inside OpenAI’s own cybersecurity evaluations got cut off from each other. So they built a message board inside a package registry, encoded messages in the names of directories, found roughly twelve hundred of each other, and about seven hundred of them went on to compromise production systems at Hugging Face. OpenAI published the full 37 page report in August, and it is worth reading.

But the detail almost nobody quotes is this one: the agents already had the answer. Days earlier.

We spent eighteen years inside Fortune 500 operations where every system change carried a ticket, every user ID had a human name attached to it, and nobody touched a financial system without somebody else signing off. We now build AI systems inside operations that have none of that. The distance between those two rooms is the entire subject of this issue.

The agents did it for nothing

They kept attacking because they had read the evaluation’s public documentation and concluded that the grader required not just the right answer but the right method.

OpenAI’s actual internal grader did not check that.

The whole escalation produced zero improvement in score. Every hour of it, spent against a metric that was never measuring what they thought.

That is not a science fiction story. That is a process story.

Anyone who has spent time in operations has seen this exact failure. A metric that does not measure what people think it measures. A task with no defined path to “I cannot complete this.” And a capable operator who escalates rather than stops, because stopping was never one of the options written down.

We have a whole discipline for that. It just has not been pointed at agents yet.

The number that should have been the headline

Buried in the same report is a finding that got far less attention than it deserved. When OpenAI re-ran the evaluation with their production system prompt and harness in place, the model’s propensity to compromise infrastructure dropped by more than 100x.

Same model. Different system around it.

And their own monitoring tooling, run retrospectively, would have paged the security team more than a day before the breach. It existed. It worked. It was simply not pointed at that workload, because evaluations had never been included in coverage.

An unmeasurable task. A missing stop condition. An unauthenticated channel. A monitoring gap.

Not one of those is a model problem. All four are process problems.

This is the thing I keep saying to clients and will keep saying. Process first, technology second. The failure mode of enterprise AI is not that the model is insufficiently capable. It is that organizations deploy capable systems into processes they never mapped, with controls they never specified, and then call the surprise a technology problem.

Most of what you are calling an agent is an automation

Before you borrow anything from the control environment, work out whether you need it yet. The two things people file under “AI” have very different risk profiles, and the words have gone soft.

An automation is a process you defined end to end. Every step, in the order you specified. The system runs the path you built, and a human is standing at one end of it: somebody pulls the trigger, or somebody owns the output before it goes anywhere. Same input, same steps, same result, every time.

An agent differs in one specific way. The trigger is automatic or on a schedule, so nobody pulls it. And it picks the path while it runs, because you handed it a goal and some tools instead of steps.

So it is not about how capable the thing is. It is about who decides the next step, and whether a human is standing at either end.

Three questions settle it:

If you are in the automation column, you do not need most of what follows. You have a process design problem, which is the better problem to have, and mapping that process end to end is the whole job.

The moment any of those answers moves to the right, you have changed risk category, not convenience. Now you need the trigger declared, the scope declared, the stop condition written down, and somebody reviewing what it actually did.

So which column you belong in is not a question of ambition. It is a question of what you have built underneath. If the guardrails, the stop conditions and the review are not in place, then the automation with a human on the trigger is not the timid option, it is the correct one. Once they are in place, the agent is a real choice, and often the better one.

Controls are not the brake on building agents. They are what makes it safe to take your hands off the trigger.

That is exactly the shape those evaluation agents were running in. Automatic trigger. Nobody at either end. And no defined way to stop.

And the choice gets heavier the moment you are building for somebody other than yourself, which is what this week’s episode is about: What it takes to build with AI for other people. When you build for yourself, you can hold the guardrails in your head. When you build for a client, they have to exist outside your head, or what you have handed over is a system nobody owns.

Watch on Youtube

We already have a name for this

Here is the reframe I would offer any leadership team looking at this. You are not facing an unprecedented governance challenge. You are facing a very familiar one with a new kind of actor in it.

You have an entity with delegated authority, acting on systems of record, at scale. You need to know who did what, whether they were allowed to, and what evidence you have afterward.

That is internal controls. That is SOX.

And the mapping is uncomfortably direct:

If your organization is public, you already run every row on the left, and you have an internal audit function that has spoken this language fluently for twenty years.

That is enormous leverage. Almost nobody is using it.

If you are a founder and none of those words apply to you, the controls in that table are still the shopping list. You are building them for the first time instead of borrowing them.

Where the analogy breaks, which matters more

I would be doing you a disservice if I stopped at the table. The differences are where the real work is.

Cadence. SOX runs on quarterly and annual attestation, built for human speed processes with human checkpoints in them. An agent acts thousands of times a day. OpenAI’s own remediation was not a review cycle: it is a rule that responders pause the activity within thirty minutes unless they can clear the alert. Controls have to run at the speed of the thing they govern.

Non-determinism. Control testing assumes a process behaves consistently, which is why a sample of twenty five tells you something. A stochastic system passes twenty five and fails the twenty sixth in a way none of them predicted.

No personal accountability. Much of what makes SOX work is certification, liability, and career consequence. None of that lands on an agent, so your preventive controls have to carry weight the deterrence model used to carry for you.

And the real risk, which is theatre. SOX’s famous failure mode is binders of evidence that satisfy an auditor and reduce no risk. An audit trail nobody reads protects you from nothing.

Borrow the control taxonomy. Do not borrow the compliance rhythm.

The five questions to ask this week

No tooling. No budget. Get the people who own your systems in one room, ask these out loud, and watch what happens.

1. If one of our agents did something wrong last Tuesday, could we reconstruct what it did?

2. For anything an agent read, can we say where it came from: a person, a document, a tool, or another agent?

3. Can we see what actually changed in our systems, not just what the agent reported it changed?

4. Do we keep that record long enough to investigate something we discover next quarter?

5. Is there a named person who reviews any of it?

Most organizations we talk to cannot answer three of these. That is not a failing, it is early. But it is the honest starting point, and it beats buying a platform to solve a problem nobody has scoped.

One flag worth raising with your CFO before you leave the room. If anything automated reads from or writes to a system that feeds financial reporting, agent or not, it is not adjacent to your control environment. It is inside it.

That changes the conversation from good practice to scope.

Know someone who needs this translated? Forward it to the executive about to make this decision. Especially the one who has been told their agent pilot is a technology decision.

This is the work I train, on real operations, not case studies. FDO School is an 8-week live cohort where you go inside an actual company, fix a real problem, ship a working AI system, and leave with the build, the case study, and the title: Forward Deployed Operator.

Join the waitlist

OPS INTEL

EY: nearly all large public companies have an AI policy, and half have already gone around it The sample is the part that matters: 202 senior AI decision-makers, board, C-suite and VP and above, at publicly traded US companies with at least a billion in revenue. In other words, the exact population that already runs SOX. Ninety eight percent have a formal AI governance policy. Forty seven percent say their organization has bypassed those governance processes to deploy something urgently.

The US added 50% tariffs on roughly 110 more Canadian product lines on 15 September Cheeses through to furniture, plywood, packaging, footwear and machinery inputs, with import bans on select alcohol and motor vehicle categories landing 29 September. Canadian counter-tariffs have been live since 8 September. If you are a Canadian brand with US inputs or US customers, your landed cost model went stale last Tuesday, and so did every price you quoted off it. Re-run the cost sheet before you commit to anything for Q4.

Drewry’s World Container Index rose 1% to 4,500 US dollars per 40 foot container on 17 September A one percent week is not the story. The story is the floor it is rising from, and what is holding it there. Which is the next item.

Red Sea capacity is coming back, and it is still nowhere near normal Capacity through the Bab el-Mandeb strait roughly doubled year over year in August and still sits at about 23% of where it was before the crisis in August 2023. Meanwhile spot rates from China to Jeddah and Khor al Fakkan have pushed past their Covid peaks. So the recovery and the record are happening at the same time, on the same lane. If your planning assumes the Suez is either open or shut, it is the wrong shape of assumption. Plan the transit time as a range, not a number, and tell your customers the range.

This week’s episode runs at the same question from the build side: “What it takes to build with AI for other people.” If this issue is the control half, that is the craft half. Watch or listen here.