TL;DR
- The argument you cannot settle is not the one on your desk. Whether frontier AI is an existential risk is unresolvable from outside the building. Whether your own deployment can be traced, stopped, and checked by someone who does not report to you is answerable this week.
- An assistant that can read messages and files and take actions across apps landed on your employees’ phones this week. It arrived with iOS 27, nobody in IT approved it, and whether a given employee has it depends on which country and which device they are on.
- This week, pick your largest AI deployment and try to answer three questions in writing: what records did it act on, who can stop it and how fast, and who outside the owning team has checked it.
We build agents for a living, and Monday morning over coffee with my husband we argued about whether the industry had just had an honest week or a very well-managed one. He landed in one place. I landed in another. Neither of us could prove it, which turned out to be the actual point.
I have spent years inside operations at companies where you are not allowed to say a thing is safe, you are required to show the test that proves it. Aerospace and industrial manufacturing do not run on conviction. They run on evidence somebody outside your reporting line signed.
That is the lens I want to hand you this week, because the AI conversation has become very loud and almost none of it is decision-grade for an enterprise.
What actually happened
Three things, in eight days.
On 8 September, Jacob Coxon, a pretraining researcher who had worked at both OpenAI and Anthropic, resigned and posted that the labs “are racing straight to self-improving superintelligence and gambling with our lives.” The post reportedly passed 100 million views. Axios reported he walked roughly two months before his equity vested and gave it up.
On 11 September, the All-In podcast ran the counter-read. David Sacks argued the amplification pattern looked orchestrated, pointing out that several well-funded advocacy organisations pushed nearly identical messaging within about fifteen minutes of each other.
On 12 September, Anthropic’s CEO Dario Amodei published “We Must Pace the Frontier,” arguing the industry should buy itself an extra year or two before certain capability levels, and committing Anthropic unilaterally to embedding third-party evaluators on site, with employee-level access and the right to publish findings the company cannot edit. Sam Altman and Elon Musk both said they agreed, within a day.
So: a resignation with a real personal cost, a credible accusation of astroturfing, and a CEO proposal that got instant agreement from his fiercest competitors. If that combination leaves you unsure what to think, you are reading it correctly.
The distinction that makes this usable
The part that took me the weekend to see clearly was really that this is not a disagreement about how capable the models are. Both sides of it accept the trajectory. It is a disagreement about who is allowed to check, and neither side can currently settle it, because the evidence that would settle it sits inside three or four private companies.
Coxon’s warning is a judgement formed from the inside. That is not dishonest, and giving up unvested equity is a real cost. But a judgement is not a dataset, and he did not leave with one. Amodei’s commitment is the most substantive answer anyone has offered to exactly that gap, and the clause that matters is not the badges or the desks. It is that the evaluators can publish findings the company does not like. That is the difference between an audit and a press release.
And it has not happened yet. So both positions are, today, unverifiable from where you and I sit.
What it looks like from where we build
Let me be plainly unimpressive about the thing we actually do.
We build agents and put them to work inside real operations. At the level available to those of us who are not sitting on frontier training runs, the systems are very good at recognizing patterns they have seen before and genuinely bad at inventing a way of working nobody has tried. I have not once watched one find a new method. I have watched them apply an old method faster than a person could, which is valuable and is not the same thing.
Getting an agent to self-direct without errors is hard, slow, unglamorous work. And the mistakes ours make are not civilizational. They are a duplicated vendor record, a purchase order raised against the wrong contract, a confident summary of a number that was stale before anyone touched it.
That is not an argument that the frontier is safe. Our floor is not their ceiling, and I would not pretend otherwise. It is an argument that the operational risk on your desk this quarter is mundane, specific, and entirely addressable, and that it deserves your attention more than a debate you cannot adjudicate.
The pattern, and where it does not apply
In the All in Podcast episode they spoke about other major historical events with this same pattern… so I went looking for other moments when the loudest prediction was catastrophe, because I wanted to know how those actually resolved.
Early railways killed people. The answer was not to stop rail travel, it was signalling and brakes. During the War of the Currents, Edison branded alternating current the “executioner’s current” to frighten the public. AC carried real high-voltage risk, and engineering and oversight turned it into the grid. Y2K is the sharpest case: the disruption largely did not arrive, and the GAO’s conclusion was that this was because government and industry took it seriously and fixed the systems, not because the fear was silly. Predicted catastrophe at the Large Hadron Collider did not materialize. Decades on, the National Academies finds no validated evidence that GMO foods carry special health risks, while still calling for continued study.
The honest through-line is not “fear was stupid.” Several of those risks were real. It is that the loudest catastrophe prediction usually outran the available evidence, while testing, engineering, rules and adaptation made the technology governable.
Which is why I am not against regulating this. I want it regulated the way we regulate an aircraft engine or a pressure vessel: independent certification, published test results, and a named authority that can ground the thing. Not a press cycle. If the risk is as high as the warning says, then show the data, publish the tests, and let the people who have to make decisions see them.
The question worth asking about any warning
When someone tells you a technology is dangerous, ask what the claim costs the person making it. Not to be cynical, but because cost is the cheapest available proxy for sincerity when you cannot see the evidence.
Coxon’s claim cost him unvested equity. That is real, and it deserves to be said plainly by people who disagree with him. Amodei’s claim costs Anthropic something only if the embedded evaluators genuinely publish unfavourable findings, which is a promise with a testable future. And it is fair to notice, as several people have this week, that a company which has raised extraordinary sums has something to gain from a narrative in which the technology was simply too powerful to be commercialized on schedule. I am not asserting that is the motive. I am saying an executive is allowed to hold both readings until one of them produces evidence.
That is not cynicism. It is the same posture you already take with a supplier who tells you their lead time is fourteen days.
Three questions, and none of them require a view on doom
Take your largest AI deployment in production, or the one closest to going live. Answer these in writing. Blank answers are the finding.
1. Can you trace it? Name the records the system acted on and reproduce one real decision end to end. If you cannot point to the rows, you do not have an AI problem. You have a data problem wearing an AI costume, and it was there before the model arrived.
2. Can you stop it? Name the person with the authority to halt it, the mechanism they would use, and the elapsed time from decision to stopped. “We would call the vendor” is not a mechanism. This is the single control that turns most AI risk from existential to expensive.
3. Can someone outside the owning team check it? The team that built it cannot be the team that certifies it. This is the enterprise-scale version of exactly what Amodei just committed to, and you can implement it this month without waiting for a treaty. Internal audit, a second business unit, or a contracted reviewer. What matters is that they can report a finding the owner would rather they did not.
Then write those three answers into your next vendor agreement as requirements, not aspirations. A vendor who cannot name the records their system acted on, or give you a kill switch with a stated response time, has told you something specific. That is more than this week’s discourse gave you.
And then run them on the deployment you did not choose.
Siri AI shipped with iOS 27 on Monday September 14th. It reads messages, email and photos, it knows what is on screen, and it can take actions across apps. On any staff phone that updated, that is an agent with reach into corporate mail that nobody in your organisation approved. Question one has no answer: there is no log you can pull. Question two has a blunt one: a single switch that turns the whole thing off on company-owned phones, and nothing at all on personal ones. Question three has no owner. You can disable it. You cannot scope it. That gap between off and governed is where most enterprise AI risk is going to live for the next two years.
The AI you have to govern is rarely the AI you deployed.
What not to do: do not pause a working deployment over this. Nothing published in the last eight days changes the risk profile of an invoice-matching agent or a demand-planning assistant. Reacting to frontier-lab news by freezing mid-market automation is the expensive version of panic.
You cannot verify a warning, and you cannot verify a promise. You can verify a process, which is why that is the only part of this you should be spending Tuesday on.
Know someone who needs this? Forward it to the executive about to make this decision. Especially the one being asked to approve an AI deployment this quarter by people who are quoting this week’s headlines at them.
This is the work I train, on real operations, not case studies. FDO School is an 8-week live cohort where you go inside an actual company, fix a real problem, ship a working AI system, and leave with the build, the case study, and the title: Forward Deployed Operator.
OPS INTEL
Siri AI shipped with iOS 27 on 14 September, and it can act inside your employees’ apps ↑ The new assistant has onscreen awareness, personal context across messages, email and photos, and the ability to take actions across apps. That is an agent with reach into corporate mail and files, arriving by default on staff devices, approved by nobody in your organisation. Two things to do this week: check what your mobile device management policy actually allows, and decide it deliberately rather than by update.
The EU AI Act deadline most teams are preparing for is the wrong one 👀 In plain terms: the rules about telling people they are dealing with an AI are already in force, since 2 August 2026, and regulators can enforce them now. The much heavier rules for high-risk systems, the ones people are actually budgeting for, were pushed back to December 2027 and August 2028. So teams are racing to prepare for obligations that are years away while quietly missing the ones that already apply.
CBP wants industry input on a supply chain visibility push ↑ The 2 September rulemaking proposal floats collecting foreign export documentation and using supply chain tracing technology to catch illegal imports. Read it as the same question as this week’s AI argument, pointed at your freight: can you show, on demand, where a thing came from. Comment while the window is open, because compliance built around a trace you cannot produce is a cost you will carry for years.
All four major US parcel carriers have set holiday surcharges, starting this month ↑ Peak season fees start as early as September this year and run higher than 2025. If your Q4 margin model still uses last year’s surcharge assumptions, it is already wrong, and the fix is a two-hour update rather than a project.
Anthropic commits to embedded third-party evaluators 👀 The essay is worth reading yourself rather than through coverage of it. The operative clause for an operator is not the pacing argument, it is that the evaluators may publish findings the company cannot edit. If that holds, it becomes the first externally checkable claim any frontier lab has made, and a reasonable template for what you demand of your own vendors.
If you are being asked to approve an AI deployment this quarter and you cannot answer the three questions above, reply to this email. I read every one.