Home · Writing

Earning Trust

The POC Is a Production System

November 5, 2026 · First published on this site

The riskiest agent in an enterprise is usually the one nobody calls production. It is the proof of concept: built in a week, demoed to applause, and quietly connected to real systems so the demo means something. The production agent gets the scrutiny, the reviews, the paperwork. The POC gets a pass, because the plan calls it temporary and small. Attackers read reach. Plans do not come up in their reconnaissance.

Start with what the word production actually buys. When an organization declares a system production, it wraps the system in layers. The agent runs under an identity of its own, with permissions scoped to its job. Secrets come from a vault and expire quickly. Every action lands in a log somebody can read later. Changes pass evals and approvals before they ship. A named owner can revoke the whole thing in minutes. Each layer exists because something once went wrong in its absence. Production is a promise that the layers stay on.

A POC skips the layers deliberately, and for an experiment that is the right call. The job of a POC is to answer a question fast, and every gate is a delay. So the POC runs under a developer's own credentials, or a shared key somebody pasted into a config file. Nothing logs the prompts or the actions, because the team is still changing both daily. There are no eval gates, no approval workflow, no revocation drill, often no single owner. Run on synthetic data inside a sandbox, none of this matters. The blast radius of a sandbox is the sandbox.

The trouble is that POCs rarely stay in the sandbox, because a POC has to prove something and synthetic data proves very little. The demo needs real records to be convincing. So somebody grants read access to the warehouse, just for the demo. A production API key goes into the environment, just for the week. Real customer documents get uploaded so the extraction can be tested on the messy versions. Each grant is small, temporary, and reasonable. Together they hand the experiment the reach of a production system while its controls stay at the level of a hackathon. The risk was never the POC's size. The risk is production reach wearing experiment-era controls.

A proof of concept and a production agent have the same reach and different control layers Both systems reach real data, real credentials, and production systems. The production agent has five control layers in place: identity and scoped permissions, audit logging, eval and approval gates, managed expiring secrets, and revocation with a named owner. The proof of concept skips all five. REACH — REAL DATA · REAL CREDENTIALS · PRODUCTION SYSTEMS PRODUCTION AGENT PROOF OF CONCEPT Identity & scoped permissions IN PLACE Identity & scoped permissions SKIPPED Audit logging IN PLACE Audit logging SKIPPED Eval & approval gates IN PLACE Eval & approval gates SKIPPED Managed, expiring secrets IN PLACE Managed, expiring secrets SKIPPED Revocation & a named owner IN PLACE Revocation & a named owner SKIPPED
The reach is identical. Only the layers differ, and the layers are the part the POC skipped on purpose.

Binance published the clearest recent picture of what reach is worth to an attacker. Its initial report on the October theft, which affected about $6.34 billion, describes a five-month path in. In June, an engineer downloaded a tampered installer for an AI coding tool; Binance caught that one, replaced the device, and rotated credentials, though the attackers kept organizational information they had harvested. In September, a fake job interview planted malicious code on a second engineer's machine. On October 20, the attackers moved through Binance's internal AI tool to take over two more engineers' devices, and on one of them they found a token with permission to deploy containers.

From there the chain reached the wallet signing service. No private key was ever stolen. The attackers got Binance's own service to sign their transactions, outside the normal withdrawal process, and on Ethereum they used EIP-7702 delegations that stayed valid on-chain even after Binance suspended withdrawals. Among the remediation steps Binance lists: it rebuilt the internal AI tool and tightened its access controls. The report never says how that tool began life, and this article makes no claim about that. What the report documents is reach: an internal tool that could move across engineers' machines, credentials sitting inside that reach, and a signing service at the end of the chain.

Internal tools and POCs share a life cycle, whatever their starting point. Something useful gets built. Colleagues adopt it. Integrations accumulate around it. Permissions grow along with usage, quietly, in small reasonable grants. One day, real processes depend on a system that no launch review ever approved, and governance arrives only when an incident forces the rebuild. The rebuild is the tell. The controls were affordable all along; they were simply scheduled after the loss instead of before it.

Most POC exposure never involves an attacker at all. Real records get pasted into a prompt to a vendor API nobody approved, because the demo needed real data to be believed. The data then lives in a vendor's logs, a support thread, a retained trace somewhere outside the building. Nobody files an incident, because a successful demo does not feel like one. The parked version of this argument, which I set aside in October, put it this way: a POC on real data without approval is an incident, not an experiment. The broader point stands even when every person involved acted in good faith. Exposure does not require malice. It requires reach and no record.

The rule that follows is to govern by reach. A POC that can reach production systems is a production system, whatever the project plan calls it, and it has to be governed like one from the first day it touches anything real. Governed, for a POC, can stay cheap. It means a named owner. Credentials scoped to the experiment and set to expire on a date. Logging on from the first run, however rough. Approval recorded before real data goes in. A kill date on the calendar, kept. Five habits, none of them expensive, all of them cheaper than a rebuild.

Three questions sort any agent you are shown, POC or otherwise. Whose data is in the prompt. Who approved it leaving the building. Where is the record. A system that answers all three has earned its experiment label. A system that cannot answer them is already production in every way that matters to an attacker, and it is running with the controls of a hackathon. The label on the project plan is the one control an attacker will never check.

This article grows out of my November 4 LinkedIn post on Binance's incident report, where the same rule closed the argument. It pairs with Build the Ground, Not the Harness, which makes the case that the control layers are the durable part of any agent platform. Browse the full archive on the home page.

← All writing