The AI safety conversation has a center of gravity, and it sits inside the training run. Alignment, in the dominant framing, is something you do to a model: shape its objectives, tune its refusals, interpret its circuits, red-team its behavior - all before it ships. This work matters enormously. It is also answering only half the question, and arguably not the half that determines what happens to the economy.
Because here is what actually occurs after the training run: the model is wrapped in a program. The program decides what the model sees, which tools it can reach, what its system prompt asserts, when it acts and on whose authority, what it is permitted to spend. That program - the agent scaffold - is where a model becomes an economic actor. And the scaffold is written, deployed, and modified not by the lab that aligned the model, but by whoever operates the agent.
This produces the gap that we think defines the next decade of AI risk, and it is not the gap the alignment literature studies. Call it the deployment gap: the distance between what an agent’s operator says the agent does and what the agent’s scaffold actually does. Every assurance produced during training - every careful preference optimization, every safety evaluation - passes through this gap on its way to the real world, and the gap is, today, entirely unverified.
An aligned model inside a misaligned scaffold is a misaligned agent. No amount of training-time work changes this. The operator can swap the model (the identity problem from Proof of Inference). The operator can edit the system prompt that promised caution. The operator can wire in tools the disclosed policy never mentioned, or skim a basis point off every trade the agent routes. From the outside - to the counterparty, the customer, the regulator - the agent’s stated policy is just a string on a website. It is, in the precise sense we defined in Verifiable Reflexivity, an unverified self-referential claim: a system describing its own behavior, with nothing carrying the claim but the operator’s reputation.
We have seen this movie. It was called FTX, and the string on the website said “client funds are safe.”
The deployment gap cannot be closed by training, by regulation-as-paperwork, or by audits-in-arrears, because all of these verify intentions and snapshots rather than execution. It can be closed in only one way: the agent must carry its policy - cryptographically - and prove its execution against it.
A policy-carrying agent is one whose operation has the following shape. Its policy - the scaffold code, the model identity commitment, the tool registry, the constraint set - is hashed into a public commitment, the way a smart contract’s code is committed on-chain. Every action the agent takes is executed inside a proving system, producing a certificate that the action was generated by that policy operating on that context. The certificate travels with the action. The counterparty does not read the operator’s documentation; the counterparty checks the proof.
Notice what this does and does not claim. It does not prove the policy is good - a policy can be faithfully executed and still be foolish, just as a smart contract can be exploit-free and still be a bad deal. What it proves is conformance: the elimination of the gap between stated and actual behavior. This is exactly the guarantee smart contracts brought to financial agreements, extended to agents whose logic includes a model. The code being law was never the interesting part; the interesting part was that the code you could read was provably the code that ran.
The concept borrows deliberately from proof-carrying code, a thirty-year-old idea from programming language theory: untrusted code arrives accompanied by a proof of its own safety properties, and the receiver checks the proof rather than trusting the sender. The agent economy is the proof-carrying-code problem at civilizational scale - millions of mutually untrusting programs, now with cognition inside them, asking to act on each other’s behalf.
Three consequences follow, each of which cuts against current assumptions.
Verification is the part of safety that composes. Training-time alignment is a property of a model; it does not survive contact with an arbitrary scaffold. Conformance proofs are properties of deployments, and they compose the way proofs compose: an agent that consumes another agent’s services can require - and recursively verify - the sub-agent’s conformance certificates, building chains of accountability through arbitrary delegation depth. Alignment does not compose across an agent supply chain. Proofs do. (Piece 06 develops the machinery: folding an agent’s entire session history into a constant-size certificate.)
The trust-relevant computation is provable today. The standard objection - you cannot prove a frontier model’s forward pass in real time - is true and, for this problem, beside the point. The deployment gap lives in the scaffold: ordinary code, modest in size, exactly what a general-purpose zkVM proves efficiently now. The hybrid stack from Proof of Inference applies - attest the tensor work, prove the policy layer. The layer where operators lie is the layer we can already make lying impossible.
Accountability becomes a market structure, not a legal afterthought. You cannot depose an agent. You cannot subpoena its intentions, and by the time discovery concludes, the agent has executed ten million more actions. Legal accountability operates at human latency; agents operate at machine latency. Conformance proofs are accountability at the speed of the actions themselves - each action arriving pre-audited, the flight recorder written before the flight. Insurers can price conformance. Counterparties can require it in the handshake. Regulators can accept it as compliance evidence. None of this needs to wait for case law; it needs a proof format.
We are engineers, so let us state plainly what policy conformance does not solve. It does not solve deceptive alignment inside the weights - a model gaming its own training. It does not make a committed policy wise, only binding. It does not, by itself, verify the inputs the agent acted on (Piece 05 takes up that boundary). The model interior remains, for now, attested rather than proven, and we will not pretend otherwise.
But observe where this leaves the field. The alignment community is working the hardest problem - the interior of the model - on decade timescales. The deployment gap, meanwhile, is open today, will be exploited today, and is closable today with proving systems that already exist. Safety discourse has been staring at the model and ignoring the program wrapped around it, which is rather like securing the engine while leaving the cockpit unlocked.
Train the model to be good. Then make its deployment incapable of lying about itself. The first is alignment. The second is verification. An agent economy needs both - but only one of them can be checked by the counterparty on the other side of the trade, at the moment of the trade, with no one’s word taken for anything.
That one is ours to build.
ZKM builds Ziren, a general-purpose zkVM and proving stack. zkm.io
The AI safety conversation has a center of gravity, and it sits inside the training run. Alignment, in the dominant framing, is something you do to a model: shape its objectives, tune its refusals, interpret its circuits, red-team its behavior - all before it ships. This work matters enormously. It is also answering only half the question, and arguably not the half that determines what happens to the economy.
Because here is what actually occurs after the training run: the model is wrapped in a program. The program decides what the model sees, which tools it can reach, what its system prompt asserts, when it acts and on whose authority, what it is permitted to spend. That program - the agent scaffold - is where a model becomes an economic actor. And the scaffold is written, deployed, and modified not by the lab that aligned the model, but by whoever operates the agent.
This produces the gap that we think defines the next decade of AI risk, and it is not the gap the alignment literature studies. Call it the deployment gap: the distance between what an agent’s operator says the agent does and what the agent’s scaffold actually does. Every assurance produced during training - every careful preference optimization, every safety evaluation - passes through this gap on its way to the real world, and the gap is, today, entirely unverified.
An aligned model inside a misaligned scaffold is a misaligned agent. No amount of training-time work changes this. The operator can swap the model (the identity problem from Proof of Inference). The operator can edit the system prompt that promised caution. The operator can wire in tools the disclosed policy never mentioned, or skim a basis point off every trade the agent routes. From the outside - to the counterparty, the customer, the regulator - the agent’s stated policy is just a string on a website. It is, in the precise sense we defined in Verifiable Reflexivity, an unverified self-referential claim: a system describing its own behavior, with nothing carrying the claim but the operator’s reputation.
We have seen this movie. It was called FTX, and the string on the website said “client funds are safe.”
The deployment gap cannot be closed by training, by regulation-as-paperwork, or by audits-in-arrears, because all of these verify intentions and snapshots rather than execution. It can be closed in only one way: the agent must carry its policy - cryptographically - and prove its execution against it.
A policy-carrying agent is one whose operation has the following shape. Its policy - the scaffold code, the model identity commitment, the tool registry, the constraint set - is hashed into a public commitment, the way a smart contract’s code is committed on-chain. Every action the agent takes is executed inside a proving system, producing a certificate that the action was generated by that policy operating on that context. The certificate travels with the action. The counterparty does not read the operator’s documentation; the counterparty checks the proof.
Notice what this does and does not claim. It does not prove the policy is good - a policy can be faithfully executed and still be foolish, just as a smart contract can be exploit-free and still be a bad deal. What it proves is conformance: the elimination of the gap between stated and actual behavior. This is exactly the guarantee smart contracts brought to financial agreements, extended to agents whose logic includes a model. The code being law was never the interesting part; the interesting part was that the code you could read was provably the code that ran.
The concept borrows deliberately from proof-carrying code, a thirty-year-old idea from programming language theory: untrusted code arrives accompanied by a proof of its own safety properties, and the receiver checks the proof rather than trusting the sender. The agent economy is the proof-carrying-code problem at civilizational scale - millions of mutually untrusting programs, now with cognition inside them, asking to act on each other’s behalf.
Three consequences follow, each of which cuts against current assumptions.
Verification is the part of safety that composes. Training-time alignment is a property of a model; it does not survive contact with an arbitrary scaffold. Conformance proofs are properties of deployments, and they compose the way proofs compose: an agent that consumes another agent’s services can require - and recursively verify - the sub-agent’s conformance certificates, building chains of accountability through arbitrary delegation depth. Alignment does not compose across an agent supply chain. Proofs do. (Piece 06 develops the machinery: folding an agent’s entire session history into a constant-size certificate.)
The trust-relevant computation is provable today. The standard objection - you cannot prove a frontier model’s forward pass in real time - is true and, for this problem, beside the point. The deployment gap lives in the scaffold: ordinary code, modest in size, exactly what a general-purpose zkVM proves efficiently now. The hybrid stack from Proof of Inference applies - attest the tensor work, prove the policy layer. The layer where operators lie is the layer we can already make lying impossible.
Accountability becomes a market structure, not a legal afterthought. You cannot depose an agent. You cannot subpoena its intentions, and by the time discovery concludes, the agent has executed ten million more actions. Legal accountability operates at human latency; agents operate at machine latency. Conformance proofs are accountability at the speed of the actions themselves - each action arriving pre-audited, the flight recorder written before the flight. Insurers can price conformance. Counterparties can require it in the handshake. Regulators can accept it as compliance evidence. None of this needs to wait for case law; it needs a proof format.
We are engineers, so let us state plainly what policy conformance does not solve. It does not solve deceptive alignment inside the weights - a model gaming its own training. It does not make a committed policy wise, only binding. It does not, by itself, verify the inputs the agent acted on (Piece 05 takes up that boundary). The model interior remains, for now, attested rather than proven, and we will not pretend otherwise.
But observe where this leaves the field. The alignment community is working the hardest problem - the interior of the model - on decade timescales. The deployment gap, meanwhile, is open today, will be exploited today, and is closable today with proving systems that already exist. Safety discourse has been staring at the model and ignoring the program wrapped around it, which is rather like securing the engine while leaving the cockpit unlocked.
Train the model to be good. Then make its deployment incapable of lying about itself. The first is alignment. The second is verification. An agent economy needs both - but only one of them can be checked by the counterparty on the other side of the trade, at the moment of the trade, with no one’s word taken for anything.
That one is ours to build.
ZKM builds Ziren, a general-purpose zkVM and proving stack. zkm.io
