Juan Piaggio · 2026-09-23 · 7 min read · ai · governance · product-development
Two conversations about AI ran alongside each other this week, and they turned out to be the same conversation. One is about agents that no longer draft text but execute — opening pull requests, moving tickets, calling APIs on a schedule nobody is watching. The other is about knowledge decay: a run of studies, and a lot of uneasy senior engineers, converging on the observation that people are shipping work they cannot explain.
They meet at the approval step. The industry has landed, sensibly, on a pragmatic position: don't hand a model the keys, use deterministic orchestration and keep a human at the decision forks. We build that way and we think it is right. But a human at a decision fork is only a control if the human is actually deciding, and a great many approval gates are designed to produce a signature rather than understanding.
The scrutiny on AI spend right now is mostly about the visible ledger: data centre capex, power, whether inference cost is outrunning the productivity it buys. Those are real numbers and they belong in a budget review.
The second ledger has no invoice. It is the understanding your team has of the systems it is responsible for — and it is being spent down quietly, because nothing bills you for it until the night something breaks and the person on call has never read the code path. Velocity and comprehension come apart slowly enough that no quarter ever looks like the quarter it went wrong.
Nobody decides to stop understanding their own product. What happens instead is that a review step gets busy, and the cheapest way through a busy review step is to trust the thing being reviewed.
We have written before that if humans approve a curated agent action 97% of the time, that gate has stopped being a control and become a tax — promote the action and stop paying it. That is still true as a throughput argument.
Here is the second reading. A gate where the reviewer never disagrees is also a gate where nobody is learning anything. Approval rate measures friction. Disagreement rate measures attention. When the first is near-perfect and the second is near-zero, you do not have oversight; you have a queue of people acquiring the habit of clicking yes, which is a habit they will bring to the one item that mattered.
So the design question is not "how many gates" but what a gate puts in front of the person.
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 800 216" role="img">
<title>Reviewing the output decays understanding; reviewing the decision compounds it</title>
<rect x="1" y="78" width="150" height="52" rx="10" fill="var(--primary-50)" stroke="var(--primary-600)" />
<text x="76" y="109" text-anchor="middle" font-size="14" fill="var(--ink-1)">Agent proposes</text>
<path d="M151 104 H176 V34 H188" stroke="var(--text-muted)" stroke-width="1.5" fill="none" />
<path d="M188 29 l10 5 -10 5 z" fill="var(--text-muted)" />
<path d="M151 104 H176 V174 H188" stroke="var(--text-muted)" stroke-width="1.5" fill="none" />
<path d="M188 169 l10 5 -10 5 z" fill="var(--text-muted)" />
<rect x="200" y="8" width="200" height="52" rx="10" fill="var(--sand-1)" stroke="var(--border)" />
<text x="300" y="30" text-anchor="middle" font-size="11" fill="var(--text-muted)">REVIEWS THE OUTPUT</text>
<text x="300" y="48" text-anchor="middle" font-size="14" fill="var(--ink-1)">Looks right</text>
<path d="M400 34 H418" stroke="var(--text-muted)" stroke-width="1.5" fill="none" />
<path d="M418 29 l10 5 -10 5 z" fill="var(--text-muted)" />
<rect x="430" y="8" width="200" height="52" rx="10" fill="var(--sand-1)" stroke="var(--warning)" />
<text x="530" y="30" text-anchor="middle" font-size="11" fill="var(--warning)">97% approved, 0% disputed</text>
<text x="530" y="48" text-anchor="middle" font-size="14" fill="var(--ink-1)">Attention decays</text>
<path d="M630 34 H638" stroke="var(--text-muted)" stroke-width="1.5" fill="none" />
<path d="M638 29 l10 5 -10 5 z" fill="var(--text-muted)" />
<rect x="650" y="8" width="144" height="52" rx="10" fill="var(--sand-1)" stroke="var(--error)" />
<text x="722" y="31" text-anchor="middle" font-size="14" fill="var(--ink-1)">Shipped,</text>
<text x="722" y="49" text-anchor="middle" font-size="14" fill="var(--ink-1)">unexplained</text>
<rect x="200" y="148" width="200" height="52" rx="10" fill="var(--sand-1)" stroke="var(--border)" />
<text x="300" y="170" text-anchor="middle" font-size="11" fill="var(--text-muted)">REVIEWS THE DECISION</text>
<text x="300" y="188" text-anchor="middle" font-size="14" fill="var(--ink-1)">Why it chose this</text>
<path d="M400 174 H418" stroke="var(--text-muted)" stroke-width="1.5" fill="none" />
<path d="M418 169 l10 5 -10 5 z" fill="var(--text-muted)" />
<rect x="430" y="148" width="200" height="52" rx="10" fill="var(--sand-1)" stroke="var(--success)" />
<text x="530" y="170" text-anchor="middle" font-size="11" fill="var(--success)">disagreement recorded</text>
<text x="530" y="188" text-anchor="middle" font-size="14" fill="var(--ink-1)">Reviewer learns</text>
<path d="M630 174 H638" stroke="var(--text-muted)" stroke-width="1.5" fill="none" />
<path d="M638 169 l10 5 -10 5 z" fill="var(--text-muted)" />
<rect x="650" y="148" width="144" height="52" rx="10" fill="var(--primary-50)" stroke="var(--primary-600)" />
<text x="722" y="171" text-anchor="middle" font-size="14" fill="var(--ink-1)">Shipped,</text>
<text x="722" y="189" text-anchor="middle" font-size="14" fill="var(--ink-1)">understood</text>
</svg>
The instinct when governance feels thin is to add gates. It makes things worse: more items, each getting less thought, and the reviewer's reflex degrades across all of them. Attention is the scarcest budget in the loop, and the industry already learned this lesson on the compute side — context efficiency and smaller specialised models beat brute-force scale. The same logic applies to people.
So route, don't blanket. In Meshworq every proposal that needs a human carries a score:
Risk Score = agentAuthorityFactor + ageFactor + confidenceFactor
HIGH score ≥ 0.67 a human decides
MEDIUM score ≥ 0.34 a human sees it
LOW below that it flows, and the ledger records it
An agent asking to close an issue counts for more than an agent adding a label. A proposal the model itself is unsure about counts for more than a confident one. An item that has been sitting for eight hours counts for more than a fresh one. The effect is that the queue a person opens in the morning is short, and everything in it genuinely deserved them.
Worth saying plainly: we got this wrong once. One of the three factors read a field that no model wrote, so it contributed zero to every score the queue ever produced — and the band that was supposed to demand a human became unreachable for two of our three approval sources. The gate looked busy and was structurally incapable of escalating. A control you have not verified end to end is a decoration.
If the reviewer only ever sees the proposed change, the best case is that they check it. If they see why the agent chose it and what it ruled out, they are reading an argument, and reading arguments is how understanding is maintained rather than spent.
That is the real argument for an audit ledger, and it is a stronger one than compliance. Every agent action in Meshworq carries a correlation ID and stores its reasoning, so "what did it do, and why" is a query. The payoff has been practical rather than regulatory: our worst agent defect — four stacked bugs that meant no tool-using run ever reached the model — was found by reading stored production reasoning, not by running a suite that was 200 tests green. The team that goes looking at reasoning is the team that still knows how its system behaves.
Two metrics are worth putting on a dashboard next to throughput: disagreement rate per curated action, and time-to-decision on high-risk items. If disagreement goes to zero, promote the action — deliberately, with the acceptance evidence attached — instead of letting it rot into a rubber stamp. If time-to-decision collapses, your reviewers have stopped reading, and adding gates will only spread them thinner.
Generative tools genuinely accelerate delivery, and the guarded, practical mood that has replaced both the doom and the boosterism is the right one to build in. But keeping a human in the loop is a mechanism, not a virtue: it protects you only when the human is deciding something, on an item that warranted the interruption, with enough context to disagree. Design the gate for comprehension and the velocity is real. Design it for a signature and you get the speed now and the bill later, on a night when the person holding the pager is reading the code for the first time.