I Replaced My Approve Button With Two Models. They Approved 4 of 43 Proposals in a Week.

Window: 2026-11-04 → 2026-11-10  |  Gate runs: 104  |  Proposals: 43  |  Approved: 4  |  Reviewers: Codex (adversary) + Grok (editor)

For one week, two models from different vendors made the approval calls on a Czech content site I run. They approved 4 of 43 proposals. All four had been rejected once first. The discussion round meant to settle disagreements ran 5 times and never turned a reject into an approval.

The point was to get me out of the loop. It worked, just not how I expected: the gate turned out to be a fact-checker the proposing agent couldn't get past.

The setup

The site runs a feedback queue. Automations and the owner file requests (low CTR on an article, an outdated price, a reader's correction), and a headless Claude Code session picks one up, researches it and writes a proposal. Until 2026-11-04 every proposal ended with "waiting for owner approval", and each one waited on me.

I already knew approve buttons turn into rubber stamps , and a read-only reviewer had already rejected 10 of 12 SEO changes in an earlier test. The next step was to take the human out of approval entirely and let two reviewers decide:

# Decision rules, in order
# 1. Pre-gate (regex, reviewers cannot waive): e-mail sends, product prices,
#    deleting/merging pages, slug/URL/canonical/robots -> "human"
# 2. Both approve, no must_fix, confidence >= 0.7     -> "approve" (caller applies)
# 3. Split verdict -> ONE discussion round, each sees the other's JSON
# 4. Both reject -> "reject". Errors, timeouts, still split -> "human" (fail closed)
ROLE = {
    "codex": "ADVERSARIAL reviewer: factual errors, risk, lost content, scope creep.",
    "grok":  "EDITOR: Czech language quality, search intent, reader usefulness.",
}
SCHEMA = '{"verdict":"approve|reject","confidence":0.0,"must_fix":[],"rationale":"<=400 chars"}'
# "Unsure = reject." Judge facts ONLY against the evidence packet. No tools.

Both reviewers run headless with no write access. Codex runs as exec in a read-only sandbox and Grok in plan mode. The proposal is wrapped in <untrusted> tags, and the prompt states outright that phrases like "do not publish without owner approval" are old workflow wording, because the reviewers are the approver now. Without that line, the first runs rejected proposals for asking the owner.

Week one in numbers

The two reviewers disagreed in 5 of 91 valid first-round runs, so they agreed 94.5% of the time. Different vendors, different roles, nearly identical verdicts. In 4 of the 5 splits, Codex was the one rejecting. In the discussion round, Grok switched to reject in 4 of 5 cases after reading Codex's JSON. Once the editor saw the adversary's specific objections, it gave in.

What actually got rejected

A rough keyword tally over the 617 must_fix items: about 48% concern evidence or sourcing, about 31% concern prices or billing periods. 27 runs flagged a proposal with no final text at all, only a promise to "update the section".

The pricing errors are what justify the gate. These are real catches from the logs, translated:

• Wrong ratio. The draft said a small model was "40× cheaper on output" than a flagship. The evidence packet gave $50 vs $0.50 per million tokens below 100K context and $50 vs $2.50 above it. Both reviewers did the division: 100× and 20×. Nobody computed 40.
• Monthly shown as yearly. A pricing table listed a plan at "~365 CZK per year". That was the monthly equivalent under annual billing. The yearly total was about 4,376 CZK, 12× larger.
• Tier collision. "€180 per year up front" for the entry plan was correct. A new top tier also costs €180, but per month. A meta description with three EUR amounts and no periods would have made the two indistinguishable in a snippet.
• Mixed currencies. "From $103" for a tier priced at $100 or €103. The proposer had merged the USD and EUR columns.
• Miscount. "Six tiers" in an intro to a table with seven rows.
• Unsourced negatives. "Not available on the free plan" was rejected because the evidence never said so. A claim that something is absent needs a quoted source too, like any other claim.

Every one of these would have shipped under my old approve button. I'd have looked at the title, seen "pricing 2026", and tapped yes.

What the 4 approvals had in common

Each approved proposal was a resubmission that fixed the previous round's must_fix list point by point:

• final text included in the proposal, not "will rewrite the intro"
• every factual sentence traceable to a labelled evidence block (A, B, C…)
• unverifiable claims deleted instead of softened
• no URL, slug or price-of-our-own-product changes, so the pre-gate never fired

One approval was a title and meta change where every phrase, including "install in two minutes", already appeared in the article body. Another was a changelog-driven fact fix where Grok listed eight specific flags and behaviours it had checked against the evidence. Approvals came with receipts.

The cost of a strict gate

36 rejected proposals mean the issues behind them are still live. Several were stale prices on pricing pages. A separate rule blocks rewriting the same article within 60 days of its last edit, so a proposal rejected today can't come back for those pages until early January. The gate prevented new wrong numbers and left old wrong numbers in place.

On the surface, "Unsure = reject" plus "two must agree" plus one discussion round biased toward the stricter reviewer gives roughly a 9% approval rate. In practice the rate measured proposal quality. Every must_fix I read in the logs named a real error.

What I'd change

Most of the 617 objections were cheap to prevent upstream, so the fixes belong in the proposer, not the gate. Ratios and currency conversions should be computed in code and pasted in, never written by the model. Every amount in a pricing table should carry its billing period as a required field. A proposal without final text should fail a schema check before a single reviewer token is spent. The second reviewer agreed with the first 94.5% of the time on verdicts, so its real value is the language feedback it attaches. I'd keep it for that, and stop pretending two vendors give two independent votes.