Nix Knows When
the Agent Is Wrong

Jason Odoom

adastracomputing.com

$ whoami
Jason Odoom

$ cat /etc/motd
Ad Astra Computing. One rule:
when software acts on your behalf,
it leaves a record you can inspect.

I built a tool to help maintainers patch nixpkgs CVEs faster.

I optimized the wrong thing.

Frontier models can increasingly do the work.

The work was never the hard part.

Remediation is evidence work

• which package is actually affected
• which upstream commit fixes it
• what the new hash is
• does it still build
• do the tests still pass
• why should a maintainer believe any of it

Patches are cheap.

Proof is not.

Nix is the verifier

An ordinary project tells an agent very little: it compiled, the tests passed. Nix tells it:

• pinned inputs
• reproducible builds
• content-addressed sources
• passthru.tests
• a sandbox that can rebuild the vulnerable past

How Nix knows the fix works

Write a small test that triggers the bug. Run it against the package built without the fix, then with it. Everything else stays identical.

Both builds hermetic. Both in CI. Both signed. The model never votes.

The test that proved nothing

Stack overflow bugs only crash if the stack limit is the normal 8 megabytes.

Inside the build sandbox that limit was unlimited, inherited from the Nix daemon. So those tests never crashed, and quietly proved nothing.

We found it because a run failed for the wrong reason. The fix was one line.

Trace

An agent-owned nixpkgs fork that produces signed, reproducible CVE evidence bundles instead of drive-by pull requests.

Scout → Upstream fix check → Patch writer → Reviewer → Builder → Bug test → Steward

blue: ordinary code, does the work that must be trusted
gold: a model, and the two must come from different families

The routing rules are code

the default answer is reject

merge into our fork  ·  send to a human
publish as advice only  ·  reject

Merging automatically means into our own fork. Nothing is ever pushed to nixpkgs.

What a signature actually proves

Every approved bundle: Sigstore keyless OIDC, anchored in the Rekor transparency log, locked workflow identity, verifiable with cosign verify-blob.

A signature tells you who produced this and what they claimed.
It tells you nothing about whether they were right.

One bundle, end to end

finding.json
upstream-ref.json
patch.diff
patch-rationale.md
review.md

reproducer.nix
reproducer-log.txt
replay.json
bundle.json
bundle.sig

What Nix caught.
And what it could not.

The agent wrote a fix upstream did not write

Upstream fixed the bug one way. The agent fixed it another way, with its own bounds check.

Compared against the upstream fix, that is a mismatch. Any text comparison marks it wrong.

But the test crashes without it and passes with it. The fix works.

The agent that refused

The upstream fix is made of 34 separate edits. 14 of them do not apply to the version nixpkgs has.

The agent refused, and named the missing pieces, instead of writing something that would apply but leave the bug in place.

An honest refusal is stronger evidence than a low score.

The reviewer that lied while being right

Correct verdict: reject.

But its written explanation described a safety check that is not in the code.

We caught it with a check that goes and looks for the quoted code. If the evidence is not in the file, the review fails.

The reviewer that would have rejected the human fix

The frontier reviewer scored a correct fix zero, judging it wrong.

It applied a rule about C to a Lisp program. That same rule would reject the human fix too.

The claims we retired

• our small fine-tuned model beats the frontier model, 100% vs 90.9%
  that was 11 examples, and one of them only differed by output formatting
  re-run on 34 fresh examples: ours 0.794, and it waved through 7 bad patches. Frontier 0.971
• training it on its own mistakes so it would write the fix correctly · measured strictly harmful
• preserved: the first buggy run, in _first-run-archive/
  deleting it would erase the evidence the pipeline was wrong before it was right

Most of the work is deciding not to act

$8.20 spent on bundles ready to merge. $19.38 spent documenting the ones it turned down.

Where it actually stands

Our fine-tuned model does not match the frontier models. It still cannot write the fix exactly right on its own. What works today is asking it 16 times and letting the build choose the one that passes.

What maintainers should demand

• provenance you can verify
• a build you can reproduce
• a test that fails without the fix
• a signed record of who claimed what
• a policy that routes, with refusal as a first-class outcome

The workflows we built for human-only contribution will not survive contact with agents unchanged.

The goal is not to convince nixpkgs to trust an agent.

The goal is to define what an agent would have to prove before it deserves maintainer attention.

Trace · Apache-2.0 · github.com/Ad-Astra-Computing

adastracomputing.com · [email protected] · @odoom:matrix.org

Same bet, different layer: Vega · a Nix binary cache that only promotes builds cross-checked by independent rebuilders · vega-cache.dev