nodrail

All guides

How to stop an AI coding agent from opening a pull request before QA and review

Preview guide. We’re verifying these steps against the public installation.

By Nodrail · 10 min read

On this page
  1. The problem: the order of work, not one tool call
  2. Why per-call rules can't express it
  3. Runs, steps and receipts
  4. The gate
  5. What "since" means
  6. When history is missing
  7. What happens when the gate is unmet
  8. Check the gate before you rely on it
  9. Limits
  10. A ready-made module for GitHub calls
  11. What if the pull request is already open?
  12. Next steps

To stop an AI coding agent from opening a pull request before QA and code review passed, write a gate: a policy entry that holds the pull request action until the run contains a passing QA step and an approving review step, both finished after the last code change. With Nodrail, the gate lives in your rules file next to your other rules. If the steps are missing, the action is denied or sent to a person. Gate git push as well, or the main agent can push around it.

This guide uses Claude Code and three example agents: dev writes code, qa tests it, reviewer reviews it. A fourth agent, github, opens the pull request. The names are yours to change.

The agent names, rules and numbers in this guide are illustrative. Adapt them to your own agents and test them before relying on them.

The problem: the order of work, not one tool call

A coding agent that can dispatch subagents can also skip them. It writes a fix, decides the fix is small, and dispatches github to open the pull request. Or QA passes, dev makes one more change, and the pull request goes out on code nobody tested.

What you want is a rule like this:

Do not open the pull request until QA passed and review approved, after the last code change.

That rule is about history. It depends on what happened earlier in the same piece of work and in what order.

Why per-call rules can't express it

A normal rule looks at one action: this tool, this operation, this resource, these parameters. It can say "dispatching github needs approval" or "git push is denied". It cannot say "allowed, if qa passed after dev last worked", because the answer is not in the call. It is in the record of the run.

You could require a person to approve every pull request. That works, and it is tiring. People start approving without looking. The gate keeps the person for the case that needs one: the required steps did not happen.

Runs, steps and receipts

Nodrail records work against a run: one piece of work, with a key the integration picks. In the Claude Code integration the run key is the NODRAIL_RUN environment variable when it is set. Otherwise it is the repository and branch the session started on, such as github.com/acme/app/fix-42, kept for the whole session so switching branch doesn't switch runs. On the default branch, or with no branch, it is the Claude Code session.

Inside a run, each subagent's work is a step, from start to finish. The integration sees the dispatch, so it knows which agent the step belongs to. When the agent finishes, its last message can end with a receipt: a short block with its verdict. Ask for it in each agent's instructions.

```nodrail-receipt
verdict: pass
findings: [correctness]
```

Each tool call the integration decides inside the run is recorded as an action. Together these give the engine what it needs: who worked, in what order, and what each finished agent reported.

For more on receipts, see Agent receipts and run history (preview).

The gate

Here is a rules file that covers only dispatching agents and pushing. It allows both, then adds two gates on top. On its own it is too narrow for real work: the local hook blocks any call no rule covers, such as a file edit. In your project, add the list and the two gates to your existing rules, and keep the allow rules you already have. Rule names must be unique across the files you load, so drop any rule here that you already define, such as the repo-work starter pack's Never force push.

- required_engine_version: 3

- list: builders
  items: [dev, infra]

- rule: Dispatch agents
  tool: claude-code
  operation: agent
  resource: { kind: agent, match: any }
  effect: allow

- rule: Push branches
  command: { prefix: git push }
  effect: allow

- rule: Never force push
  command: { prefix: git push --force }
  effect: deny

- gate: Review before the pull request
  tool: claude-code
  operation: agent
  resource: { kind: agent, match: exact, value: github }
  requires:
    - { agent: qa, verdict: pass, since: { list: builders } }
    - { agent: reviewer, verdict: [ship, ship_with_debt], since: { list: builders } }
  otherwise: require_approval

- gate: Review before a push
  command: { prefix: git push }
  requires:
    - { agent: reviewer, verdict: [ship, ship_with_debt], since: { list: builders } }
  otherwise: deny

How to read it:

  • In Claude Code, starting a subagent is tool: claude-code, operation agent. The resource is the subagent type (kind agent), so value: github matches dispatching github.
  • requires lists the steps that must exist. Each names an agent, one or more accepted verdicts, and since.
  • otherwise is what happens when a requirement is unmet: require_approval or deny.
  • A gate matches like a rule, with the same fields. A command gate uses the same word-boundary prefix and compound-command handling as a command rule, so git status && git push is still a push.
  • Files with gates need required_engine_version: 3.

A gate never allows anything. A satisfied gate adds nothing, and the action still needs an allow rule. That is why the file has the two allow rules at the top. An unsatisfied gate adds its otherwise effect, and the usual precedence decides: deny, then review, then allow. The force-push rule stays a deny whatever the gate says.

What "since" means

since: { list: builders } means: count only a step that started and finished after the latest work of any agent in the builders list. "Latest work" is the latest event of that agent in the run: a step start, an action, or a step finish. A builder that is still running counts as the latest work, and a QA step that was already running when the last change happened doesn't count: it may not have seen it.

Builders aren't the only ones who can change code. With since, a change by any other agent also counts as work: the main agent editing a file, a general-purpose subagent committing, any shell command other than looking around (ls, git status, git log, git diff). Agents the gate waits for are checking the work, so only their file edits count: a reviewer running the tests undoes nothing, a reviewer editing a file undoes QA's pass.

Take this run:

OrderEventIs "qa passed since builders" met?
1dev finishes a changeNo
2qa finishes with verdict: failNo, wrong verdict
3dev makes a fix and finishesNo
4qa finishes with verdict: passYes
5The main agent edits one more fileNo, the pass is now older than the last change

Row 5 is the case the gate exists for. Without since, any matching step at any time in the run counts.

Verdicts compare in lowercase, so Pass and pass are the same.

When history is missing

The gate fails closed. These all leave a requirement unmet:

  • no run key, or a run with no recorded history;
  • a step that ended without a receipt, or with a receipt that has no verdict;
  • a builder that is still running;
  • a QA or review step that started before the last change.

Unmet means the otherwise effect applies. A missing record never counts as a pass.

What happens when the gate is unmet

Locally, with the open-source Claude Code hook, otherwise: require_approval makes Claude Code ask the person at the keyboard. The prompt says which steps are missing. You can let the action through or stop it. otherwise: deny blocks it, with the same explanation for the agent.

In Nodrail Cloud, an unmet gate with otherwise: require_approval creates a request in Approvals, which shows the gate that holds it. With otherwise: deny, the action is denied. The person who decides is recorded with the decision, and Insights counts how often each gate held work, what was usually missing and how often a person allowed a held action anyway.

There is no text escape hatch. An agent cannot write "skip QA, docs only" in its own brief and get past the gate. If a change really needs no QA, a person approves that one action, and that approval is on record.

Check the gate before you rely on it

Try the gate in a scratch repository first:

  1. Ask the main agent to open a pull request straight after a dev step. The dispatch of github should stop and ask.
  2. Run qa and reviewer with passing receipts, then dispatch github. It should go through.
  3. Make one more dev change and dispatch github again. It should stop again.
  4. Run git push from the main agent before review. It should be denied.

The local CLI shows what the run recorded:

nodrail-policy runs show github.com/acme/app/fix-42

Limits

  • Receipts are self-reported. A receipt is what the agent said. An agent told to say pass will say pass. The gate does not prove QA was good. It proves a qa step ran after the last change and reported a verdict, with time, step and run recorded. Nodrail does not check receipts against CI results.
  • Step ownership comes from the dispatch. The integration knows which agent a step belongs to because it saw the dispatch. An agent field written in a receipt is ignored.
  • Gate the dispatch and the push. Gating only the github dispatch leaves git push from the main agent open. The example gates both. If your agents can open pull requests another way, such as a gh command or an MCP tool, gate those too.
  • The run key is declared by the integration. Like agent names, it is reported, not authenticated. Locally the history is a file in your repository's git directory. Anyone who can edit that file can edit the history: deny agent writes under .git, as the repo-work starter pack does, but a test the agent writes and runs can still write to it. Local gates keep an honest team of agents in order; they don't stop an agent set on getting around them. Nodrail Cloud keeps the history on its server, out of reach of code the agent runs.
  • Keep the rules out of the agent's reach. Put the rules file under .claude/nodrail/. The hook refuses file edits to the rules it loaded, but an agent allowed to run arbitrary commands could still change them.
  • The hook is the boundary. As with any rule, an agent or process that can change hook settings or reach GitHub with its own credentials can bypass it. See the APF boundary notes.

A ready-made module for GitHub calls

The gate above holds the Claude Code dispatch. If your agents' GitHub calls go through an adapter that describes them as tool github, such as pr.open on the repository owner/name, the catalog module @nodrail/open-reviewed-pr packages the same idea. A publisher agent, github, may open draft pull requests after QA, a gatekeeper and a reviewer accept the current candidate, and may not start those checkers itself. Merging stays denied.

nodrail add @nodrail/open-reviewed-pr

Give it your repository, review the permission changes and lock them, as described in Playbooks (preview). It takes effect only where that adapter supplies the candidate and its evidence. The local Claude Code hook doesn't map GitHub calls, so keep the dispatch and push gates above for Claude Code.

What if the pull request is already open?

The run gate above covers the agent action that opens a PR or pushes a change. Once a PR exists, GitHub decides whether it can merge under the target branch’s rules. In our GitHub merge check pilot, we required a status for the current head, then tested whether a passing QA receipt from the previous commit could satisfy it. GitHub held the merge until fresh QA passed. Read the GitHub merge check pilot.

Next steps

Use APF to hold a covered action until the required steps report the expected verdict after the latest change. Give the integration a run history and test both a satisfied gate and missing evidence. Read the gate reference (preview) or set up the local Claude Code hook.