How to stop an AI coding agent from opening a pull request before QA and review
Preview guide. We’re verifying these steps against the public installation.
By Nodrail · 10 min read
On this page
- The problem: the order of work, not one tool call
- Why per-call rules can't express it
- Runs, steps and receipts
- The gate
- What "since" means
- When history is missing
- What happens when the gate is unmet
- Check the gate before you rely on it
- Limits
- A ready-made module for GitHub calls
- What if the pull request is already open?
- Next steps
To stop an AI coding agent from opening a pull request before QA and code review passed, write a gate: a policy entry that holds the pull request action until the run contains a passing QA step and an approving review step, both finished after the last code change. With Nodrail, the gate lives in your rules file next to your other rules. If the steps are missing, the action is denied or sent to a person. Gate git push as well, or the main agent can push around it.
This guide uses Claude Code and three example agents: dev writes code, qa tests it, reviewer reviews it. A fourth agent, github, opens the pull request. The names are yours to change.
The agent names, rules and numbers in this guide are illustrative. Adapt them to your own agents and test them before relying on them.
The problem: the order of work, not one tool call
A coding agent that can dispatch subagents can also skip them. It writes a fix, decides the fix is small, and dispatches github to open the pull request. Or QA passes, dev makes one more change, and the pull request goes out on code nobody tested.
What you want is a rule like this:
Do not open the pull request until QA passed and review approved, after the last code change.
That rule is about history. It depends on what happened earlier in the same piece of work and in what order.
Why per-call rules can't express it
A normal rule looks at one action: this tool, this operation, this resource, these parameters. It can say "dispatching github needs approval" or "git push is denied". It cannot say "allowed, if qa passed after dev last worked", because the answer is not in the call. It is in the record of the run.
You could require a person to approve every pull request. That works, and it is tiring. People start approving without looking. The gate keeps the person for the case that needs one: the required steps did not happen.
Runs, steps and receipts
Nodrail records work against a run: one piece of work, with a key the integration picks. In the Claude Code integration the run key is the NODRAIL_RUN environment variable when it is set. Otherwise it is the repository and branch the session started on, such as github.com/acme/app/fix-42, kept for the whole session so switching branch doesn't switch runs. On the default branch, or with no branch, it is the Claude Code session.
Inside a run, each subagent's work is a step, from start to finish. The integration sees the dispatch, so it knows which agent the step belongs to. When the agent finishes, its last message can end with a receipt: a short block with its verdict. Ask for it in each agent's instructions.
```nodrail-receipt
verdict: pass
findings: [correctness]
```
Each tool call the integration decides inside the run is recorded as an action. Together these give the engine what it needs: who worked, in what order, and what each finished agent reported.
For more on receipts, see Agent receipts and run history (preview).
The gate
Here is a rules file that covers only dispatching agents and pushing. It allows both, then adds two gates on top. On its own it is too narrow for real work: the local hook blocks any call no rule covers, such as a file edit. In your project, add the list and the two gates to your existing rules, and keep the allow rules you already have. Rule names must be unique across the files you load, so drop any rule here that you already define, such as the repo-work starter pack's Never force push.
- required_engine_version: 3
- list: builders
items: [dev, infra]
- rule: Dispatch agents
tool: claude-code
operation: agent
resource: { kind: agent, match: any }
effect: allow
- rule: Push branches
command: { prefix: git push }
effect: allow
- rule: Never force push
command: { prefix: git push --force }
effect: deny
- gate: Review before the pull request
tool: claude-code
operation: agent
resource: { kind: agent, match: exact, value: github }
requires:
- { agent: qa, verdict: pass, since: { list: builders } }
- { agent: reviewer, verdict: [ship, ship_with_debt], since: { list: builders } }
otherwise: require_approval
- gate: Review before a push
command: { prefix: git push }
requires:
- { agent: reviewer, verdict: [ship, ship_with_debt], since: { list: builders } }
otherwise: deny
How to read it:
- In Claude Code, starting a subagent is
tool: claude-code, operationagent. The resource is the subagent type (kindagent), sovalue: githubmatches dispatchinggithub. requireslists the steps that must exist. Each names an agent, one or more accepted verdicts, andsince.otherwiseis what happens when a requirement is unmet:require_approvalordeny.- A gate matches like a rule, with the same fields. A command gate uses the same word-boundary prefix and compound-command handling as a command rule, so
git status && git pushis still a push. - Files with gates need
required_engine_version: 3.
A gate never allows anything. A satisfied gate adds nothing, and the action still needs an allow rule. That is why the file has the two allow rules at the top. An unsatisfied gate adds its otherwise effect, and the usual precedence decides: deny, then review, then allow. The force-push rule stays a deny whatever the gate says.
What "since" means
since: { list: builders } means: count only a step that started and finished after the latest work of any agent in the builders list. "Latest work" is the latest event of that agent in the run: a step start, an action, or a step finish. A builder that is still running counts as the latest work, and a QA step that was already running when the last change happened doesn't count: it may not have seen it.
Builders aren't the only ones who can change code. With since, a change by any other agent also counts as work: the main agent editing a file, a general-purpose subagent committing, any shell command other than looking around (ls, git status, git log, git diff). Agents the gate waits for are checking the work, so only their file edits count: a reviewer running the tests undoes nothing, a reviewer editing a file undoes QA's pass.
Take this run:
| Order | Event | Is "qa passed since builders" met? |
|---|---|---|
| 1 | dev finishes a change | No |
| 2 | qa finishes with verdict: fail | No, wrong verdict |
| 3 | dev makes a fix and finishes | No |
| 4 | qa finishes with verdict: pass | Yes |
| 5 | The main agent edits one more file | No, the pass is now older than the last change |
Row 5 is the case the gate exists for. Without since, any matching step at any time in the run counts.
Verdicts compare in lowercase, so Pass and pass are the same.
When history is missing
The gate fails closed. These all leave a requirement unmet:
- no run key, or a run with no recorded history;
- a step that ended without a receipt, or with a receipt that has no verdict;
- a builder that is still running;
- a QA or review step that started before the last change.
Unmet means the otherwise effect applies. A missing record never counts as a pass.
What happens when the gate is unmet
Locally, with the open-source Claude Code hook, otherwise: require_approval makes Claude Code ask the person at the keyboard. The prompt says which steps are missing. You can let the action through or stop it. otherwise: deny blocks it, with the same explanation for the agent.
In Nodrail Cloud, an unmet gate with otherwise: require_approval creates a request in Approvals, which shows the gate that holds it. With otherwise: deny, the action is denied. The person who decides is recorded with the decision, and Insights counts how often each gate held work, what was usually missing and how often a person allowed a held action anyway.
There is no text escape hatch. An agent cannot write "skip QA, docs only" in its own brief and get past the gate. If a change really needs no QA, a person approves that one action, and that approval is on record.
Check the gate before you rely on it
Try the gate in a scratch repository first:
- Ask the main agent to open a pull request straight after a
devstep. The dispatch ofgithubshould stop and ask. - Run
qaandreviewerwith passing receipts, then dispatchgithub. It should go through. - Make one more
devchange and dispatchgithubagain. It should stop again. - Run
git pushfrom the main agent before review. It should be denied.
The local CLI shows what the run recorded:
nodrail-policy runs show github.com/acme/app/fix-42
Limits
- Receipts are self-reported. A receipt is what the agent said. An agent told to say
passwill saypass. The gate does not prove QA was good. It proves aqastep ran after the last change and reported a verdict, with time, step and run recorded. Nodrail does not check receipts against CI results. - Step ownership comes from the dispatch. The integration knows which agent a step belongs to because it saw the dispatch. An
agentfield written in a receipt is ignored. - Gate the dispatch and the push. Gating only the
githubdispatch leavesgit pushfrom the main agent open. The example gates both. If your agents can open pull requests another way, such as aghcommand or an MCP tool, gate those too. - The run key is declared by the integration. Like agent names, it is reported, not authenticated. Locally the history is a file in your repository's git directory. Anyone who can edit that file can edit the history: deny agent writes under
.git, as therepo-workstarter pack does, but a test the agent writes and runs can still write to it. Local gates keep an honest team of agents in order; they don't stop an agent set on getting around them. Nodrail Cloud keeps the history on its server, out of reach of code the agent runs. - Keep the rules out of the agent's reach. Put the rules file under
.claude/nodrail/. The hook refuses file edits to the rules it loaded, but an agent allowed to run arbitrary commands could still change them. - The hook is the boundary. As with any rule, an agent or process that can change hook settings or reach GitHub with its own credentials can bypass it. See the APF boundary notes.
A ready-made module for GitHub calls
The gate above holds the Claude Code dispatch. If your agents' GitHub calls go through an adapter that describes them as tool github, such as pr.open on the repository owner/name, the catalog module @nodrail/open-reviewed-pr packages the same idea. A publisher agent, github, may open draft pull requests after QA, a gatekeeper and a reviewer accept the current candidate, and may not start those checkers itself. Merging stays denied.
nodrail add @nodrail/open-reviewed-pr
Give it your repository, review the permission changes and lock them, as described in Playbooks (preview). It takes effect only where that adapter supplies the candidate and its evidence. The local Claude Code hook doesn't map GitHub calls, so keep the dispatch and push gates above for Claude Code.
What if the pull request is already open?
The run gate above covers the agent action that opens a PR or pushes a change. Once a PR exists, GitHub decides whether it can merge under the target branch’s rules. In our GitHub merge check pilot, we required a status for the current head, then tested whether a passing QA receipt from the previous commit could satisfy it. GitHub held the merge until fresh QA passed. Read the GitHub merge check pilot.
Next steps
- Every field, the hook setup and how to test gates in CI: Runs, gates and budgets (preview).
- Learn the rule format in Agent Policy Format.
- Set up the local Claude Code hook with Getting started.
- Add limits per step and per run with budgets for AI coding agents (preview).
- For a single reviewed action, read how to add human approval.
Use APF to hold a covered action until the required steps report the expected verdict after the latest change. Give the integration a run history and test both a satisfied gate and missing evidence. Read the gate reference (preview) or set up the local Claude Code hook.