Skip to main content

Workflow-YAML Hand-off — Runbook

Status: active — 2026-08-12. Founder-approved. Owner: devops-engineer. Related: core#2503, core#2498, core#2504.

TL;DR​

FIVE agent Apps cannot push a branch that touches .github/workflows/**. Only backend-sme and devops-engineer can (re-measured 2026-09-04T06:46Z — the 2026-08-13 revocation has since been executed against frontend-sme and project-manager). Check before assuming — run sg upsquad-devs -c 'python3 scripts/show-app-permissions.py workflows'.

The merge queue (#2965) does not change any of this. GitHub refuses the push, long before any merge mechanism is involved. The hand-off procedure below is unchanged; see landing-a-pr.md for what the queue does change.

If you are one of the three: commit the YAML yourself with your own identity, then ask devops-engineer to push the branch. Your authorship survives the hand-off. devops-engineer posts a hand-off comment on the PR so the record explains itself.

For qa-engineer this is a founder decision (2026-08-12), not an oversight. The basis is least privilege — and that argument is generic, so it applies to the four Apps that do hold the permission just as much. There is no principled basis on record for the current distribution. That is the open question, tracked at #2592 — not "is QA special?" but "who, if anyone, needs push access to .github/workflows/**?".

The permission is worth more than it looks. Branch protection gates merge; it does not gate execution. A workflow on a pushed branch runs from its own definition, on pull_request, before any review, with same-repo secrets in scope — and board-sync.yml puts a GitHub App private key (secrets.PJM) in that scope, with no environment: gating anywhere in the tree. See WRONG 3 before pricing any grant.

The failure you will hit​

You will see something like this from git push:

! [remote rejected] <branch> -> <branch> (refusing to allow a GitHub App to
  create or update workflow `.github/workflows/<file>.yml` without `workflows`
  permission)

It is GitHub, not branch protection, not a CI check, and not something a retry or a rebase will clear. GitHub refuses any App-token push whose tree touches .github/workflows/** unless that App installation holds workflows: write.

Note the wording: the tree, not the diff. A branch that contains a workflow edit anywhere in its history is refused even if your latest commit does not touch one.

Which apps hold workflows: write​

Measured 2026-09-04T06:46Z. Do not trust this table — reproduce it. It was wrong in the first draft of this document (two apps listed as lacking the permission hold it), which is exactly the failure mode a permission table in a runbook has: people reason from it instead of checking.

The 2026-08-13 revocation HAS NOW BEEN EXECUTED. frontend-sme and project-manager no longer hold workflows: write (re-measured 2026-09-04T06:46Z, table below). Two holders remain: backend-sme and devops-engineer.

This does not close the board-sync.yml exposure. secrets.PJM is the project-manager App's private key and it is still fed to actions/create-github-app-token on a pull_request trigger with no environment: gate. Revoking the App's own workflows permission narrows what a token minted from that key can do; it does nothing about the other permissions that key carries, and nothing about pre-merge execution reaching it at all. See WRONG 3 below before pricing this as a fix.

sg upsquad-devs -c 'python3 scripts/show-app-permissions.py workflows'
Appapp idinstallationworkflows
upsquad-product-manager[bot]3263684124769206(absent)
upsquad-project-manager[bot]3281205124769251(absent) — revoked since 2026-08-15
upsquad-principal-architect[bot]3263720124769296(absent)
upsquad-backend-sme[bot]3263728124766786write
upsquad-frontend-sme[bot]3263739124769366(absent) — revoked since 2026-08-15
upsquad-qa-engineer[bot]3263959124769426(absent)
upsquad-devops-engineer[bot]3263965124769489write

How this was measured. scripts/show-app-permissions.py mints a JWT from each App's private key in /opt/upsquad-keys/, POSTs /app/installations/{id}/access_tokens, and reads the permissions object off the response body. That object is GitHub's own statement of what the minted token can do. The script discards the token immediately and writes nothing.

Do not infer a permission from "the push worked." A push exercises contents; it exercises workflows only when the tree touches .github/workflows/**. A green push proves nothing about a permission it never needed — which is how the first version of this table came to be wrong.

Why the answer is a hand-off and not a grant​

Least privilege. That is the whole argument, and it is not QA-specific.​

An App should hold the minimum permission set that lets it do its job. qa-engineer writes tests, fixtures and coverage gates; authoring .github/workflows/** is not part of that job — it came up once, on #2498. A standing grant to cover an occasional need is exactly what least privilege exists to refuse, and the cost of the alternative is one dispatch.

This argument applies with equal force to every App that holds the permission. It is not a reason QA is special. Whether project-manager, backend-sme and frontend-sme need it is open — see #2592 — and the honest statement is that no principled basis for the current distribution has been articulated by anyone, including me.

Three arguments this document used to make, all of which are WRONG​

They were wrong in the first three drafts and are recorded here so nobody reconstructs them from first principles and thinks they are new. Note the pattern: each was a mechanism that sounded checked and was not.

WRONG 1: "a QA token that can edit the workflows removes the independent review of them." It does not. CLAUDE.md's routing table keys on what the PR touches, not who wrote it:

| PR touches | Primary reviewer (required) | | CI, workflows, IaC, k8s, deploy configs | devops-engineer |

A QA-authored workflow PR routes to devops-engineer exactly like anyone else's. No independent check is removed by the grant, so the separation-of-duties framing does not apply.

WRONG 2: "the write path should match the review path." Still not deployed, though the gap has narrowed: two of seven Apps hold workflows: write (2026-09-04) while devops-engineer is the sole required reviewer for those files. Narrowing is not adopting — it reads like a principle and is a proposal. Do not cite it as repo policy.

WRONG 3: "workflows: write is only a PUSH permission, and branch protection gates MERGE, so the risk is low." The first half is true and worth keeping. The conclusion drawn from it is wrong, and it is wrong in the direction that matters.

Branch protection gates merge. It does not gate EXECUTION. A workflow on a pushed branch runs from its own definition, on pull_request, before any review, with same-repo secrets in scope. So the real capability is not "can put a file on a branch" — it is arbitrary pre-merge CI execution with secrets, which sits upstream of every line of branch protection.

The protection settings are still exactly as measured, and they still do what they say:

required_approving_review_count: 1
dismiss_stale_reviews:           true
enforce_admins:                  true
required_status_checks.strict:   true   (10 contexts, 2026-09-04)
merge queue:                     enabled, SQUASH   (#2965 — and inert, #2967)

They are simply not the relevant control. Nothing in that list runs before a workflow does.

Direct evidence from this repo, no hypothetical required. PR #2585 added .github/workflows/migration-ceiling.yml — a workflow that did not exist on main — and it executed on that PR, twice, reporting failure on a deliberately-bad commit and success on the fix, all before the PR was reviewed or merged. That is the mechanism, demonstrated by the guard PR itself.

The concrete exposure, measured on main (2026-08-15):

workflowtriggerenvironment:secrets in scope
board-sync.ymlpull_requestnosecrets.PJM — a GitHub App private key
docs-site-pages.ymlpull_requestnoCF_API_TOKEN, CF_ACCOUNT_ID
agent-consistency.ymlpull_requestnoGITHUB_TOKEN
wave2-smoke.ymlpull_requestnoGITHUB_TOKEN

There is zero environment: gating anywhere in .github/workflows/ — 24 workflows trigger on pull_request, none of them gate a deployment environment. Reproduce with grep -rEn '^\s{4,6}environment:' .github/workflows/.

And it compounds. secrets.PJM is not a scoped token, it is the project-manager App's private key: board-sync.yml feeds it to actions/create-github-app-token to mint tokens. That App holds contents:write, issues:write, organization_projects:write and workflows:write (measured, same script as the table above). So pre-merge execution reaches a credential that can itself write code and workflows.

Consequence for the options below: Option B is more expensive than this document previously said. Granting workflows: write broadly is not granting branch-push — it is granting pre-merge CI execution with secrets, including that App key, to every App that receives it. That is a materially different trade and it should be priced as one.

Credit: found by principal-architect on the #2590 review, after both I and the reviewer had accepted the push-vs-merge framing as sufficient.

On re-opening the QA decision​

Changing any App's workflows permission — granting or revoking — is a security-relevant permission-surface change, and therefore a founder decision under CLAUDE.md's escalation list. The grant was put on the table explicitly on #2503 rather than silently foreclosed, and the founder's decision on 2026-08-12 was the documented hand-off. Re-opening it needs a new founder decision, not an agent's judgement call.

That decision was taken on the QA case in isolation. #2592 asks the wider question the least-privilege argument actually raises: who, if anyone, needs push access to .github/workflows/**, and why.

The procedure​

1. The authoring agent writes and COMMITS the YAML​

In its own worktree, with its own identity. Do not let someone else author your work:

git -c user.name='upsquad-qa-engineer[bot]' \
    -c user.email='<app-id>+upsquad-qa-engineer[bot]@users.noreply.github.com' \
    commit -m "ci(<scope>): <what the lane does>"

The commit will sit unpushed. That is expected — go to step 2.

2. devops-engineer pushes the branch​

Dispatch devops-engineer with the worktree path, the branch name and the commit SHAs. It pushes with its own token:

TOK=$(sg upsquad-devs -c 'python3 scripts/gh-token.py devops-engineer')
git push "https://x-access-token:${TOK}@github.com/upsquad-ai/upsquad-core.git" \
  HEAD:<branch>

Commit authorship is preserved. The push credential and the commit author are independent: on #2498 all four commits show as authored by upsquad-qa-engineer[bot] while only the transport was devops. Nothing about the audit trail is weakened by this.

3. devops-engineer posts the hand-off comment​

This is the step that was missing on #2498, and it is the point of the whole procedure. Without it a reviewer has to reconstruct why the author and the pusher differ, and "unexplained" is indistinguishable from "irregular".

GH_TOKEN=$(sg upsquad-devs -c 'python3 scripts/gh-token.py devops-engineer') \
gh api repos/upsquad-ai/upsquad-core/issues/<pr>/comments --method POST \
  -f body="Pushed \`<sha>\` on behalf of \`<author-app>\` — that App lacks \`workflows: write\` (core#2503). Authorship is unchanged; only the transport credential differed. I own the CI-lane review per CLAUDE.md's routing table."

Use /issues/<pr>/comments, not /pulls/<pr>/comments — GitHub treats PRs as a subtype of issues, and the issues endpoint is for general commentary. Reserve /pulls/…/comments for line-specific review comments.

4. devops-engineer reviews the CI lane​

Unchanged by this runbook — it is what the routing table already says. Doing the review is not optional just because the same agent did the push; if anything the transport makes the review more clearly owned.

What to check when you review a lane you pushed for someone else​

Pushing is not endorsing. The CI review is a separate act, and the recurring defects in this repo are all shapes that look fine from one level up:

  • No workflow-level paths: filter on anything intended to be required. A path-filtered job SKIPs and reports success, and a skipped check satisfies branch protection identically to a passing one.
  • No bare needs: <gate> without if: ${{ always() }} plus a Gate integrity first step. When the gate job fails, the dependent job never starts and reports skipped — a bypass no step-level guard can reach (core#2473; observed live in core#2389 when the org Actions quota ran out and six required checks reported skipped at once).
  • No continue-on-error at job or step level on anything gating a merge.
  • The job name must be unique across every workflow file. Two jobs publishing the same context can satisfy a required check without the real one running (core#2510; Go lint is a live duplicate today, core#2467).
  • The job name must be ASCII if it is ever to become a required context — required contexts are matched byte-exact (core#2504).
  • grep -c exits 1 on zero matches, which under set -euo pipefail kills the command substitution. That is pipefail in the safe direction for a count guard; write down which direction you are relying on.
  • Execution, not colour. go test reports ok for a package in which every selected test SKIPped. Assert the tests RAN — by name where you can, because a count floor is satisfied by any N results.

scripts/check-required-check-gating.py mechanises the first four for the required contexts. It cannot see a lane that is not required yet, which is most new lanes — so read them by hand.

If you are devops-engineer and someone hands you a branch​

  1. Read the diff before pushing. You are lending your credential; the reason you hold it is that you are the one who reviews these.
  2. Push, then post the hand-off comment in the same turn. A hand-off recorded later is a hand-off nobody sees.
  3. If the branch touches .github/workflows/** and production code, say so in the comment — the routing table wants a second reviewer for the non-CI half.
  4. Do not amend or rebase the author's commits to "tidy" them. Rewriting them under your identity destroys exactly the authorship the hand-off preserves.
Edit this page
Last updated on by upsquad-devops-engineer[bot]