# security-scan.yml — PR-gate security scanning (SOC2/FedRAMP compliance).
#
# Runs two independent scan jobs on every PR and every merge-queue entry:
#   1. trivy          — Container image scan for CRITICAL CVEs
#   2. class-coverage — Data-class registry coverage gate (defence-in-depth, LLD 18)
#
# PR gate: fails on actionable findings only.
#
# Scope history (#740, PRD #738):
#   * Phase 1 (#740) dropped the `govulncheck` and `gosec` jobs from the PR
#     gate. They ran as `continue-on-error: true` here (stdlib CVEs + the
#     G115/G118/G703 ADR-0010 baseline) which added ~2-3 billed minutes per
#     PR for signal that was either non-actionable or informational.
#     Both scanners now run on main HEAD via `.github/workflows/
#     daily-security-sweep.yml`, where a new finding (i.e. not in the ADR
#     baseline) files a dated P0/P1 issue via `scripts/daily-finding-filer.py`.
#
# Path filtering (see #728 P2):
#   A `changes` gate detects whether each job needs to run work. Both jobs
#   still execute (so their required check-runs report green — branch
#   protection depends on `Container Image Scan` and `Data-Class Registry
#   Coverage` reporting on every PR) but the heavyweight steps are gated
#   behind `if: needs.changes.outputs.relevant` so prose-only PRs finish in
#   ~10s each. NB 'config-only' is no longer in that set — see #2544 below;
#   config IS an input and the old filter's silence about it was the bug.
#
# GATE-JOB FAILURE MUST NOT SKIP-GREEN THESE CHECKS (issue #2473):
#   This file hosts TWO of the eight required contexts, and until #2473 a
#   single `changes` failure disabled both at once. `needs: changes` alone
#   means that when `changes` FAILS — rather than reporting "nothing relevant
#   changed" — GitHub never starts the dependent job, so it reports `skipped`,
#   and a skipped required check SATISFIES branch protection. The step-level
#   `if:` pattern described just above does nothing about it: the failure is
#   one level up, at the job, so there is no step left to emit anything.
#
#   Observed on 2026-07-31 (#2389, org Actions quota exhausted): every gate job
#   died at start, all six required checks reported `skipped`, and PRs read
#   mergeable with zero CI executed.
#
#   Fix on both `trivy` and `class-coverage` below, a pair that must be read
#   together — neither half works alone:
#     1. `if: ${{ always() }}` at job level so the job STARTS regardless of the
#        gate's conclusion; it can then never report `skipped`.
#     2. A `Gate integrity` FIRST step failing closed unless every upstream job
#        reached `success` AND every consumed output is a literal 'true'/'false'.
#        Without (2), (1) is worse than the original bug: a failed gate leaves
#        the outputs EMPTY, every long `if:` chain below evaluates false, and
#        the job reports green having scanned nothing.
#
#   `always()` not `!cancelled()`: the latter SKIPS on cancelled runs, and
#   skipped satisfies branch protection — the same defect, narrower.
#
#   The `CONSUMED_OUTPUTS` on each guard lists exactly the outputs that job's
#   own `if:` chains read — since #2544 that is the single `changes.relevant`
#   for both. scripts/check-required-check-gating.py derives the same set from
#   the job and fails if the two disagree, so adding a new filter output and
#   gating a step on it cannot leave that output unverified.
#
# THE FILTER IS NOW A DENY-LIST (issue #2544) — the "was it CORRECT" axis
# ------------------------------------------------------------------------
#   #2473 above made both required jobs fail closed when the gate job DIES.
#   It does nothing about a gate job that SUCCEEDS and returns an honest but
#   too-narrow `false`. Until this change the filter held four hand-maintained
#   ALLOW-lists (`go`, `docker`, `migrations`, `workflow`), so a change outside
#   the enumerated globs produced a load-bearing green from two REQUIRED checks
#   that did no work. Same class as #2449 and #2465, on the two checks in this
#   file.
#
#   Converted to the shape those two established: `'**'` plus reasoned
#   negations under `predicate-quantifier: 'every'`, i.e. exact set
#   subtraction. Everything is an input by default; each exclusion has to earn
#   its place, and a new directory is covered the day it lands rather than the
#   day someone remembers to add a glob.
#
#   FOUR RULES COLLAPSED TO ONE, and that is a finding rather than a tidy-up.
#   Under a deny-list the two jobs' input sets are provably IDENTICAL: the
#   whole tree minus prose and binary assets. `docker` vs `migrations` only
#   distinguished them while both were enumerations of what someone
#   remembered. One rule cannot drift from itself; two identical ones can.
#
#   FAILURE DIRECTION, stated because it is the whole point: if
#   `predicate-quantifier` were ever dropped or ignored, dorny falls back to
#   'some' — under which the bare '**' matches every path and both jobs ALWAYS
#   RUN. Degradation is toward more execution, never toward a skip-green.
#
#   Do NOT add a positive pattern beside '**': under `every` that becomes an
#   intersection and silently NARROWS the filter to nothing.
#
#   #2474 IS DISARMED IN THIS FILE, not merely avoided. The trap is that
#   `predicate-quantifier` is ACTION-level, so adding it to a step holding
#   allow-list rules makes every rule with >=2 positive patterns unsatisfiable
#   — `go` and `docker` were 2 of the 4 rules here, and both required checks
#   would have gone green-forever. There is now exactly one rule and it is a
#   deny-list, so there is no allow-list left in this step for the quantifier
#   to break. This does NOT close #2474: `publish-images.yml` (18/18 rules) and
#   `golden-flow-e2e.yml` (2/2) are still armed, and the guard that ticket asks
#   for is still worth building.
#
#   Coverage is a STRICT SUPERSET of what the four allow-lists matched,
#   verified by replaying both shapes through dorny v3's real matcher
#   (picomatch 2.3.1, `{dot: true}`, per-pattern compilation) over all 3996
#   tracked files: zero files trigger the old filter and not the new one. That
#   is why there is no blanket `!**/*.md` here even though `db.yml` and
#   `wave2-smoke.yml` both carry one — the old `deploy/**` and
#   `migrations/**` globs reach markdown inside those trees, so excluding
#   markdown wholesale would have made this conversion a NARROWING in two
#   places. Measured price of the absolute property: 0.5 percentage points of
#   trigger rate. Cheap, and it means this change cannot possibly reduce
#   coverage, which is the entire concern of #2544.

name: Security Scan

on:
  # NO `push: main` (#2989). This lane is merge-queue-covered: the `merge_group`
  # run below validates the exact tree that lands, so a post-merge re-run only
  # re-scans an identical SHA. Do not re-add it while the queue is enabled. Both
  # jobs keep a standing `main` verdict, by TWO DIFFERENT routes — do not
  # collapse them into one claim: `Container Image Scan` via the scheduled
  # `daily-security-sweep.yml` (Trivy over the GHCR service images), and
  # `Data-Class Registry Coverage` via its twin
  # `test/lint/data_class_coverage_test.go`, which `main-green-nightly.yml`
  # exercises in `go test -race ./...` against merged `main`. The sweep does NOT
  # run `-mode=class-coverage`: that flag is invoked from this file and from
  # nowhere else in `.github/workflows/**`.
  pull_request:
    branches: [main]
  # merge_group (#2967): the merge queue dispatches this event against a
  # `gh-readonly-queue/main/...` ref. Without it the required contexts below
  # never report inside the queue, the entry burns the 3600s timeout and is
  # ejected — i.e. nothing can land at all.
  merge_group:


permissions:
  contents: read
  # dorny/paths-filter needs pull-requests:read to compute the diff on PR
  # events. Without it, the `changes` job fails with "Resource not accessible
  # by integration" and all downstream jobs skip.
  pull-requests: read

# Cancel superseded runs for the same PR ref (#2989, CI cost Move 1). The
# condition scopes cancellation to `pull_request` only: `push`, `merge_group`,
# `schedule` and `issues` runs always run to completion, so no merged SHA and no
# queue entry ever loses its verdict.
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
  changes:
    name: Detect relevant changes
    runs-on: ubuntu-latest
    outputs:
      # merge_group (#2967): the queue's composed tree has no PR base to diff
      # against, so the filter cannot answer. Force TRUE rather than letting it
      # guess — an unfiltered run is the only safe default on the one path that
      # reaches main, where a wrong 'false' is a skip-as-pass straight to main.
      relevant: ${{ github.event_name == 'merge_group' && 'true' || steps.filter.outputs.relevant }}
    steps:
      - uses: actions/checkout@v6
      - name: Filter changed paths — deny-list (#2544)
        id: filter
        if: github.event_name != 'merge_group'   # no PR base to diff (#2967)
        uses: dorny/paths-filter@v3
        with:
          # `every` means a file counts as relevant only when it satisfies
          # EVERY pattern in the rule: it matches '**' (all files, `dot: true`,
          # so dotfiles and .github/** are included) and it matches every
          # negation, i.e. none of them excluded it. Verified against the
          # pinned action source at tag v3: each entry is compiled through
          # picomatch(p, {dot:true}) INDEPENDENTLY and then `patterns.every`,
          # so a leading '!' yields a matcher true for non-matching paths and
          # `every` is exact set subtraction. An invalid quantifier value makes
          # the action throw, so a typo fails the job rather than degrading it.
          predicate-quantifier: 'every'
          filters: |
            relevant:
              # Everything is an input by default. `Container Image Scan`
              # builds ./cmd/context-engine from a Dockerfile whose builder
              # stage is `COPY . .` with an EMPTY .dockerignore, and
              # `Data-Class Registry Coverage` runs a Go binary over the
              # migration tree. New packages, new services, new seeds and new
              # Dockerfiles are covered the day they land.
              - '**'

              # ---- exclusions; each needs a reason that survives review ----

              # Prose TREES. Note there is deliberately no blanket
              # `!**/*.md` — see the file header. The old `deploy/**` and
              # `internal/context/store/migrations/**` globs reach markdown
              # inside those trees, so a wholesale markdown exclusion would
              # make this conversion a NARROWING there, and dorny exclusions
              # are FINAL (a path excluded by one pattern cannot be
              # re-included by a later one). Excluding whole prose trees costs
              # 0.5pp of trigger rate and keeps the superset property absolute.
              - '!docs/**'
              - '!docs-site/**'
              # Agent role contracts and planning scratch (tasks/todo.md,
              # tasks/lessons.md). Not a build input for any image; not read by
              # any running service; contain no .go, .sql or Dockerfile.
              - '!.claude/**'
              - '!tasks/**'
              # TypeScript/JS only: 0 Go, 0 SQL, 0 Dockerfile inputs to EITHER
              # job. The image built here is `./cmd/context-engine`, whose
              # runtime stage carries the static Go binary plus deploy/seeds
              # and nothing else, so no browser asset can reach Trivy's scan
              # surface. (`ui/caveman/gen/**` IS a build input to proto.yml and
              # is deliberately not excluded there — see that file.)
              - '!ui/**'
              - '!frontend/**'
              - '!src/**'
              # Binary assets.
              - '!**/*.png'
              - '!**/*.jpg'
              - '!**/*.jpeg'
              - '!**/*.gif'
              - '!**/*.svg'
              - '!**/*.ico'
              #
              # DELIBERATELY NOT EXCLUDED, each for a concrete reason — resist
              # the urge to add them back for speed:
              #   **/*_test.go   Excluding them WOULD be sound (`go build`
              #                  ignores them) but it is a NARROWING relative
              #                  to the old `**/*.go` allow-list, and the
              #                  strict-superset property is worth more than
              #                  the handful of runner-minutes. If it is ever
              #                  wanted, argue it as its own change.
              #   .github/**     dorny exclusion is FINAL, so excluding
              #                  .github/** would also exclude THIS file and
              #                  make the workflow unable to gate its own
              #                  changes. It also replaces the old `workflow`
              #                  rule, which existed solely for that.
              #   scripts/**, deployments/**, infra/**, dev/**, deploy/**
              #                  all contain .sql, .sh or seed YAML that one of
              #                  the two jobs reads or ships.

  trivy:
    name: Container Image Scan
    # Migrated to the self-hosted Hetzner fleet (#3242 phase 3, wave a).
    # Rollback for this lane == revert this PR; nothing else changed.
    runs-on: [self-hosted, hetzner-ci]
    needs: changes
    # #2473. STARTS regardless of what `changes` concluded, so this required
    # check can never report `skipped`. Paired with the `Gate integrity` step
    # below, which is what stops an ungated start from becoming a green one.
    # See the file header for why this is `always()` and not `!cancelled()`.
    if: ${{ always() }}
    steps:
      # ─── #2473 GATE-INTEGRITY GUARD — read the file header before editing ───
      # No `if:` on this step, deliberately: it must run on every start of this
      # job, including the starts that happen only because of `if: always()`.
      # `CONSUMED_OUTPUTS` names every gate output this job's own step-level
      # `if:` conditions read. The `run:` script is byte-identical across all
      # six required jobs; scripts/check-required-check-gating.py asserts that.
      - name: Gate integrity — upstream gate must have delivered a verdict
        env:
          NEEDS: ${{ toJSON(needs) }}
          CONSUMED_OUTPUTS: changes.relevant
        run: |
          set -euo pipefail
          python3 - <<'PY'
          import json, os, sys

          needs = json.loads(os.environ.get("NEEDS") or "null")
          consumed = os.environ.get("CONSUMED_OUTPUTS", "").split()
          problems = []

          if not isinstance(needs, dict) or not needs:
              problems.append(
                  "the `needs` context is empty or unparseable — this job declares "
                  "`needs:` but received no upstream state at all")
              needs = {}

          # A gate job that FAILED, was CANCELLED, or was itself SKIPPED cannot
          # have decided anything. Refuse to stand in for a decision nobody made.
          for job in sorted(needs):
              result = (needs.get(job) or {}).get("result")
              if result != "success":
                  problems.append(
                      f"upstream job `{job}` concluded `{result}` (expected `success`)")

          # A gate job can also succeed while emitting nothing — a renamed step
          # id, a renamed filter rule, a `continue-on-error` swallowing the real
          # failure. The step-level `if:`s below read those outputs with `==
          # 'true'`, so an EMPTY value silently means "skip everything" and the
          # job goes green having done no work. Demand a literal verdict.
          for spec in consumed:
              job, _, name = spec.partition(".")
              value = ((needs.get(job) or {}).get("outputs") or {}).get(name)
              if value not in ("true", "false"):
                  problems.append(
                      f"gate output `{spec}` is {value!r}; expected the literal "
                      "string 'true' or 'false'")

          if problems:
              print("::error::GATE INTEGRITY FAILURE (#2473) — this job's run/skip "
                    "decision depends on an upstream gate that did not deliver a "
                    "usable verdict, so it cannot honestly report this REQUIRED "
                    "check green. Failing closed.")
              for p in problems:
                  print(f"::error::  - {p}")
              print("needs context as received:")
              print(json.dumps(needs, indent=2, sort_keys=True))
              sys.exit(1)

          print("gate integrity OK — every upstream job succeeded and every "
                "consumed output carries a literal 'true'/'false':")
          for spec in consumed:
              job, _, name = spec.partition(".")
              print(f"  {spec} = {needs[job]['outputs'][name]}")
          PY

      - name: Skip if no relevant changes
        if: needs.changes.outputs.relevant != 'true'
        run: |
          echo "Diff is prose/assets only (deny-list #2544) — skipping container scan."

      - name: Checkout
        if: needs.changes.outputs.relevant == 'true'
        uses: actions/checkout@v6

      - name: Set up Docker Buildx
        if: needs.changes.outputs.relevant == 'true'
        uses: docker/setup-buildx-action@v4

      - name: Build context-engine image for scanning
        if: needs.changes.outputs.relevant == 'true'
        run: |
          docker build \
            -t upsquad/context-engine:scan \
            --build-arg SERVICE=context-engine \
            .

      - name: Run Trivy vulnerability scanner
        if: needs.changes.outputs.relevant == 'true'
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: upsquad/context-engine:scan
          format: table
          exit-code: '1'
          severity: CRITICAL
          trivyignores: .trivyignore

  class-coverage:
    # Defence-in-depth for LLD 18 (#461): re-runs the data-class registry
    # coverage gate from the CLI so a broken Go-test (test/lint/data_class_coverage_test.go)
    # cannot silently bypass the invariant. Both gates are expected green;
    # both cover the same check — that every RLS-enabled migration row
    # maps to a classregistry.Scope entry.
    #
    # Follow-up from closed PR #469 / issue #470.
    name: Data-Class Registry Coverage
    # Migrated to the self-hosted Hetzner fleet (#3242 phase 3, wave a).
    # Rollback for this lane == revert this PR; nothing else changed.
    runs-on: [self-hosted, hetzner-ci]
    needs: changes
    # #2473. STARTS regardless of what `changes` concluded, so this required
    # check can never report `skipped`. Paired with the `Gate integrity` step
    # below, which is what stops an ungated start from becoming a green one.
    # See the file header for why this is `always()` and not `!cancelled()`.
    if: ${{ always() }}
    steps:
      # ─── #2473 GATE-INTEGRITY GUARD — read the file header before editing ───
      # No `if:` on this step, deliberately: it must run on every start of this
      # job, including the starts that happen only because of `if: always()`.
      # `CONSUMED_OUTPUTS` names every gate output this job's own step-level
      # `if:` conditions read. The `run:` script is byte-identical across all
      # six required jobs; scripts/check-required-check-gating.py asserts that.
      - name: Gate integrity — upstream gate must have delivered a verdict
        env:
          NEEDS: ${{ toJSON(needs) }}
          CONSUMED_OUTPUTS: changes.relevant
        run: |
          set -euo pipefail
          python3 - <<'PY'
          import json, os, sys

          needs = json.loads(os.environ.get("NEEDS") or "null")
          consumed = os.environ.get("CONSUMED_OUTPUTS", "").split()
          problems = []

          if not isinstance(needs, dict) or not needs:
              problems.append(
                  "the `needs` context is empty or unparseable — this job declares "
                  "`needs:` but received no upstream state at all")
              needs = {}

          # A gate job that FAILED, was CANCELLED, or was itself SKIPPED cannot
          # have decided anything. Refuse to stand in for a decision nobody made.
          for job in sorted(needs):
              result = (needs.get(job) or {}).get("result")
              if result != "success":
                  problems.append(
                      f"upstream job `{job}` concluded `{result}` (expected `success`)")

          # A gate job can also succeed while emitting nothing — a renamed step
          # id, a renamed filter rule, a `continue-on-error` swallowing the real
          # failure. The step-level `if:`s below read those outputs with `==
          # 'true'`, so an EMPTY value silently means "skip everything" and the
          # job goes green having done no work. Demand a literal verdict.
          for spec in consumed:
              job, _, name = spec.partition(".")
              value = ((needs.get(job) or {}).get("outputs") or {}).get(name)
              if value not in ("true", "false"):
                  problems.append(
                      f"gate output `{spec}` is {value!r}; expected the literal "
                      "string 'true' or 'false'")

          if problems:
              print("::error::GATE INTEGRITY FAILURE (#2473) — this job's run/skip "
                    "decision depends on an upstream gate that did not deliver a "
                    "usable verdict, so it cannot honestly report this REQUIRED "
                    "check green. Failing closed.")
              for p in problems:
                  print(f"::error::  - {p}")
              print("needs context as received:")
              print(json.dumps(needs, indent=2, sort_keys=True))
              sys.exit(1)

          print("gate integrity OK — every upstream job succeeded and every "
                "consumed output carries a literal 'true'/'false':")
          for spec in consumed:
              job, _, name = spec.partition(".")
              print(f"  {spec} = {needs[job]['outputs'][name]}")
          PY

      - name: Skip if no relevant changes
        if: needs.changes.outputs.relevant != 'true'
        run: |
          echo "Diff is prose/assets only (deny-list #2544) — skipping class-coverage gate."

      - name: Checkout
        if: needs.changes.outputs.relevant == 'true'
        uses: actions/checkout@v6

      - name: Set up Go
        if: needs.changes.outputs.relevant == 'true'
        uses: actions/setup-go@v6
        with:
          go-version-file: go.mod
          cache: false

      - name: Run class-coverage gate (CLI)
        if: needs.changes.outputs.relevant == 'true'
        run: |
          go run ./cmd/schema-coverage-aggregator \
            -mode=class-coverage \
            -migrations-dir internal/context/store/migrations