Runbook — backfilling metadata["summary"] on legacy memory-review approvals

Tool: cmd/tools/memory-summary-backfill Issue: #2325 · Root fix: #2331 · Pattern precedent: #2287 / cmd/tools/memory-backfill

When to run this

A governance_approvals row with action_type='memory_review' has no metadata->>'summary', so the approvals queue renders it as an opaque upsquad-memory.review · 3b186dd6-… card — a reviewer cannot tell what they are approving without opening the detail view.

#2331 stamps the bounded, redacted preview at request time, so every approval opened since it shipped renders correctly. It repaired no history. Nothing re-requests an approval that already exists, so pre-#2331 rows cannot heal themselves — on beta-dev that is 13 pending and 7 approved rows, none with a summary key.

Run this once per environment after #2331 has deployed, and whenever the detection query below returns rows.

Detection

SELECT g.id, g.org_id, g.status, g.metadata->>'memory_id' AS memory_id, g.requested_at
FROM governance_approvals g
WHERE g.action_type = 'memory_review'
  AND COALESCE(g.metadata->>'summary', '') = ''
ORDER BY g.requested_at;

This is exactly the tool's predicate. The COALESCE(...) = '' form (rather than NOT (metadata ? 'summary')) also catches rows whose summary is present but empty or JSON null — those render just as badly.

Procedure

1. Dry run first — always

A dry run needs only DATABASE_URL. No embedder, no Redis, no gRPC, no writes.

DATABASE_URL=postgres://... \
  go run ./cmd/tools/memory-summary-backfill -dry-run

It logs one line per candidate with approval_id, memory_id, withheld, and the exact summary the row would receive. Read them. A run where every row comes back withheld=true almost always means the role cannot read agent_memory — see "Things that will bite you".

2. Backfill

DATABASE_URL=postgres://... \
  go run ./cmd/tools/memory-summary-backfill \
    [-org <org-uuid>] [-approval <uuid,uuid>] [-status pending,approved] [-batch 200] [-limit N]

For a first live run, narrow the blast radius: -limit 1, then look at the card in the inbox before sweeping the rest.

3. Verify

Re-run the detection query — it should return zero rows. Then confirm in the approvals queue that the cards now carry a readable Review <memory type> — <fact> line, and spot-check that a previously-approved row is still approved with its original expires_at.

SELECT status, expires_at,
       metadata->>'summary'               AS summary,
       metadata->>'summary_backfilled_at' AS backfilled_at
FROM governance_approvals WHERE id = '<uuid>';

backfilled_at must be non-null on every row this run touched — that is the provenance marker, and its absence on a repaired row is a bug, not a cosmetic gap.

Flags

Flag	Default	Notes
-dry-run	false	Derive and log every summary; write nothing.
-org	—	Narrows the predicate to one org.
-approval	—	Comma-separated approval ids. A filter, never an override: a row that already has a summary is still skipped.
-status	—	Comma-separated statuses (pending,approved,…). Default: all.
-batch	200	Keyset page size.
-limit	0	Max rows to process. 0 = no limit. Applies to dry runs and live runs alike.

DATABASE_URL is the only environment input.

Properties you can rely on

• Idempotent. A stamped row leaves the predicate, so a re-run is a cheap no-op. The UPDATE additionally repeats the predicate as a compare-and-set guard, so two concurrent operators cannot double-write and the second write is refused, not merged.
• Additive. The write is a JSONB merge — metadata || jsonb_build_object('summary', …, 'summary_backfilled_at', …) — so memory_id, memory_type and every other key survive untouched. An existing non-empty summary is never overwritten.
• Self-declaring. Every repaired row carries summary_backfilled_at, written in the same statement, so a backfilled preview can never be mistaken for one a reviewer actually saw. See "Things that will bite you".
• Governance state is untouched. One column, one key. status, expires_at, requested_at, decided_at are never written. An approved row stays approved; a pending row keeps its original deadline. This is display read-repair, not a governance action.
• Same producer as the online path. The preview comes from review.BuildSummary, the identical function the requester calls. The shared redact.ScrubStringDetailed scrubber, the scrub-before-truncate ordering, the fail-closed SummaryWithheld fallback and the 240-rune bound therefore hold by construction. There is no scrub logic in this tool.
• Fails closed on secrets. Content that trips the scrubber — or content the writer had already redacted, detected via the scrubber's [redacted: marker — is stamped content withheld pending review — open the detail view, never the raw text. A backfilled row is withheld under the same condition the online requester would have withheld it.
• Never writes an empty summary. An empty summary is the original defect, and approval.ErrMissingSummary rejects it at write. A degraded row (memory deleted, memory_id absent or malformed, content empty) still yields a sane type-labelled summary; a blank derivation is refused rather than persisted.
• Resumable. Keyset-paged by approval id; interrupt it and re-run.
• Fail-safe on partial failure. One bad row is logged and skipped, stays in the predicate, and is retried by a re-run. The process exits non-zero if any row failed.

Things that will bite you

The write is TERMINAL. There is no undo and no re-stamp. Read this before the live run, not after it.

A stamped row leaves the predicate, so every safety mechanism in this tool then works against you:

• There is no -force. None is planned.
• -approval is a filter, not an override. Naming a row that already has a summary skips it; it does not re-derive it.
• Re-running does nothing. Idempotency means "a second run is a no-op", not "a second run corrects the first".

Whatever review.BuildSummary emits on the day you run it is what those rows carry permanently. If a later fix improves the derivation, the already-stamped rows do not heal — they are no longer candidates. The only route back is hand-written UPDATE ... SET metadata = metadata - 'summary' - 'summary_backfilled_at' against a production governance table, which is a data-destructive operation requiring founder sign-off.

The mitigation is the dry run, and it is the whole mitigation: -dry-run prints the exact string each row would receive. Read every line before you write anything. -limit N bounds a dry run and a live run identically.

Do not run the live sweep while a known BuildSummary derivation defect is open. At the time of writing that is #2340: collapseWhitespace splits on unicode.IsSpace only, so ANSI escapes, C0 controls, bidi overrides (U+202A–U+202E, U+2066–U+2069) and zero-width characters survive into the preview verbatim. Combined with the terminal write, sweeping first would bake a spoofable preview into a security-decision surface and #2340's fix would not reach those rows. Grep the dry-run output for those code points; if any candidate is affected, wait for the derivation fix.

Every repaired row is marked. The write stamps metadata["summary_backfilled_at"] (RFC-3339 UTC) in the same statement as the summary. This exists because the sweep also repairs already-decided rows: without it, an approved card would show a preview the approver never saw and nothing would say so. The rule for anyone auditing later:

Row	Means
summary present, no summary_backfilled_at	Request-time stamp (#2331). The reviewer saw this text before deciding.
summary present, summary_backfilled_at present	This tool wrote it after the fact. On a decided row, the decision was made without it.

Do not remove the key, and do not add it by hand — it is the only thing that distinguishes the two.

The read requires a role that BYPASSES RLS — not the owner. The sweep joins agent_memory, which FORCEs an org and agent scoped policy (migration 008); the tool has no agent to scope by. Two things make a wrong role loud rather than empty: since #3498 dbguard.AssertCrossOrgReader refuses first, naming this tool and the connected role, and SET LOCAL row_security = off remains on the read transaction as the database-side backstop. Without them, an ordinary role would join to NULL content for every row and permanently stamp "content withheld" on all of them — which looks like success and is unrecoverable without manual SQL, because a stamped row leaves the predicate.

This runbook used to say "if you get a row_security permission error, use the owner role". That advice was wrong and would be refused a second time: FORCE ROW LEVEL SECURITY removes the OWNER's exemption too, which is the entire point of FORCE. Use a role with the BYPASSRLS attribute — the migration role today (dev/postgres/migrate-role.sql), the #1908 platform_rw role when it lands. After the #3259 cutover the app DSN is app_rw; give this invocation the migrate DSN explicitly rather than inheriting the service environment. See docs/runbooks/migration-role-rls-posture.md.

Writes are still scope-correct. The UPDATE transaction sets app.org_id from the row's own org_id and repeats org_id in the predicate, so the write is tenant-correct even under a BYPASSRLS role.

Withheld is a correct outcome, not a failure. Rows whose content trips the scrubber are counted separately (withheld) in the summary line. A handful is expected. All of them is a red flag (see above).

Run it after #2331 is deployed, not before. If the requester is still writing summary-less approvals, the backfill will be chasing a moving target.

Scope boundary

Only action_type='memory_review' approvals are touched. Other action types are excluded by the predicate — their card rendering is a separate concern and this tool has no opinion about it.