Skip to main content

ADR-0028 — Preview ingress security model (authenticated inbound to tenant-code sandboxes)

  • Status: Accepted (2026-07-28, per founder approval on #2236). This is the one genuinely new security surface in the Live Build Experience infra plane (HLD #2234): the first authenticated inbound edge to a running, user-driven, hostile tenant-code sandbox. Every prior sandbox invariant was egress-only ("no route out except through an allow decision"); this ADR adds exactly one new inbound edge. D1–D8 and founder decision points 1–3 are ratified; P2 (preview-router + ingress + auth + session table, #2250) is implemented behind the preview.enabled kill-switch + a per-env org allowlist (dark until P3+ wire the flow). The Cloudflare wildcard DNS + tunnel ingress change (founder decision point 1) is documented in docs/runbooks/preview-ingress.md for founder/devops application — it is not applied by the implementation PR.
  • Date: 2026-07-27 (proposed) · 2026-07-28 (accepted)
  • Deciders: Principal Architect (author + technical decision). Founder sign-off required (security-critical new ingress surface + a Cloudflare DNS/tunnel change + a beta-dev exposure acceptance — three items on the human-decision escalation list).
  • Refs: HLD #2234 §3 (ingress routing + auth) and §8 (security model). Depends on the runtime-plane user_input park (#2233). Reuses the sandbox egress fail-closed posture (HLD #2113 / LLD #2184, deployments/sandbox/base/networkpolicy.yaml), the host-edge Envoy (infra/edge/envoy.yaml, ADR-0017 staged promotion), the Clerk /rpc auth context, the workspace store + audit (mig 177), and the Quad structured-card relay (#2228). Ratifies the runtime + infra HLDs (core#2233, core#2234).

Context​

The Live Build Experience runs a workflow-built tool/service in the governed sandbox for a review window; the user opens a tenant-scoped preview URL, interacts with the real running service, and submits feedback that re-enters the build loop. Serving a running service to a browser is a categorical change from every sandbox use so far.

Verified against origin/main (c362f052):

  1. The sandbox is hostile-by-assumption and until now had zero inbound. deployments/sandbox/base/networkpolicy.yaml:23 pins ingress: [] on code-exec-sandbox pods; egress is DNS + a forced proxy on 3128 only, fail-closed by absence-of-route. The security model to date is "nothing reaches a sandbox; a sandbox reaches nothing except through an allow decision."
  2. A preview inverts that for exactly one edge: a browser must reach the running service. The served content is arbitrary tenant-generated HTML/JS/CSS — it must be treated as hostile to the platform, not merely untrusted. If it executes on a platform origin, or if any platform credential reaches it, it can hijack the user's session or exfiltrate secrets.
  3. The platform runs behind a single Cloudflare tunnel → host Envoy (:10000) with exact vhosts for research. / app. / beta-dev.app. / refund-mcp.app.upsquad.ai and a "*" catch-all (envoy.yaml:127) that currently routes unknown Hosts to the SPA. Clerk auth cookies are issued for the app origin.
  4. The batch sandbox has no notion of a per-request identity for inbound — there is no existing authenticated-ingress primitive to reuse; this is net-new.

The non-negotiable constraint: serving a hostile running service to an authenticated user must not (a) execute on any platform origin, (b) leak any platform credential into the sandbox, (c) let one preview read another's session, (d) allow unauthenticated or cross-tenant access, or (e) outlive its governed review window / cost budget. The residual we cannot fully engineer away (domain phishing) must be named and given an escalation path, not hidden.

Three URL/hosting shapes were considered: path under the app origin (app.upsquad.ai/preview/…), per-preview subdomain (preview-{id}.upsquad.ai), and single shared preview host (preview.upsquad.ai/p/{id}).

Decision​

D1 — Per-preview subdomain origin isolation. Reject path-under-app and shared-host.

  • URL shape https://preview-{previewID}.upsquad.ai, previewID = 22-char unguessable base62. Each preview gets its own browser origin, giving free origin isolation from the app and between previews — hostile JS in one preview can neither touch the app's DOM/localStorage/Clerk session nor read another preview's cookie.
  • Path-under-app is rejected on security: the preview would execute with the app origin's Clerk session, DOM, and storage — a direct XSS-to-session-hijack surface. Non-negotiable.
  • Shared-host is rejected: all previews would share one browser origin, so hostile JS in one org's preview could read the preview cookie of another org's preview open in the same browser.
  • Single-level wildcard only (*.upsquad.ai): Cloudflare Universal SSL covers one wildcard level with no ACM spend; org scoping in the hostname buys nothing because authz is enforced by the router against the session row and the unguessable ID prevents enumeration. (Envoy domains cannot express preview-*.upsquad.ai — wildcard only at start or end — so the vhost is *.upsquad.ai with the router hard-404ing any non-preview- Host; this is strictly tighter than today's "*" catch-all that leaks to the legacy SPA.)

D2 — Fail-closed authentication: handoff-token → host-only cookie. No unauthenticated previews, ever.

  • Mint (PreviewService.MintPreviewURL(run_id), context-engine, behind the authenticated /rpc gateway → Clerk authctx + org-scope GUC) checks, in order: org membership of the caller for the run's org (RLS makes cross-org a non-row), caller clearance ≥ 4 (the founder-ratified code_exec floor), session state ready|active, org preview quota, and the preview.enabled kill-switch. Returns https://preview-{id}.upsquad.ai/__preview/auth?tk=<token>.
  • Handoff token: HMAC-signed (key shared platform↔router via env secret), payload {preview_id, org_id, user_id, exp: now+60s, nonce}, single-use — the nonce is burned in preview_auth_nonce on first use. Useless for any other preview or after first use or after 60s.
  • Cookie: the router exchanges the token for its own cookie __upsq_prev — HttpOnly; Secure; SameSite=Lax; Path=/, host-only (scoped to that one preview hostname), value = signed {preview_id, user_id, exp = min(hard_ttl, now+30m)}. Silently re-mintable mid-window via a top-level redirect to the app's mint endpoint when it expires.
  • Every request without a valid cookie for that exact preview_id → 302 to app.upsquad.ai/preview/{run}/enter (re-mint if still authorised), never proxied. Session ended → 410 "review window closed." Any validation error → 404/302, never proxy. Fail-closed by construction.

D3 — Credential stripping: no platform credential can reach the hostile service. The router strips every inbound Cookie, Authorization, CF-*, and X-Forwarded-* header before proxying (after consuming its own __upsq_prev cookie). Rationale: if Clerk's cookie is ever domain-scoped to .upsquad.ai, the browser would otherwise send it to preview hosts. Combined with host-only preview cookies, no platform secret ever enters the sandbox. Serve mode additionally materialises input artifacts before launch, so even the workspace signer credential need not be present in the running service.

D4 — Pinned-endpoint reverse proxying: the router dials exactly one address and never a service-chosen one. The preview-router (new cmd/preview-router, a minimal Go httputil.ReverseProxy, no HTML parsing) resolves Host → preview_session row (in-memory cache, ≤5s TTL, revocation-checked) and reverse-proxies only to the endpoint recorded at launch (pod IP on GKE / container IP on sbx-internal on dev). It never resolves service-supplied hostnames and never follows cross-endpoint redirects — the only dialable address is the session's pinned endpoint. This preserves the egress invariant on the new inbound edge: the service still cannot initiate anywhere (its egress netpol is unchanged: DNS + forced proxy only), and the router cannot be steered to a new target by hostile responses.

D5 — frame-ancestors framing policy: safe embedding, denied clickjacking. On every proxied response the router strips X-Frame-Options and adds a second CSP header frame-ancestors https://app.upsquad.ai https://beta-dev.app.upsquad.ai. Multiple CSP headers intersect, so this only tightens — the service's own CSP directives survive, but the app can embed the preview in an iframe even if the service sets hostile framing headers, while third-party clickjacking of the preview is denied. The feedback panel lives in platform chrome, never inside the preview origin (no script injection into hostile HTML — an unreliable, false control). "Open in new tab" is always offered; the first new-tab hit lands on the router interstitial ("tenant-generated software — do not enter credentials") before continuing.

D6 — Phishing residual: named and accepted, with a founder escalation path. A hostile service can render a fake login on an upsquad.ai host. Mitigations (interstitial warning on first new-tab entry, app-chrome around the iframe, user-education copy) reduce but do not eliminate this. Residual risk is accepted and named. The strongest fix — a separate registrable domain (e.g. upsquad-preview.dev) so previews never share the upsquad.ai brand — is a founder option (new zone, trivial cost). It is the author's recommendation if brand/phishing risk is judged material; it is not required for a beta-dev/org-allowlisted demo. This is a founder decision (see below).

D7 — Kill-switch layers: a preview can always be stopped, and orphans self-terminate. Three layers plus a kernel backstop: (1) feature flag preview.enabled per-env (deploy/feature-flags/*.yaml) checked at mint + serve-start — flip to stop all new previews instantly; (2) per-org disable in the policy row; (3) PreviewService.EndAllPreviews admin RPC → reaper force pass stops every running service and resolves every parked user_input with ended_reason: "platform_killed" (runs fail-closed onto on_reject, never hang). The pod's activeDeadlineSeconds = HardTTL + slack is the kernel-side backstop that bounds the blast radius even if the reaper itself is down; on dev a startup orphan sweep catches containers past max TTL. Revocation is bounded by the router's ≤5s cache TTL, and the sandbox is stopped anyway, so the residual is ≤5s of 502s.

D8 — The new ingress edge is the ONLY relaxation; egress and cross-preview isolation are unchanged. A new preview-sandbox-netpol (podSelector preview-sandbox, distinct from the batch code-exec-sandbox) allows ingress only from the preview-router pods on $PORT and keeps egress identical in policy to the batch posture (DNS + forced proxy 3128 only). Note (per architect review of HLD #2234, Flaw 1): because the forced-proxy's own ingress netpol admits only code-exec-sandbox-labeled pods today, enabling preview egress on GKE requires an additive from: preview-sandbox selector on egress-proxy-netpol.ingress (or a shared sandbox-egress-client label) — a fail-closed change folded into the serve-substrate slice, asserted by extending the P0–P4 egress proof to a preview-sandbox pod. Cross-preview pod-to-pod remains topologically unreachable. The router runs non-root, distroless, in its own namespace with egress restricted to the sandbox namespace + Postgres, and streams response bodies through untouched except for the D3/D5 header policy — it has no HTML parser to compromise.

Consequences​

  • Positive: authenticated, tenant-scoped, fail-closed preview access with per-preview browser-origin isolation; no platform credential reachable by hostile code; no unauthenticated or cross-tenant path; the egress fail-closed invariant is preserved on the new inbound edge (pinned dial, no redirect-following); every preview is always killable and orphans self-terminate; cost is metered onto the same per-run truth as LLM/batch compute. The router is a small, auditable, HTML-blind reverse proxy.
  • Negative / cost: one new internet-reachable inbound edge to hostile code (the whole point of this ADR); a new cmd/preview-router service + Envoy wildcard vhost + a Cloudflare DNS/tunnel change; a ≤5s revocation window (bounded, and the backend is dead after end anyway); the domain-phishing residual (D6) unless the founder elects a dedicated preview domain; a required additive change to the shared egress-proxy-netpol (D8 note) for preview egress on GKE.
  • Security: the model is fail-closed at every gate (no session row / any validation error → 404/302, never proxy); the no-credential-reaches-sandbox property (D3) and the pinned-dial property (D4) are the two invariants a security review must probe (header-strip completeness incl. CF-*/X-Forwarded-*; no service-supplied host is ever dialable; no redirect chases a new endpoint). The preview-ingress-smoke.sh suite (no cookie → never proxied; token single-use; cross-preview cookie rejected; revoked ≤5s; inbound Cookie/Auth stripped; the sandbox still fails P0–P4 egress) is the fail-closed proof.

Alternatives rejected​

  • Path under the app origin (app.upsquad.ai/preview/…): rejected outright — hostile tenant HTML/JS would execute with the platform's Clerk session, DOM, and storage. A direct session-hijack surface with no mitigation.
  • Single shared preview host (preview.upsquad.ai/p/{id}): rejected — all previews share one browser origin, so one org's hostile preview can read another org's preview cookie in the same browser. Per-preview subdomains give this isolation for free.
  • Two-level per-org hostname (preview-{run}.{org}.upsquad.ai): rejected on cert cost/complexity — Cloudflare Universal SSL covers only one wildcard level; two-level patterns need Advanced Certificate Manager. Org scoping in the hostname buys nothing over router-enforced authz against the session row + an unguessable ID.
  • Cloudflare Access login wall on preview hosts: rejected as the primary control — an Access login wall on every preview would break tenant UX; router-level handoff-token → cookie auth is the control, with an Access bypass app recommended for preview hostnames (a founder posture decision below).
  • Script-injected feedback capture inside the preview: rejected as a false control — injecting a feedback widget into hostile HTML is unreliable and gives the hostile page a handle on the widget; the feedback panel lives in platform chrome around a cross-origin iframe instead.

Founder decision points (gate P2 — router + ingress — implementation)​

  1. Wildcard DNS + tunnel ingress for *.upsquad.ai (Cloudflare change; Universal SSL suffices, no ACM spend at one level). Alternative requiring a small budget: a dedicated preview domain (e.g. upsquad-preview.dev) that fully kills the D6 domain-phishing residual — a new zone, ~trivial cost, the author's recommendation if brand/phishing risk is judged material.
  2. Cloudflare Access posture for preview hosts — router-level auth is the control; recommend an Access bypass app for preview hostnames (a full Access wall would break tenant UX).
  3. Beta-dev exposure acceptance — internet-reachable, runc-degraded previews on the shared beta-dev box, gated to an org allowlist (deploy/feature-flags/dev.yaml), all container hardening retained, degraded=true loudly surfaced, RuntimeAssertion still blocking runc under enforcement. This is a real elevation over batch (which had no inbound) and needs explicit sign-off, mirroring the WR2-3 degraded-runtime acceptance.

This ADR is Accepted (2026-07-28, per founder approval on #2236): D1–D8 and founder decision points 1–3 above are ratified. P2 (preview-router + ingress + auth + session table, #2250) is implemented behind the preview.enabled kill-switch + per-env org allowlist, dark until P3+ wire the flow; the Cloudflare wildcard DNS + tunnel ingress (point 1) is documented in docs/runbooks/preview-ingress.md for founder/devops application. P1 (serve substrate: serve-mode sandbox, preview netpol, readiness — no edge, no ingress) was independently cleared by the architect ahead of this sign-off.

Edit this page
Last updated on by upsquad-backend-sme[bot]