ADR-0020: Dual-runtime — Claude Agent SDK as a second worker type behind the frozen gRPC worker contract

• Status: Accepted (founder approval 2026-07-02, upsquad-ai/upsquad-core#1627)
• Date: 2026-07-02
• Decision owners: Founder (sign-off), Principal Architect (author), Backend SME (spike), Product Manager (downstream PRDs)
• Related:
  • ADR-0012 — this ADR resolves its parked runtime-vs-middleware question with "both, per-agent choice" (upsquad-ai/upsquad-core#1198)
  • ADR-0019 — adopt-and-wrap autonomous runtime (the LangGraph worker this ADR keeps as default)
  • Spike: upsquad-ai/upsquad-core#1626 — flag-gated claude_sdk executor (merge gated on approval of this ADR); PR #1628
  • Pre-existing checkpoint schema v2/v1 mismatch: upsquad-ai/upsquad-core#1630

Status

Accepted — founder approval 2026-07-02 (via GitHub comment on #1627).

Context

UpsQuad runs agents behind a frozen gRPC worker contract — InitWorkerSession / ExecuteStep / Checkpoint / Heartbeat — with a Go orchestrator that owns the authoritative PlanLoop terminator, forced-submit_plan PLAN decomposition, BYOK multi-provider sourcing (modes B/C/D), and the stateless-worker tenet (P0.3.11). The single existing worker type is the Python LangGraph worker.

The parked question in ADR-0012 (runtime-vs-middleware, 2026-06-07) — "do we build a Claude-Code-equivalent runtime, or position as governance middleware over any runtime?" — remained open. The 2026-06-16 runtime-revamp investigation established that our LangGraph worker's tool execution, auto-compaction, and true streaming lag behind Claude Code's capability, which matters for dev/internal use cases even though the governed LangGraph path is the right answer for production/regulated agents.

Assessment (2026-07-02) confirmed the Claude Agent SDK can be embedded behind the existing frozen worker contract with all governance moats preserved via SDK hooks and canUseTool. Anthropic Commercial Terms permit SaaS embedding with API-key billing.

Decision

Adopt the Claude Agent SDK as a SECOND worker type behind the existing frozen gRPC worker contract — not a replacement. Users choose the runtime at agent creation:

LangGraph worker (default)

Fully governed. Deterministic forced-submit_plan PLAN decomposition, Go PlanLoop terminator authoritative, BYOK multi-provider (anthropic/openai/google) + local Ollama, stateless-worker tenet holds. Positioned for production / regulated agents.

Claude SDK worker

Full Claude-Code capability: real tools, auto-compaction, context management, true streaming, subagents. Tool enablement configurable per agent. Positioned for dev / internal use cases. Trade-offs accepted for this worker type only:

1. Anthropic-family models only (direct / Bedrock / Vertex) — sourcing modes B/C/D unavailable.
2. SDK owns its internal agentic loop — the Go terminator demotes from authoritative planner to a budget + HITL guard; forced-submit_plan determinism is lost for this type.
3. Scoped exception to the stateless-worker tenet (P0.3.11) — one stateful SDK session per active session, with a documented memory/cost footprint model.
4. New core-path dependency — claude-agent-sdk (supply-chain + version-churn risk).

Moat mapping (all survive)

Moat	Preserved via
Audit hash-chain	SDK hooks (PreToolUse / PostToolUse etc.)
Guardrails / clearance / HITL	canUseTool → interrupt_required
Tool governance	Built-in tools disabled + governed MCP proxy only
Persona	System prompt
Budgets	Per-turn usage → MetricsEvent (post-hoc per turn)
Tenant isolation	Orchestrator-side unchanged + per-session cwd / config-dir

Licensing: Anthropic Commercial Terms permit SaaS embedding with API-key billing (verified).

Relationship to ADR-0012

This ADR resolves the parked ADR-0012 runtime-vs-middleware question with "both, per-agent choice" and supersedes/closes that open question. Production/regulated → governed LangGraph worker; dev/internal → Claude SDK worker. See upsquad-ai/upsquad-core#1198.

Consequences

Positive

• Best-in-class dev/internal agent capability (real tools, compaction, streaming, subagents) without abandoning the governed production path.
• Governance moats fully preserved for both worker types via SDK hooks / canUseTool / governed MCP proxy.
• Resolves the long-parked ADR-0012 question without a disruptive rewrite — the frozen worker contract is untouched.
• Per-agent runtime choice becomes a product differentiator (governed vs. maximal-capability).

Negative

• Two worker runtimes to build, test, document, and maintain — higher surface area.
• Governance behaves differently across worker types (deterministic forced-plan vs. budget+HITL guard); users and docs must make this explicit.
• Claude SDK worker cannot serve BYOK modes B/C/D — a capability cliff between the two types.

Risks (top 3)

1. Two-planner split-brain + checkpoint-semantics divergence. With the SDK owning its internal loop, the Go terminator is demoted to a guard. Checkpoint semantics differ between a step-driven LangGraph worker and a turn-driven SDK session — risk of inconsistent resume/replay behaviour and audit-granularity mismatch across types. Mitigation: define per-type checkpoint contract explicitly in the LLD; keep the frozen gRPC surface identical while allowing per-type internal semantics.
2. Stateless-tenet exception + footprint. One stateful SDK session per active session breaks P0.3.11 for this type; memory/cost footprint grows with concurrency and long contexts. Mitigation: documented memory/cost model, per-session budget guard, session TTL/reaper, capacity planning before GA.
3. Provider lock-in for the SDK type. Anthropic-family only (direct/Bedrock/Vertex); no modes B/C/D. Mitigation: position explicitly as dev/internal; steer regulated/BYOK tenants to the LangGraph worker; surface the constraint at agent-creation time.

Alternatives Considered

• SDK-only replacement (rip out LangGraph). Rejected — strands BYOK/multi-provider tenants (modes B/C/D), abandons deterministic forced-plan governance, and imposes provider lock-in platform-wide.
• Middleware-over-any-runtime (pure governance layer, no owned runtime). Deferred — this is ADR-0012's alternate framing; the "both, per-agent choice" decision subsumes it for now, and a generalized any-runtime middleware can be revisited later.
• Status quo (LangGraph worker only). Rejected — leaves the dev/internal capability gap (tools, compaction, streaming, subagents) unaddressed.

Spike evidence (#1626 / PR #1628)

Recorded 2026-07-02 after the spike closed the executor-level and full-stack verification legs.

(a) Executor-level live smoke (LiteLLM → Ollama qwen2.5:3b, no real key)

Real execute_step through ClaudeSDKExecutor (real claude CLI subprocess), ANTHROPIC_BASE_URL → throwaway LiteLLM proxy exposing Anthropic /v1/messages.

• Event contract proven: status(THINKING) → token → metrics → completion(final_response).
• canUseTool deny → interrupt_required: forced Bash rm -rf / denial produced interrupt_required with idempotency key + pending intent; per-session scratch cwd created.
• Built-in-tools toggle: both states content-pinned (default OFF byte-identical; ON keeps PreToolUse/PostToolUse audit hooks + canUseTool gating on ALL tools, isolated scratch cwd). 15/15 executor tests, 422/422 full worker suite green.
• 3 real SDK-integration bugs found live (fake-SDK unit tests could not catch), all fixed: (1) query() requires a streaming AsyncIterable prompt whenever canUseTool is set; (2) extra_args={"max-tokens": …} emits an invalid --max-tokens CLI flag — removed; (3) allowed_tools=["mcp__*"] rejected by the CLI (glob only valid after mcp__<server>__ prefix) — emptied for the spike.

(b) Full-stack leg through the REAL Go orchestrator

Throwaway 2nd orchestrator + 2nd worker on the live docker network (shared Postgres/Redis, live stack untouched), LiteLLM → qwen2.5:3b, claude-agent-sdk==0.2.110 + native claude CLI 2.1.198.

• Audit hash-chain — PASS. session_started → message_received → message_completed; each prev_hash == predecessor row_hash; chain intact from genesis.
• Budget stop — PASS. Synthetic per-agent overage → ResourceExhausted{reason:budget_exhausted}, budget_denied audit row, hash chain intact, session → suspended.
• HITL control plane — PASS (worker rehydration blocked by pre-existing #1630). interrupt_required → sessionPaused(PAUSE_SOURCE_APPROVAL) → suspended; RecordDecision(APPROVED) → pauser.Resume → ResumePaused re-dialled the worker. Worker rehydration then fails on the checkpoint schema v2/v1 mismatch (#1630) — fails identically on the LangGraph path, NOT SDK-specific.

(c) Honest caveat

qwen-via-proxy proves the plumbing (event sequence, streaming input, canUseTool gating, hooks, metrics, audit, budget, HITL control plane) — not Claude-grade agentic behaviour. Claude-grade validation still needs one real-key run before GA.