UpsQuad — Agent Ecosystem: Component Onboarding Guide
Audience: a new engineer joining any part of the backend. This is the master map — read it first, then dive into the focused KTs (e.g. Governed MCP & the Team Gateway).
Goal: know what every component does, where it lives, and — the whole point — what is real vs stubbed vs dev-only vs parked, verified against the code, not the wiki.
Status legend used throughout: 🟢 LIVE (merged, in the prod code path) · 🟡 STUBBED (placeholder / present but not final) · 🔵 DEV-ONLY (only wired for beta-dev/docker) · 🟠 PENDING/PARKED (designed and/or coded but gated off / blocked).
How to read each component: What it does · Key files · Status · Stubs/gaps in it · Upstreams (who calls it) · Downstreams (what it drives) · Edge stubs. Trust the Status and Stubs rows — every one below was re-checked at file:line against origin/main.
Changelog since 2026-07-06 (imported from Knowledge-Base Google Doc)
The base of this guide was the founder's living component doc (KB Doc, ~2026-07-06). This revision re-verified every Status/Stubs claim against origin/main at 9858af5f (2026-07-24). What changed:
| # | Claim in the 2026-07-06 base | Now | Evidence / driver |
|---|
| 1 | ❌ langgraph worker's Context-Engine client is a stub (context/client.py returns system_prompt="", no channel) — "the single biggest looks-wired-isn't edge" | 🟢 CLOSED — client is REAL. Both runtimes now call the live Context Engine over Connect-HTTP (AssembleContext/ValidatePrompt/IngestEvent), fail-open with an integrity_check_skipped audit on error. langgraph turns are no longer knowledge-blind. | services/agent-worker/src/agent_worker/context/client.py (rewritten, #2030/#2031, transport Option C founder-approved 2026-07-23; dev bypass sequencing #2034); wired at graph/nodes.py:161,250 |
| 2 | ❌ L4 persona assembly empty → prompt_builder emits "[No agent persona configured]" | 🟡 STILL STUBBED (confirmed). No production writer inserts an L4 row into immutable_layers; prompt_builder.go:124 still emits the placeholder. Persona reaches the model on claude_sdk via snapshot.persona; on langgraph the model now gets real L1–L3 + RAG context but still no L4 persona. | internal/context/assembly/prompt_builder.go:124; no INSERT INTO immutable_layers for L4 in prod code; initializer.go:459 only reads L4 |
| 3 | MCP tool execution = dev, via ContextForge services/mcp-proxy plugin chain | 🟠→🟢 Re-platformed. ContextForge removed (services/mcp-proxy gone, #1909/#1912); governed MCP is now the Go edge Team Gateway PEP (internal/gateway/mcpproxy) per ADR-0024, with an egress/SSRF guard per ADR-0025. Ships feature-OFF (MCP_GATEWAY_ENABLED default false); prod parked on #1864. | internal/gateway/mcpproxy/{dispatch,edge_policy}.go; internal/mcp/egress/*; ADR-0024/0025 |
| 4 | Guardrail turn-boundary re-pin LIVE | 🟢 Confirmed unchanged. | internal/runtime/server/lifecycle_repin.go, internal/context/guardrail/resolver.go, autoloop/repin.go |
| 5 | Decision-receipts + governed approvals LIVE | 🟢 Confirmed. Receipt emit/verify pipeline present. | internal/governance/receipt_emit.go, internal/governance/receiptverify/verify.go, cmd/verify-receipt |
| 6 | T4 autonomy LIVE on dev, founder-gated for prod | 🟢 Confirmed. EnableAutonomousLoop is off by default. | internal/runtime/server/grpc.go:400 ("Off by default"), internal/runtime/autoloop/*; gate #1422 |
| 7 | External-agent connect (BYO) + /oauth/token | 🟢 NEW since base. DB trust-table + client-credentials mint; prod parked on #1864. | internal/agent/external_service.go, internal/auth/oauth/*, mig 155_external_agent_clients |
Schema advanced from the base's ~mig 126 to mig 168. New ADRs since base: ADR-0023 (workflow substrate), ADR-0024 (MCP governance canonical surfaces), ADR-0025 (MCP egress / private upstreams).
Architecture
The original hand-drawn PNG (Arch_June29_2026.png) still lives in the Knowledge-Base Google Doc. The dead Drive link has been replaced by the numbered, colour-coded Mermaid diagram below, which is the maintained source of truth.
Colour key: 🔵 blue = client surface / agent runtime · 🟢 green = context engine · 🟡 amber = governance / LLM egress · 🟣 purple = MCP team gateway PEP · 🔴 red = egress/SSRF + upstream · ⚪ grey = data stores.
Walkthrough (numbers match the diagram)
- User turn. The SPA (or the per-user Quad surface) calls
SendMessage over Connect to the context-engine /rpc gateway.
- Proxy. The /rpc gateway proxies
LifecycleService (and OrgUnitService/ModelRegistryService) through to the Orchestrator.
- Step dispatch. The orchestrator authenticates, checks budget/security, fetches BYOK keys, and dials the Agent Worker via
ExecuteStep (gRPC mTLS).
- Context assembly. The worker calls the Context Engine
AssembleContext + ValidatePrompt over Connect-HTTP — now a real call on both runtimes (was a stub on langgraph pre-#2030).
- Governance. Pre-LLM and per-tool
Check calls hit the embedded Governance PDP (fail-closed).
- LLM.
call_llm goes to the AI Gateway (BYOK / budget) or Ollama.
- Tools.
tools/call carries a Hop-A JWT to the Team Gateway edge PEP at /mcp/team/{unit}.
- Edge gates → forward. The PEP runs ordered fail-closed gates, rewrites to the raw tool, and forwards through the egress guard.
- Egress. Only allowlisted/pinned hosts are reachable; private IPs and SSRF sinks are denied.
- Autonomy. For T4 sessions the orchestrator's autoloop drives successive steps and parks in-step interrupts against the approval backend.
- Audit. Every verdict / receipt / LLM+tool call is appended to Postgres — the audit-as-source-control tenet.
Components
1. Orchestrator — Go core (agent-orchestrator)
| |
|---|
| What it does | The outer authority. LifecycleService.SendMessage: auth → budget → security → fetch BYOK keys → dial worker → relay events → audit. Hosts the governance PDP, the two coordinators (peer/workflow + delegation/subagent), the RelayWaiter (live-bubbling), the model registry, and the Go-side autonomous PlanLoop (T4). |
| Key files | cmd/agent-orchestrator/main.go · internal/runtime/server/{grpc,lifecycle_*,child_dispatch,relay_waiter,lifecycle_repin,lifecycle_observer}.go · internal/runtime/autoloop/{loop,driver,plan,terminator,interrupt,repin}.go |
| Status | 🟢 LIVE — single-agent turns, agent→agent delegation, and the orchestrator-side autonomous PlanLoop run end-to-end. In-step interrupts park via approval.Service + PauseManager and resume on RecordDecision. Per-agent runtime selection (langgraph | claude_sdk) is plumbed here (ADR-0020). |
| Stubs / gaps in it | 🔵 T4 is DEV-ONLY by default — EnableAutonomousLoop is off (grpc.go:400); prod enablement is founder-gated (#1422). The deep-autonomy driver lives here (internal/runtime/autoloop), not in the worker — the worker only supplies PLAN-step decomposition. |
| Upstreams | context-engine /rpc gateway (proxies LifecycleService from the SPA/Quad); the user via SendMessage. |
| Downstreams | agent-worker (ExecuteStep, gRPC mTLS) · governance.Engine (embedded PDP) · approval.Service + PauseManager · Postgres (sessions, audit, policies) · coordinators dispatch child sessions · Team Gateway mints the Hop-A JWT per ExecuteStep. |
| Edge stubs | The worker it drives now does get real context on both runtimes (see #3) — the base's "langgraph runs with empty system prompts" edge is closed. The one residual is L4 persona (see #2/#8). |
2. Context Engine — Go (context-engine)
| |
|---|
| What it does | Layered prompt assembly (L1 platform → L2 tenant → L3 team → L4 persona), hybrid RAG retrieval, versioning DAG, and compaction. Also the /rpc gateway the dashboard SPA talks to (proxies AgentService, OrgUnitService, LifecycleService, ModelRegistryService), and hosts the JWKS / issuer routes for Hop-A trust. |
| Key files | internal/context/assembly/{pipeline,prompt_builder,layer_loader,budget_packer}.go · internal/context/server/server.go · internal/context/guardrail/resolver.go · cmd/context-engine/main.go |
| Status | 🟢 LIVE. Assembly/RAG/versioning real and running. Compaction wired (ADR-0019: async producer + sync fallback + SummaryLoader). It now has a live out-of-process consumer: the worker's Context-Engine client is real (#2030), so AssembleContext/ValidatePrompt are exercised on every turn on both runtimes. Turn-boundary guardrail re-pin (ResolveGuardrailLayers) is served here. |
| Stubs / gaps in it | 🟡 immutable_layers L4 still has zero production writers → L4 always empty → prompt_builder.go:124 still emits "[No agent persona configured]". Persona lives in agent_configurations.persona (→ snapshot), not in L4 — the two are still unbridged. |
| Upstreams | agent-worker (AssembleContext/ValidatePrompt/IngestEvent, Connect-HTTP — now live). The SPA calls the proxied services. |
| Downstreams | Postgres + pgvector (vector store, guardrail store, session store, immutable_layers), Redis (resolver cache + compaction streams). Proxies to agent-orchestrator for the SPA's Lifecycle/ModelRegistry calls. |
| Edge stubs | L4 writer missing (no admin RPC / UI / seed). |
3. Agent Worker — Python LangGraph + Claude SDK (agent-worker)
| |
|---|
| What it does | The reasoning loop, stateless, re-pulls context each turn: assemble_context → validate_prompt → [PEP pre-LLM] → call_llm → parse_response → [PEP per-tool] → execute_tools → ingest_result → loop. Hosts two executors behind the same gRPC surface: the default LangGraph executor and a ClaudeSDKExecutor, selected per-session via snapshot.runtime. |
| Key files | services/agent-worker/src/agent_worker/graph/nodes.py · .../executor/claude_sdk_executor.py · .../context/client.py · .../mcp/client.py · .../config.py |
| Status | 🟢 LIVE as a single-turn relay and as the step-executor for the Go-side autonomous PlanLoop (worker supplies structured PLAN-step decomposition, #1614/#1617). Second runtime: ClaudeSDKExecutor selectable per-agent via snapshot.runtime (ADR-0020); local qwen/llama/mistral route to Ollama. |
| Stubs / gaps in it | 🟢 Context-Engine client is now REAL (context/client.py, Connect-HTTP, #2030/#2031/#2034) — the base's headline stub is gone; it fails open with an integrity_check_skipped audit if the CE is unreachable. 🔵 mcp_stub_mode code-default is TRUE (config.py:87) but the dev deploy overrides it false (WORKER_MCP_STUB_MODE=false, docker-compose.dev.yml:885) → tools execute on beta-dev. 🟡 No planner/reflect nodes in the worker — plan/terminate logic is orchestrator-side. |
| Upstreams | agent-orchestrator (ExecuteStep). |
| Downstreams | governance GovernanceService (pre-LLM + per-tool) · AI-gateway / Ollama for call_llm · context-engine for assemble/validate (now live) · Team Gateway /mcp/team/{unit} for tools (dev). |
| Edge stubs | Residual: on langgraph the assembled prompt carries L1–L3 + RAG but no L4 persona (see #2/#8). MCP tool execution is live in dev only. |
4. Governance — PDP / PEP (internal/governance)
| |
|---|
| What it does | PDP = governance.Engine (cascade evaluator: org/pillar/team/member policies + 5-tier clearance). PEP = the enforcement gate that calls it. Model: embedded PDP + gateway backstop over the same engine, fail-closed, read-through (no-cache) policy freshness. |
| Key files | internal/governance/engine.go · internal/governance/receipt_{emit,compute,replay}.go · internal/governance/receiptverify/verify.go · internal/context/guardrail/resolver.go · worker gates nodes.py |
| Status | 🟢 LIVE for llm_request + tool_call via the embedded worker gates. Action types: llm_request, tool_call, file_access, code_exec, external_api, a2a_egress. Turn-boundary guardrail re-pin LIVE (ResolveGuardrailLayers + monotonic version, adopt-tighten/block-drift). Decision-receipts LIVE (verifiable via cmd/verify-receipt). Under ADR-0024 the MCP tool path's PEP is now the gateway edge (not the old GovernanceService #288 residue, which was removed). |
| Stubs / gaps in it | 🟡 AI-gateway is still NOT a PEP → non-worker LLM egress has no governance backstop (embedded-only). 🟡 team model-allowlist is stored + read-filtered (internal/orgunit/model_allowlist_*) but not enforced at call time. |
| Upstreams | agent-worker (gRPC GovernanceService) · MCP edge gateway (CEL tool policies) · coordinators (delegation hop) · in-process Go callers (embedded). |
| Downstreams | Postgres policy store (cascade). Writes every verdict → Audit (#9). |
| Edge stubs | AI-gateway LLM-egress backstop still missing. |
| |
|---|
| What it does | Governed tool egress — how agents reach external tools. MCPServerService registry (team-scoped register/list/deregister/health) + the Go edge Team Gateway at /mcp/team/{unit}: the single Policy Enforcement Point for every MCP tool call, with ordered fail-closed gates (clearance floor → agent allowlist → tiered CEL tool policy → OAuth terminate → egress-guarded forward). See the dedicated Governed MCP & the Team Gateway KT. |
| Key files | internal/gateway/mcpproxy/{dispatch,edge_policy,upstream_proxy}.go · internal/mcp/egress/{guard,deny,allowlist}.go · internal/mcpserver/ · internal/auth/oauth/{agent_token,issuer_routes,token_endpoint}.go |
| Status | 🟢 Code LIVE — governed edge PEP + egress/SSRF guard, all gates fail-closed. Architecture ACCEPTED: ADR-0024 (MCP governance canonical surfaces, 9 decisions) + ADR-0025 (egress / private upstreams). 🔵 On beta-dev, tools execute (WORKER_MCP_STUB_MODE=false). 🟠 Ships feature-OFF — MCP_GATEWAY_ENABLED default false, per-tenant opt-in. |
| Stubs / gaps in it | 🟠 ContextForge services/mcp-proxy REMOVED (#1909/#1912) — idempotency folded into the Go edge (fails open). 🟠 Prod parked on #1864: no real platform signing key / JWKS yet, so Hop-A minting + external-agent /oauth/token are exercisable on dev but parked for prod. 🟡 Per-tool full inputSchema passthrough is permissive (arg validation deferred to CEL + upstream). |
| Upstreams | agent-worker (execute_tools → /mcp/team/{unit}) — dev · external BYO agents via /oauth/token (dev) · SPA registry UI (register/list/health). |
| Downstreams | External tenant-registered MCP servers (via egress guard) · governance (per-tool CEL Check) · Vault (Hop-B upstream creds) · idempotency dedup store. |
| Edge stubs | Prod enablement + a real issuer land with #1864 / the GCP migration. |
6. AI Gateway / BYOK / LLM (internal/aigateway)
| |
|---|
| What it does | LLM egress. Resolves per-tenant BYOK keys, enforces cost/budget, runs input/output guardrails. |
| Key files | internal/aigateway/{handler,provider,providerkey,budget,guardrail,catalog} · internal/billing/service.go |
| Status | 🟢 LIVE for LLM calls, including tenant BYOK provider-key write (vault-backed). Per-agent llm_usage_events attribution is built (agent_id on records, consumed by billing + budget). Dev: keyless Ollama gemma3:1b + a LiteLLM Anthropic facade for claude_sdk agents. |
| Stubs / gaps in it | 🟡 Still NOT a governance PEP (LLM backstop designed under ADR-0021, not built). 🟡 model allowlist stored (team_model_allowlists) but not enforced at aigateway call time. 🟠 Sourcing modes A/B/C/D still largely paper; multi-provider BYOK + Ollama available on the worker. |
| Upstreams | agent-worker (call_llm). |
| Downstreams | External LLM providers / Ollama · Vault (BYOK keys). |
| Edge stubs | On dev, cloud models need a tenant BYOK key in Vault; keyless gemma + the LiteLLM facade are the default working paths. |
7. Quad (internal/quad)
| |
|---|
| What it does | The per-user chat/orchestrator surface that relays to deployed agents, including the deterministic @mention delegate. |
| Key files | internal/quad/{connect_adapter,submit,session_store,deduper,store}.go · internal/runtime/server/lifecycle_mention_delegate.go · mig 115_user_quad_agents |
| Status | 🟡 Quad is a thread/Connect-RPC adapter, not an agent loop — but the per-user Quad is seeded as a real member-owned agent_configurations row (mig 115: idempotent user→Quad-agent, RLS-scoped, GetOrCreateUserQuadAgent, clearance = InferAgentClearance(user), delegate_to_agent granted). Live child-result relay bubbling merged (#1572). |
| Stubs / missing | 🟡 No quad_copilot kind string · sourcing modes paper-only · reasoning_chain_ref not on the Entry proto. |
| Upstreams | SPA (QuadService via /rpc). |
| Downstreams | orchestrator LifecycleService (SendMessage) · mention delegate → subagent Coordinator. |
8. Persona / Competency Profiles
| |
|---|
| What it does | Defines an agent's role + capabilities. |
| Key files / refs | agent_configurations.persona (TEXT) · internal/runtime/session/initializer.go · services/agent-worker/.../executor/claude_sdk_executor.py · competency catalogue issue #554 |
| Status | 🟡 Persona is a free-text TEXT field. It reaches the model on the claude_sdk runtime (initializer.go reads agent_configurations.persona → SessionSnapshot.Persona → claude_sdk_executor.py:488 system_prompt=persona). On the default langgraph runtime it is still NOT injected — assembly now runs (see #3) but L4 is empty, so persona is absent from the langgraph prompt. The FR-26 competency-profile bundle still does not exist. |
| Stubs / gaps | 🟡 No persona → immutable_layers L4 sync (no L4 writer). 🟡 No competency_profiles table; "Software Engineer L3" isn't a single configurable entity. 🟠 #554 open/awaiting-approval. |
| Upstreams | Create-agent wizard / RegisterAgent. |
| Downstreams | Flows into the session snapshot → model on claude_sdk; should also flow into Context Engine L4 for langgraph — but doesn't yet. |
9. Audit / Decision Receipts
| |
|---|
| What it does | Append-only record of every governed action (verdicts, LLM/tool calls, delegation, governed decisions). The audit-as-source-control tenet. |
| Key files | internal/runtime/audit/pgstore.go · internal/governance/receipt_{emit,compute,replay}.go · internal/governance/receiptverify/verify.go · cmd/verify-receipt |
| Status | 🟢 Audit chain live. 🟢 Governed decisions persist and are queryable — governance_approvals carries status/resolved_at/resolved_by/metadata JSONB; ListRecentDecisions projects metadata.summary. 🟢 Step-up second-factor enforcement on high-severity RecordDecision (internal/gateway/stepup.go, X-Reverification, fail-closed). 🟢 Decision-receipt verifier live (cmd/verify-receipt). 🟡 Per-action policy-bundle-version receipts (#1184) remain PARTIAL — the predicate is computed in every engine.Check with the bundle captured as a content hash and bytes persisted; but the generic MCP pre-tool hook still writes receipt-less audit events, so ordinary governed tool calls don't yet all get a verifiable receipt row. |
| Upstreams | governance PDP (verdicts) · worker · MCP edge gateway (tool-call audit) · coordinators · approval.Service. |
| Downstreams | Postgres audit log (append-only) + governance_approvals. |
| |
|---|
| Postgres + pgvector | Sessions, policies, audit, org-units, agents/agent_configurations, immutable_layers, knowledge/RAG sources + vectors, MCP registry + bindings. RLS-enforced tenant isolation. Schema at mig 168 (from the base's ~126). Notable adds since base: 155_external_agent_clients, 158_agent_configurations_runtime_external, 160_session_events, 161_orgv24_multiparent_resolver_ga, 162_mcp_min_clearance, 163_mcp_clearance_floor_denied_audit, 164_mcp_binding_kind, 165_mcp_subtree_withheld_audit, 168_mcp_egress_private_upstream. Still no competency_profiles. |
| Redis | Caches (org-v2.4 resolver effective-set, ~45s TTL) + Redis Streams (compaction producer). |
| Vault / secret store | BYOK LLM keys + MCP Hop-B upstream creds. Tenant BYOK provider-key write live. |
| Ollama | Local models; gemma3:1b keyless on dev; qwen/llama/mistral routable. |
11. Approvals / Step-up / Governed org-change
| |
|---|
| What it does | Resource-scoped (sessionless) governance approvals that gate org-restructure, W1-C governed-unit actions, and MCP HITL (-32001). Provides the approval backend the autonomous-loop interrupter parks against, plus step-up second-factor on high-severity decisions. |
| Key files | internal/runtime/approval/ · internal/orgunit/governed_service.go · internal/gateway/stepup.go · migs 038/125 |
| Status | 🟢 LIVE — Request → park → RecordDecision(APPROVED/DENIED) → Resume/Terminate; resource-scoped so it works without a live session; ListRecentDecisions + metadata.summary; approval_id recovery from enveloped bodies. |
| Stubs / gaps | 🟡 Step-up verifier is currently HeaderPresenceVerifier (real Clerk re-verify deferred to W4). 🟡 Per-action tool-call receipt emission still unwired for ordinary governed actions (#1184 — see #9). |
| Upstreams | autoloop interrupter · governed org-change · MCP edge (requires-approval) · SPA approvals UI. |
| Downstreams | governance PDP · Postgres (governance_approvals) · Audit (#9). |
12. Claude SDK Runtime — 2nd worker type
| |
|---|
| What it does | A second agent executor selectable per-agent (langgraph | claude_sdk) behind the same worker gRPC surface. Records the scoped stateless-worker tenet exception (one claude CLI subprocess per active SDK session). |
| Key files | services/agent-worker/src/agent_worker/executor/claude_sdk_executor.py · mig 158_agent_configurations_runtime_external · LiteLLM Anthropic proxy · ADR-0020 |
| Status | 🔵 DEV-ONLY. Per-agent select via snapshot.runtime. Persona reaches the model on this path (system_prompt=snapshot.persona, claude_sdk_executor.py:488) — the one runtime where persona is live end-to-end. |
| Stubs / gaps | 🟠 Needs a tenant BYOK Anthropic key in Vault for cloud; dev uses the LiteLLM facade. Not the default runtime; bigger memory footprint (subprocess per session). |
| Upstreams / Downstreams | Same as #3 (orchestrator ExecuteStep → executor → AI-gateway/LiteLLM → Anthropic). |
13. Org Model v2.4
| |
|---|
| What it does | Unit relations / manager / admins / offers model plus the multi-parent resolver + RPC read path that the /rpc gateway proxies to the SPA, and which the MCP snapshot resolution depends on. |
| Key files | internal/orgunit/resolver.go · migs 125_orgv24_wave1a, 161_orgv24_multiparent_resolver_ga, 126_mcp_server_units, 164_mcp_binding_kind |
| Status | 🟢 Multi-parent resolver GA (#1981; FlagMultiParentResolver compile-default TRUE, mig 161). Resolver READ RPCs live. Clone-on-get/put + invalidation cache fixes (#2024/#2026/#2028). |
| Stubs / gaps | 🟡 Write path flows through the governed-approvals subsystem (#11); later waves pending. ⚠️ Flipping the resolver kill-switch to v2.3 yields an empty MCP set (governed MCP invisible) — do not flip without knowing this. |
| Upstreams | SPA via context-engine /rpc (OrgUnitService) · Session Initializer (MCP snapshot resolution). |
| Downstreams | Postgres (unit tables) · governance (cascade scope) · mcp_server_units binding · MCP edge gateway. |
Reflects upsquad-core@9858af5f (2026-07-24). If you close #1864, flip MCP_GATEWAY_ENABLED, add an L4 persona writer, or enable T4 in prod, update the affected component rows and the Changelog table.