Skip to main content

HLD-C: RAG-as-Attachable-Entity — tag a source to pillar/team, inherit like MCP

FieldValue
StatusDraft — awaiting founder approval
Version1.0
Date2026-06-15
AuthorPrincipal Technical Architect Agent
Issue#1377 (HLD-C)
SupersedesHLD-A (docs/hld/knowledge-service.md, PR #1315) Design Call D3 — single owning_scope
Parent PRD#1206 (Team as Capability Container)
Proven precedent (mirror exactly)org_unit_tools (migration 064) + ToolVisibilityResolver.EffectiveTools (internal/orgunit/tool_visibility.go, tool_store.go)
Touches (already merged)migrations 098–100, 105; internal/knowledge/*; internal/context/retrieval/{vector,db,service}.go
Implementation gateFounder approval on issue #1377 before any code. Two open design calls (DC-1 migration, DC-2 unification) need an explicit pick.

1. Executive summary​

Today a knowledge source has one home scope (knowledge_sources.owning_scope ∈ {org,pillar,team} + unit_id, HLD-A D3, 1:1). The founder direction (2026-06-15) is that RAG should behave exactly like MCP/tools: a source is a standalone, reusable entity that you tag (attach) to one-or-many pillars/teams, and an agent's effective knowledge is computed by the same org→pillar→team inheritance walk the tool model already ships.

This HLD makes knowledge_sources scope-agnostic (org-owned catalog) and introduces a knowledge_source_units binding table that is a near-clone of org_unit_tools. Effective-source resolution mirrors EffectiveTools line-for-line (direct ∪ inherited-down ∪ shared-siblings, dedup direct > inherited > sibling). The retrieval path stops relying on the chunk's owning_scope for knowledge chunks and instead filters to the resolved effective source-id set; the per-agent disable list (agent_knowledge_overrides) composes on top, narrowing the set exactly as before.

We recommend a replace migration (drop owning_scope/unit_id, backfill one binding per existing source) because there is no real tenant RAG data in prod (beta-dev seed only) and the additive path leaves two competing visibility mechanisms in the code forever. We recommend a parallel knowledge_source_units table now (clarity + velocity, exact org_unit_tools shape) and flag the shared unit_attachments primitive as a deliberate tripwire once Track E (MCP gateway inheritance) gives us a third consumer.

2. Where this sits in the merged substrate (verified by inspection)​

ConcernWhat exists todayFile / migration
Tool binding + inheritance (the model to mirror)org_unit_tools(unit_id, tool_id, visibility, config_json, configured_by, soft-delete, unique-active, FORCE RLS org-isolation)migrations/064_org_unit_tools.up.sql
Effective-set resolver (the algorithm to mirror)EffectiveTools(member) = direct ∪ ancestor inherit_down ∪ sibling shared_siblings, dedup direct>inherited>sibling; ancestor walk via org_units.ancestor_pathinternal/orgunit/tool_visibility.go, tool_store.go (UnitIDsForMember, AncestorIDsForUnits, SiblingIDsForUnits, ListToolBindingsForUnits)
Source entity (1:1 today)knowledge_sources(owning_scope, unit_id, …), scope-visibility RLS policy mirroring rag_chunksmigrations/098_knowledge_sources.up.sql
Chunk ↔ source linkrag_chunks.source_id (nullable; NULL = conversation chunk)migrations/099_rag_chunks_source_id.up.sql
Per-agent disableagent_knowledge_overrides(agent_id, source_id), presence = disabledmigrations/100_agent_knowledge_overrides.up.sql
Refresh/criticality (orthogonal)last_refreshed_at, refresh_interval_seconds, criticalitymigrations/105_knowledge_source_refresh.up.sql
Chunk visibility at ingestIngestDocument stamps each chunk with the source's owning_scope → pillar_id/team_id (ingest.go §"Map the source's owning scope")internal/knowledge/ingest.go
Retrieval visibilityRLS on rag_chunks + vector ANN pre-filter e.pillar_id = GUC OR e.pillar_id IS NULL; per-agent NOT EXISTS agent_knowledge_overridesinternal/context/retrieval/vector.go
Scope path / GUCsscope.Path{Org,Pillar,Team,Agent}; GUCs app.org_id/pillar_id/team_id/agent_idinternal/context/scope/path.go, retrieval/db.go

Latest migration on main is 105. This HLD claims 106–107 at implementation.

The structural mismatch this HLD resolves: today, a chunk's visibility is a property of the chunk (its stamped owning_scope/pillar_id/team_id, enforced by RLS). In the MCP model, visibility is a property of the binding (which units the source is attached to + the inheritance mode). A source attached to three teams cannot be expressed by one owning_scope on the chunk — hence D3's 1:1 limitation. We move the visibility decision off the chunk and onto the binding-resolver.

3. Entity model​

3.1 knowledge_sources becomes scope-agnostic​

owning_scope and unit_id are removed (DC-1 = replace). A source is an org-owned catalog entry: who-can-see-it is decided entirely by its bindings.

knowledge_sources (after migration)
├── id, org_id, name, description
├── src_type, sync_status, doc_count, chunk_count
├── created_by, created_at, updated_at, deleted_at
├── last_refreshed_at, refresh_interval_seconds, criticality   (unchanged, #1367)
└── (owning_scope, unit_id  ──  DROPPED)

RLS on knowledge_sources simplifies to org-isolation only (org_id = app.org_id), exactly like org_unit_tools. The catalog is visible org-wide for management; retrieval visibility is the resolved effective set (§5), not the table's RLS. The knowledge_sources_unit_scope_chk CHECK and the scope clause of knowledge_sources_scope_isolation are dropped.

3.2 knowledge_source_units (new) — the binding table, a clone of org_unit_tools​

CREATE TABLE knowledge_source_units (
    id            UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    org_id        UUID NOT NULL REFERENCES organizations(id)     ON DELETE RESTRICT,
    source_id     UUID NOT NULL REFERENCES knowledge_sources(id) ON DELETE CASCADE,
    unit_id       UUID NOT NULL REFERENCES org_units(id)         ON DELETE CASCADE,
    visibility    TEXT NOT NULL
                  CHECK (visibility IN ('inherit_down','private','shared_siblings')),
    configured_by UUID NOT NULL REFERENCES members(id) ON DELETE RESTRICT,
    created_at    TIMESTAMPTZ NOT NULL DEFAULT now(),
    updated_at    TIMESTAMPTZ NOT NULL DEFAULT now(),
    deleted_at    TIMESTAMPTZ
);

-- One active binding per (source, unit). Re-attach after detach is OK. Mirrors
-- uniq_org_unit_tools_unit_tool_active.
CREATE UNIQUE INDEX uniq_ksu_source_unit_active
    ON knowledge_source_units(source_id, unit_id) WHERE deleted_at IS NULL;
CREATE INDEX idx_ksu_unit   ON knowledge_source_units(unit_id)   WHERE deleted_at IS NULL;
CREATE INDEX idx_ksu_source ON knowledge_source_units(source_id) WHERE deleted_at IS NULL;
CREATE INDEX idx_ksu_org    ON knowledge_source_units(org_id)    WHERE deleted_at IS NULL;

ALTER TABLE knowledge_source_units ENABLE ROW LEVEL SECURITY;
ALTER TABLE knowledge_source_units FORCE  ROW LEVEL SECURITY;
CREATE POLICY ksu_org_isolation ON knowledge_source_units
    FOR ALL
    USING      (org_id = current_setting('app.org_id')::uuid)
    WITH CHECK (org_id = current_setting('app.org_id')::uuid);

Differences from org_unit_tools, all by design:

  • No config_json. A tool binding carries a per-unit config (scopes/credentials) that must subset-validate against ancestors → the whole tool_config_conflicts override workflow. A source has no per-unit config: attaching the same corpus to a child can never "exceed" a parent ceiling. So we drop config_json and the entire conflict/override machinery (tool_config_conflicts, SubsetViolationError, RequestOverride/ResolveOverride). This is the single biggest simplification vs. the tool model — call it out in review so nobody ports the conflict code.
  • tool_id → source_id (a real FK with ON DELETE CASCADE, unlike tool_id which is an opaque UUID). Deleting a source removes its bindings automatically.
  • Same visibility enum, same constants (inherit_down|private|shared_siblings), same unique-active index, same FORCE RLS org-isolation. M:N is now first-class: a source may have N binding rows, one per unit it is tagged to — exactly the thing D3 deferred.

3.3 rag_chunks.source_id — unchanged shape, changed meaning​

The column stays (migration 099). What changes is what gets stamped at ingest: knowledge chunks become org-level at rest (owning_scope='org', pillar_id=NULL, team_id=NULL) so the chunk's RLS no longer second-guesses the binding layer (§5.3). The source_id link is what the resolver keys on.

3.4 agent_knowledge_overrides — unchanged​

Per-agent disable composes by narrowing the resolved effective set (§6). No schema change.

3.5 Org-level sources and the org-root unit​

An org-scoped source today (owning_scope='org', unit_id=NULL) must become a binding. The clean mapping, identical to "an org-wide tool": attach it to the org-root org_unit with visibility='inherit_down', so every descendant pillar/team inherits it. org_units (migration 060 dual-write) has a tenant-root node and ancestor_path includes it, so the existing ancestor walk picks it up for free. Sub-decision for the founder inside DC-1: confirm the org-root unit exists for every tenant; if a tenant has no root node, the backfill must create one (one row, idempotent). The implementer verifies this against org_units before writing migration 106.

4. Effective-source resolver (mirror of EffectiveTools)​

New file internal/knowledge/source_visibility.go + store methods in internal/knowledge/source_units_store.go. The shape is a direct copy of tool_visibility.go:

const (
    EffSourceDirect    = "direct"
    EffSourceInherited = "inherited"
    EffSourceSibling   = "sibling"
)

type EffectiveSource struct {
    Binding      SourceUnitRow // source_id, unit_id, visibility, …
    Source       string        // direct | inherited | sibling
    SourceUnitID string
}

// EffectiveSources returns the resolved source set for a member/agent.
// De-dup rule: direct > inherited > sibling (same as EffectiveTools).
func (r *SourceVisibilityResolver) EffectiveSources(ctx context.Context, memberID string) ([]EffectiveSource, error) {
    memberUnits := store.UnitIDsForMember(ctx, tx, memberID)              // reuse orgunit query shape
    direct      := store.ListSourceBindingsForUnits(ctx, tx, memberUnits)
    ancestorIDs := store.AncestorIDsForUnits(ctx, tx, memberUnits)        // org_units.ancestor_path walk
    inherited   := store.ListSourceBindingsForUnits(ctx, tx, ancestorIDs) // keep visibility='inherit_down'
    siblingIDs  := store.SiblingIDsForUnits(ctx, tx, memberUnits)
    sibling     := store.ListSourceBindingsForUnits(ctx, tx, siblingIDs)  // keep visibility='shared_siblings'
    // identical de-dup loop to EffectiveTools, keyed on source_id
}

AncestorIDsForUnits/SiblingIDsForUnits/UnitIDsForMember already exist on internal/orgunit.Store and operate on org_units — knowledge reuses the same org-tree queries (either by depending on orgunit.Store for the topology helpers, or by lifting the three read-only queries verbatim; recommend depending on orgunit.Store to avoid drift — DC-2 leans the same way). Only the binding-list query (ListSourceBindingsForUnits) is knowledge-specific and trivially mirrors ListToolBindingsForUnits.

EffectiveSourcesForScope(scope.Path) is the retrieval-side entry point: it resolves from the path's AgentID (falling back to TeamID/PillarID membership) so a non-agent caller (team chat) also gets the right set.

Resolver caching. Mirror scope.Resolver's 15-minute Redis cache keyed on the scope's RedisPrefix() (bindings change rarely). Invalidate the org's effective-source keys on any AttachSourceToUnit/DetachSourceFromUnit. Emit knowledge_source_resolver_duration_seconds{depth_bucket} exactly like tool_resolver_duration_seconds.

5. Retrieval integration (the load-bearing change)​

5.1 From "RLS decides" to "resolver decides" for knowledge chunks​

Today a knowledge chunk is visible because its stamped pillar_id/team_id matches the caller's GUCs (RLS) and survives the vector ANN pre-filter. That is the 1:1 model and cannot express a source shared across three teams. New model:

  1. Knowledge chunks are stored org-level (owning_scope='org', pillar_id=NULL, team_id=NULL), so RLS treats them as org-wide — RLS no longer scopes them.
  2. The effective source-id set (§4) is the authoritative knowledge ACL, applied as a filter in every searcher.
  3. Conversation chunks (source_id IS NULL) are untouched: their existing per-scope RLS + ANN pre-filter still governs them.

5.2 Query shape​

The retrieval Service.Search resolves the effective set once and passes the source-id slice to all three searchers. Each searcher's predicate gains a single clause. For vector.go:

SELECT c.id::text, c.content, c.metadata,
       1 - (e.embedding <=> $1::vector) AS similarity,
       c.source_doc_id::text
FROM context_embeddings e
JOIN rag_chunks c ON c.id = e.chunk_id
WHERE e.org_id = current_setting('app.org_id')::uuid
  AND (e.pillar_id = NULLIF(current_setting('app.pillar_id', true), '')::uuid
       OR e.pillar_id IS NULL)                                   -- conversation-chunk pre-filter (unchanged)
  AND (
        c.source_id IS NULL                                      -- conversation chunk: governed by RLS above
     OR c.source_id = ANY($3::uuid[])                            -- knowledge chunk: must be in effective set
      )
  AND NOT EXISTS (                                               -- per-agent disable (unchanged)
        SELECT 1 FROM agent_knowledge_overrides ako
        WHERE ako.source_id = c.source_id
          AND ako.agent_id = NULLIF(current_setting('app.agent_id', true), '')::uuid)
ORDER BY e.embedding <=> $1::vector
LIMIT $2

$3 is the resolved []source_id. When the set is empty, c.source_id = ANY('{}') is false, so only conversation chunks return — correct (the agent's units have no sources tagged). The same AND (c.source_id IS NULL OR c.source_id = ANY($k)) clause is added to bm25.go and recency.go.

Note the per-agent disable NOT EXISTS is retained as defense-in-depth even though the resolver could subtract disabled ids; keeping it in SQL means a disable takes effect without a resolver-cache flush.

5.3 Ingest change​

ingest.go stops mapping src.OwningScope → pillar_id/team_id. It always passes OwningScope:"org", PillarID:"", TeamID:"" to PersistChunks, keeping only SourceID/SourceDocID. This is the one behavioural change in the ingest path; the chunker/embedder/worker are untouched. (The embedding.ProcessParams.OwningScope/PillarID/TeamID plumbing stays — conversation ingest still uses it.)

5.4 pgvector / HNSW implications​

  • The per-pillar partial HNSW indexes are keyed on (org_id, pillar_id). Knowledge chunks now have pillar_id=NULL, so they live in the global HNSW fallback partition, not per-pillar ones. At MVP volumes this is fine; flag for the perf sandbox (#616) that as a tenant's org-wide knowledge grows, the global partition carries all of it. A dedicated WHERE source_id IS NOT NULL partial HNSW is the escape hatch if global-fallback recall degrades — deferred, not built.
  • The source_id = ANY($k) filter is a post-ANN filter on the joined rag_chunks (the existing idx_chunks_source from migration 099 supports it). Post-filtering after ORDER BY … LIMIT k can under-return when an out-of-set source dominates top-k. HLD-A already mandates over-fetch (LIMIT k * over_fetch_factor, default 3, then filter, then truncate) for the per-agent disable case; the effective-source filter rides the same over-fetch. FW owns the constant; no new index for v1.

6. Per-agent disable composition​

Unchanged semantics, now expressed as effective(agent) = resolve(units) \ disabled(agent). The retrieval SQL keeps the NOT EXISTS agent_knowledge_overrides clause (§5.2), so disable is applied at the DB layer regardless of resolver cache state. SetAgentSourceEnabled is unchanged. The override can only narrow the resolved set — never widen it — matching migration 100's documented invariant.

7. RPC surface delta​

service KnowledgeService {
  // --- unchanged ---
  rpc ListSources       (...)  // now lists the org-owned catalog (no scope filter semantics)
  rpc GetSource         (...)
  rpc UpdateSource      (...)
  rpc RefreshSource     (...)
  rpc DeleteSource      (...)  // CASCADE also drops knowledge_source_units bindings
  rpc GetSourceStatus   (...)
  rpc IngestDocument    (...)
  rpc SetAgentSourceEnabled (...)

  // --- CHANGED ---
  rpc CreateSource      (CreateSourceRequest) returns (CreateSourceResponse); // drops owning_scope + unit_id

  // --- NEW (mirror the orgunit tool attach RPCs) ---
  rpc AttachSourceToUnit   (AttachSourceToUnitRequest)   returns (AttachSourceToUnitResponse);
  rpc DetachSourceFromUnit (DetachSourceFromUnitRequest) returns (DetachSourceFromUnitResponse);
  rpc ListEffectiveSources (ListEffectiveSourcesRequest) returns (ListEffectiveSourcesResponse); // by unit | member | agent
  rpc ListSourceUnits      (ListSourceUnitsRequest)      returns (ListSourceUnitsResponse);      // a source's bindings
}

message CreateSourceRequest {            // owning_scope (=1) and unit_id (=2) REMOVED
  string name = 3;                       // field numbers preserved for the survivors
  string description = 4;
  string src_type = 5;
  int32  refresh_interval_seconds = 6;
  string criticality = 7;
}

message AttachSourceToUnitRequest {
  string source_id  = 1;
  string unit_id    = 2;
  string visibility = 3;                 // inherit_down | private | shared_siblings
}
message AttachSourceToUnitResponse { SourceUnitBinding binding = 1; }

message DetachSourceFromUnitRequest { string source_id = 1; string unit_id = 2; }
message DetachSourceFromUnitResponse { bool detached = 1; }

message ListEffectiveSourcesRequest {    // exactly one of the selectors
  string unit_id   = 1;
  string member_id = 2;
  string agent_id  = 3;
}
message ListEffectiveSourcesResponse {
  repeated EffectiveSourceEntry entries = 1; // {Source, provenance: direct|inherited|sibling, source_unit_id}
}

message SourceUnitBinding {
  string source_id = 1; string unit_id = 2; string visibility = 3;
  string configured_by = 4; google.protobuf.Timestamp created_at = 5;
}

The Source message drops owning_scope (=4) and unit_id (=5). This is a breaking proto change — the Proto Lint & Breaking-Change CI check will flag it. Because RAG is pre-GA and no external client consumes it, the recommendation is to take the break (cleaner than reserving dead fields), reserve the field numbers (reserved 4, 5; reserved "owning_scope", "unit_id";), and have DevOps allow the one-time breaking diff on the proto-CI gate for this PR. This is the kind of change that needs founder/DevOps sign-off (new breaking-change allowance) — listed as an implementation gate.

Authorization. AttachSourceToUnit/DetachSourceFromUnit reuse the clearanceManage (L3) ∩ knowledge.manage gate, plus a scope check that the caller's scope covers unit_id (RLS org-isolation + service-layer unit-membership check). No subset/override workflow (§3.2). ListEffectiveSources is clearanceRead (L1).

8. Migration decision (DC-1) — REPLACE, with rationale​

OptionWhat it meansVerdict
(a) REPLACE (recommended)Drop owning_scope/unit_id; backfill exactly one binding per source; normalize knowledge chunks to org-level; visibility lives only in bindings.Recommend. One visibility mechanism, true MCP parity, no dead columns. Cheap because there is no real prod RAG data.
(b) ADDITIVEKeep owning_scope as the source's "home" + add bindings for extra attachments.Reject for MVP. Two competing truths (home vs bindings) the resolver and retrieval must reconcile forever; the chunk-stamping ambiguity (home scope vs org-level) persists; it is the exact "shelfware-adjacent" half-migration we keep paying for. Only justified if a prod tenant had irreplaceable scoped RAG — they don't.

8.1 Replace migration steps (claim 106–107 at impl)​

106 — knowledge_source_units create + backfill (one tx):

  1. CREATE TABLE knowledge_source_units … + indexes + FORCE RLS (§3.2).
  2. Ensure each org has a root org_unit (idempotent; verify against migration 060 first — DC-1 sub-decision §3.5).
  3. Backfill one binding per non-deleted source: 
    INSERT INTO knowledge_source_units (org_id, source_id, unit_id, visibility, configured_by)
    SELECT s.org_id, s.id,
           COALESCE(s.unit_id, root.id) AS unit_id,        -- org sources → org-root unit
           'inherit_down', s.created_by
    FROM knowledge_sources s
    JOIN org_units root ON root.org_id = s.org_id AND root.parent_id IS NULL
    WHERE s.deleted_at IS NULL;

107 — chunk normalization + drop legacy columns (one tx): 4. Normalize knowledge chunks to org-level so RLS stops scoping them:

UPDATE rag_chunks       SET owning_scope='org', pillar_id=NULL, team_id=NULL WHERE source_id IS NOT NULL;
UPDATE context_embeddings SET pillar_id=NULL WHERE chunk_id IN (SELECT id FROM rag_chunks WHERE source_id IS NOT NULL);
  1. Drop knowledge_sources_unit_scope_chk; drop the scope clause from knowledge_sources_scope_isolation (replace policy with plain org-isolation); ALTER TABLE knowledge_sources DROP COLUMN owning_scope, DROP COLUMN unit_id;.

Down migrations reverse (re-add columns nullable, rebuild scope policy, drop binding table). Beta-dev seed alternative: because the seed is disposable, an equally acceptable path is purge knowledge sources + chunks and re-seed instead of steps 3–4; recommend the in-place backfill so the migration is correct for any environment, not just the seed box.

9. Unification (DC-2) — parallel table now, shared primitive as a tripwire​

The binding+resolver pattern will soon have three consumers: tools (org_unit_tools, shipped), knowledge (knowledge_source_units, this HLD), and Track E MCP-gateway inheritance (same org→pillar→team walk). Two ways to land it:

OptionVerdict
(a) Parallel knowledge_source_units (recommended now)Mirror org_unit_tools exactly, drop the unused config_json/conflict machinery. Zero risk to the shipped tool path, fastest to MVP, each table keeps its own FKs (source_id cascade vs opaque tool_id).
(b) Generalize to a shared unit_attachments(attachable_type, attachable_id, unit_id, visibility, config_json) primitive nowReject now. Refactoring the shipped org_unit_tools + EffectiveTools under MVP time pressure risks a regression in the live tool path for no MVP-visible gain, and a polymorphic attachable_id loses the per-type FK cascade we want.

Recommendation: ship the parallel table. Tripwire: when Track E starts, we will have three identical resolvers — at that point raise an ADR to extract a generic AttachmentResolver (the topology walk + dedup, parameterised over a BindingStore interface that org_unit_tools, knowledge_source_units, and the gateway table each implement). The resolver algorithm is already identical; the refactor is "extract the loop, keep three thin stores." Track E should be designed against this seam (same visibility enum, same EffectiveX signature, same Redis-cache + duration-metric convention) so the eventual extraction is mechanical. This HLD names the seam; it does not build it.

This is also why §4 leans toward knowledge depending on orgunit.Store for the three topology queries (UnitIDsForMember/AncestorIDsForUnits/SiblingIDsForUnits) rather than copying them: it keeps the org-tree walk in one place ahead of the extraction.

10. Effort split​

RA1 — schema + bindings + resolver (backend, ~5 pts)

  • Migrations 106–107 (§8): create knowledge_source_units, backfill, chunk normalization, drop legacy columns + RLS simplification; matching downs + migration tests.
  • source_units_store.go (UpsertBinding, SoftDeleteBinding, ListSourceBindingsForUnit(s), ListBindingsForSource) mirroring tool_store.go.
  • source_visibility.go (EffectiveSources, EffectiveSourcesForScope) mirroring tool_visibility.go, Redis cache + duration metric, reusing orgunit.Store topology queries.
  • Shelfware gate: the resolver must have a production caller in the same PR — §11 (retrieval) is that caller; do not land RA1 without RA2 wired, or land them atomically.

RA2 — RPC + retrieval rewire (backend, ~5 pts, depends on RA1)

  • Proto delta (§7): drop owning_scope/unit_id from Source+CreateSource (reserve numbers), add 4 RPCs + messages; regenerate pkg/contextpb and caveman goldens (ui/caveman/gen/upsquad/knowledge) — both, per the proto-regen lesson.
  • AttachSourceToUnit/DetachSourceFromUnit/ListEffectiveSources/ListSourceUnits handlers, mounted on the gRPC mux (shelfware gate); CreateSource drops scope params.
  • Retrieval: resolve effective set in Service.Search/SearchWithQuality, thread []source_id into vector.go/bm25.go/recency.go (§5.2); over-fetch constant.
  • ingest.go: store knowledge chunks org-level (§5.3).
  • DevOps: one-time proto breaking-change allowance on the CI gate (founder/DevOps sign-off).

RA3 — connector orthogonality check (no code; verified in §11): confirm #1375 is unaffected.

Frontend (Track C) and QA tracked separately under the MVP plan.

11. Impact on in-flight tracks​

  • Track C (pillar RAG UI). Reframes cleanly: the pillar/team RAG panel stops calling CreateSource(owning_scope=pillar, unit_id=…) and instead (1) CreateSource(name, src_type, …) into the org catalog, then (2) AttachSourceToUnit(source_id, unit_id, visibility). The "RAG on this pillar/team" list becomes ListEffectiveSources(unit_id) with provenance badges (direct / inherited-from-org / shared-sibling) — the exact UX the tool-attach panel already uses. Founder-visible upside: the same source now shows up under multiple teams, which is the requested behaviour. Track C's open frontend issues need their acceptance criteria updated to the attach flow; flag to the frontend SME before they start.
  • Track D / #1375 (Web URL RAG connector + refresh worker). Orthogonal — confirmed. #1375 is about fetching content into a source (connector + SSRF-hardened refresh + criticality re-fetch). It writes chunks via the same ingest path and never reads owning_scope for routing; the only overlap is that ingested chunks are now stamped org-level (§5.3), which the connector inherits automatically by calling the same IngestDocument/PersistChunks path. No conflict; #1375 can proceed in parallel. (If #1375 lands before this HLD, its chunks get the old per-scope stamping and are caught by the 107 normalization UPDATE.)
  • Track E (MCP gateway inheritance). Becomes the third consumer of the shared resolver pattern (§9). Should be designed against the AttachmentResolver seam so the eventual extraction is mechanical.

12. Open design calls for the founder​

IDDecisionRecommendation
DC-1Migration: REPLACE owning_scope with bindings (drop columns, backfill one binding, normalize chunks org-level) vs ADDITIVE (keep home + add bindings). Sub-decision: org sources bind to the org-root org_unit — confirm every tenant has a root node (create in backfill if not).REPLACE. No real prod RAG data; additive leaves two competing visibility truths forever. (§8)
DC-2Parallel knowledge_source_units table now vs generalize org_unit_tools into a shared unit_attachments primitive.Parallel now, with an ADR-tripwire to extract a shared AttachmentResolver when Track E gives us the third consumer. (§9)
DC-3Proto break: drop owning_scope/unit_id from Source+CreateSource (reserve numbers) and grant a one-time proto breaking-change CI allowance.Take the break — RAG is pre-GA, no external consumer; cleaner than dead reserved fields carried indefinitely. Needs DevOps sign-off on the CI gate. (§7)

None of these block authoring; all three must be picked before RA1/RA2 are dispatched.

13. Out of scope​

  • Connector implementations (drive|github|notion|slack, web-URL #1375) — orthogonal, §11.
  • The shared unit_attachments primitive / AttachmentResolver extraction — deferred to the Track E ADR, §9.
  • Any change to the conversation-chunk path, the chunker, embedder, or embed worker.
  • Per-source dedicated HNSW partial index — deferred to the perf sandbox finding, §5.4.
  • A MoveSource/re-parent RPC — superseded; attach/detach replaces it.
Edit this page
Last updated on by upsquad-backend-sme[bot]