Seeking architecture feedback on a governed execution platform for AI workloads on the BEAM

Hi everyone,

I have been working on a large-scale, open-source platform for orchestrating and governing AI workloads on the BEAM (currently spanning various repositories under North-Shore-AI / nshkrdotcom).

Because AI agents are inherently non-deterministic and stateful, I wanted to build a system where governance precedes execution. Rather than letting LLMs directly execute code or API calls in a wild-west loop, every semantic turn, typed command, and external execution call passes through a strict, shared governance and durable-truth chain before it touches an outside system.

Much of the core idea, particularly the work in citadel came from work done by Mike Hostetler and his team. The jido_integration package was also created per Mike’s direction. I’m grateful to the Jido team for that opportunity.

In order to work on such a large scale endeavor, I’ve created a few helper projects to deal with cumbersome monorepos: blitz and weld. Blitz makes it easy to conduct monorepo-wide quality checks, and Weld will be used to assemble the various mix.exs projects in a monorepo into a single mix.exs project, combining the test suites and code, while also assembling shared documentation. Welding these monorepos will simplify publishing to Hex.

As the architecture progresses, I’d love to get some eyes on structural boundaries and OTP design philosophy from folks here who have distributed systems and BEAM expertise. I’m using this architecture to learn distributed systems design and implementation in practice. Constructive criticism is welcomed, especially since this is still early stage greenfield development.

High-Level Architecture

The platform is split into strict, structural ownership boundaries. No layer is allowed to bypass the layer below it.


[ Products / UIs / Workbenches ]

↓

[ Northbound Surface ]

* app_kit (Product-facing entry point)

* mezzanine (Universal business machine & operational-semantic engine)

↓

[ Brain Chain ] (Semantic → Typed → Governed)

* outer_brain (Semantic reasoning)

* citadel (Governance, policy, and intent shaping)

↓

[ The Spine ] (Durable Truth)

* jido_integration (Durable intake, auth, control plane, review truth)

↓

[ Execution Substrate ] (The "Hazmat Zone")

* execution_plane (Transport, placement, sandboxing, raw facts)

* Family Kits (REST, GraphQL, CLI/Subprocess, Python/Snakepit)

[ Foundational Substrate ] (Underlies everything)

* ground_plane (IDs, fences, leases, checkpoints, generic persistence)

Core BEAM / OTP Principles

To avoid turning this into an unmanageable “process soup,” I’ve adopted a few hard rules:

  1. Strict Structural Ownership: The Execution Substrate handles transport and placement, but carries zero durable business meaning. The Spine owns durable truth. The Brain shapes intent.

  2. Data & Contracts over Processes: Define stable data, pure compilers, reducers, and projectors first. Only wrap things in OTP processes (GenServers, Supervisors) where fault-tolerance, state recovery, or concurrency explicitly require them.

  3. Traceability by Design: Because debugging AI is like forensic analysis on a dream, lineage and audit trails are first-class. Every action maps back to a governed decision.

Where I’d Love Your Input

While I am confident in the functional separation, distributed orchestration introduces specific challenges where I’d value expert insight from the community. LLM’s can certainly provide guidance on how to approach the design, tooling, and testing, but nothing beats feedback from a distributed systems engineer.

1. Distributed State & “The Spine”

jido_integration acts as the durable truth layer (intake, auth, control plane).

2. The Execution Substrate (“Hazmat Zone”)

The execution_plane isolates dangerous work (subprocess execution, Python bridges, API calls). I want to ensure that failures here cascade cleanly up to the Spine without taking down the governance layers

3. “Pure Core” vs OTP Overhead

I’m actively trying to keep business logic (in mezzanine and citadel) as pure functions, pushing side-effects to the edges. However, orchestrating multi-step LLM reasoning loops often requires suspending and resuming state.

4. Distributed Test Harness

stack_lab will be used to enable testing distributed systems on my local development machine and test environments, not as a general framework but specifically for this stack.

5. General Architectural Critique

Looking at the layer diagram above, do you spot any glaring anti-patterns or bottlenecks typical to BEAM distributed systems?

If you are interested in poking around the code, the core infrastructure lives across repos at nshkrdotcom. (Specific repos of interest might be execution_plane, citadel, and jido_integration).

While I am ultimately interested in ensuring the correctness of the code in granular detail, currently it’s expected that the code will have anti-patterns. These will be naturally resolved over time. So, I’m aware of this, but not looking for specific feedback on the minutia at this time, unless broadly relevant.

Here’s a whitepaper that describes the architecture and roadmap in more detail. The code on github is a bit out of date, pending completion/push of current phase efforts.

My goal is to nail down the big picture architecture, after which I’m willing to revamp/rebuild any library as needed. I’m willing to revisit the overall architectural boundaries if there’s a good reason. Thanks in advance for any insights, critiques, or war stories you’re willing to share!

2 Likes

Brain/Spine Platform And Governed Memory Deep Dive 1 of 2

Date: 2026-04-21

Status: technical architecture note

AI governance platforms need a deterministic control substrate around
stochastic reasoning. This stack separates product surfaces, operational
workflow truth, semantic reasoning, authority compilation, lower acceptance,
and raw execution mechanics so AI-generated work can become governable
enterprise state instead of opaque agent output.

Memory belongs in that same control model. It is not a convenience feature or
an implementation detail of retrieval. Memory is a future control input. If a
remembered fragment can influence a mutation, approval, plan, tool call,
customer-facing answer, or operator decision, then that fragment participates
in the control plane and must carry provenance, scope, staleness, revocation,
and ownership.

Primary repository surfaces:

Executive Thesis

This architecture is best understood as a monolith-compatible, physically
bounded control platform for agentic execution.

It is not a conventional monolith where every subsystem can reach into the same
database and call any internal module. It is also not a premature microservice
mesh. The intended first deployment can run as a single node or single
application assembly, but the boundaries are real: public DTOs, repo/package
ownership, no-bypass gates, lease scopes, idempotency keys, authority packets,
fences, claim-checks, audit facts, and projection contracts. The system pays
the boundary tax early so that later distribution is a deployment change and
capacity exercise, not an architectural rewrite.

The platform separates four concerns that are often collapsed in simpler agent
systems:

  1. Product-facing application surfaces.
  2. Deterministic workflow and governance state.
  3. Semantic reasoning and context assembly.
  4. Lower execution, connector, and runtime facts.

The named stack components map to those concerns:

  • AppKit is the northbound product and operator boundary.
  • Mezzanine is the reusable deterministic operational substrate: packs,
    workflows, leases, decisions, evidence, audit, projections, archival, and
    operator/control-room state.
  • OuterBrain is the semantic runtime: context packs, prompts, semantic
    journals, strategy checkpoints, normalized semantic failures, and reply
    publication.
  • Citadel is the deterministic Brain kernel: structured host ingress,
    authority, policy compilation, local continuity, planning context, and
    Brain-to-Spine handoff.
  • Jido Integration is the Spine/lower acceptance gateway: durable intake,
    connector contracts, credential and auth lifecycle, lower facts,
    tenant-scoped substrate reads, review projections, and sidecar anchors.
  • Execution Plane is the hazmat runtime zone: raw mechanics, placement,
    process/HTTP/JSON-RPC lanes, attach/stream pressure, and sandbox-adjacent
    execution concerns.
  • Ground Plane supplies shared primitive contracts.
  • StackLab proves the assembled system and its failure modes.
  • AITrace can provide reasoning and execution observability, but is not truth.

Memory integration falls out of this architecture. Memory should not be
modeled as one central database and should not be assigned wholesale to a
single sidecar. In this stack, memory integration is a governed pair of planes:

  1. The recall plane: read-only, provenance-bearing context admission into
    semantic reasoning.
  2. The promotion plane: evidence-backed, reviewable, revocable projection of
    durable outcomes into future recall surfaces.

The recall plane is where memory may influence a model. The promotion plane is
where outcomes become eligible to inform later runs. Those planes must be
separate because an AI system that can freely read and write its own memory
inside the same reasoning loop has created an unreviewed control path.

The stack already contains the beginning of that separation:

  • OuterBrain owns semantic context assembly and explicitly rejects provider
    memory, RAG engines, and model-specific cognition as platform truth.
  • Citadel owns host-local authority, planning context, and an advisory memory
    record shape.
  • Mezzanine owns the pack-level context-source declarations, trusted context
    adapter activation gates, workflow/evidence boundaries, and tenant-scoped
    lower-read leases.
  • Jido Integration owns lower execution facts and a narrow derived-state
    attachment seam for higher-order sidecars.
  • AppKit owns northbound boundary enforcement so product apps cannot bypass
    those lower owners.

That shape is more sophisticated than assigning memory wholesale to one
sidecar. A memory projection sidecar can be valid, but it should be one
implementation of the promotion plane, not the definition of memory integration
for the platform.

Why Serious Engineers Will Challenge This

The strongest objection to this architecture is correct: it is complex.

A battle-tested distributed systems engineer will ask why this is not just:

  • one application database
  • one workflow engine
  • one vector store
  • one job queue
  • one agent runtime
  • one observability stack

That simpler architecture is often the right answer for a small product. It is
not obviously the right answer for an AI governance platform where stochastic
model output can trigger external side effects, update durable enterprise
state, cross tenant boundaries, enter audit trails, or become context for later
automated action.

The platform should accept complexity only where it protects an invariant.
Every boundary in this stack needs to justify itself by preserving at least one
of these properties:

  • ownership: one component owns a category of truth
  • replay: state can be reconstructed after restart without asking a live
    process what happened
  • idempotency: retries do not duplicate effects
  • tenancy: reads and writes carry explicit tenant and installation scope
  • authority: mutations are admitted only after policy compilation
  • review: human or automated gates are durable state transitions, not UI
    affordances
  • revocation: leases, credentials, adapters, and memory fragments can be
    invalidated
  • evidence: actions and projections can be traced back to source records
  • blast-radius control: raw runtime mechanics cannot leak upward as product
    truth

If a boundary does not preserve one of those properties, it is probably
ceremony. If it does, it is not ceremony; it is the point of the system.

The core design wager is that AI agents should be treated as volatile,
non-deterministic compute nodes inside a deterministic governance substrate.
The model may reason, summarize, plan, and propose. The deterministic platform
owns identity, leases, state transitions, idempotency, review state, audit,
projection, revocation, and replay. That split is the reason the architecture
looks heavier than a normal agent loop.

The complexity ledger should be explicit:

Seam Complexity Introduced Invariant Protected Proof Mechanism
Product → AppKit Product code must use DTO surfaces instead of direct lower imports Product apps cannot bypass governed writes, operator reads, reviews, or leased lower reads no-bypass scans, AppKit surface tests, product certification DTOs
AppKit → Mezzanine Northbound DTO mapping and backend behaviours Product surfaces do not own workflow, installation, review, lease, or lower-fact truth schema registry verification, bridge contract tests
Mezzanine → OuterBrain Context-source descriptors and semantic-runtime boundary Semantic reasoning can use context without owning durable workflow truth trusted adapter validation, context-pack provenance, semantic failure carriers
OuterBrain → Citadel Semantic output becomes structured Brain intent Model output cannot directly mutate lower systems authority compilation, semantic validation, normalized failure routing
Citadel → Spine Governed handoff packets, outbox, resync Brain intent crosses into lower truth once, idempotently, with receipts stable identifiers, downstream receipts, restart reconciliation
Spine → Execution Plane Lower runtime packetization and narrowed scopes Raw mechanics cannot learn product truth or widen authority lower contracts, runtime conformance, attach/stream pressure tests
Mezzanine → memory projection Promotion coordinator and evidence refs Memory cannot become an unreviewed second ledger promotion decisions, audit facts, revocation tests
Memory → OuterBrain Read-only recall request/response shape Retrieved context cannot mutate state or smuggle hidden authority read-only adapter contract, provenance checks, negative recall tests

This is also the standard for deleting complexity. If a seam cannot name an
invariant and a proof mechanism, it should be simplified or removed. The
architecture is defensible only if its boundaries are continuously executable,
not merely described.

Why Memory Needs Governance

Memory governance is not about making a vector database bureaucratic. It is
about recognizing what memory does in an agentic system.

In a conventional application, cached context is often an optimization. In an
AI execution platform, memory can become a hidden policy input. A remembered
preference can change a tool call. A recalled incident can bias a review. A
summarized customer conversation can influence a future deployment. A stale
fragment can cause a model to reapply an obsolete instruction. A cross-tenant
fragment can leak data. A provider-native memory feature can persist secrets
outside the platform’s audit and revocation model.

So the question is not “why govern memory?” The question is “when may a stored
or retrieved fact influence a future action?” The answer must be explicit.

A governed memory fragment needs to carry at least:

  • source: what truth record, evidence record, artifact, review, or semantic
    session produced it
  • scope: which tenant, installation, actor, subject, workflow, session, and
    policy context can see it
  • class: whether it is advisory, semantic-session state, durable derived
    projection, operator annotation, or incident/forensic evidence
  • staleness: whether the underlying source has changed
  • revocation: whether the fragment or source has been suppressed, redacted, or
    invalidated
  • admissibility: which policy or trusted adapter made it available for recall
  • conflict owner: which component wins if the memory disagrees with source
    truth

Without those fields, memory is an untyped side channel. A side channel that
influences future actions is a bypass.

Platform Plane Map

%%{init: {"theme": "base", "themeVariables": {"fontFamily": "Inter, ui-sans-serif, system-ui", "primaryColor": "#f8fafc", "primaryTextColor": "#0f172a", "primaryBorderColor": "#475569", "lineColor": "#475569", "secondaryColor": "#e0f2fe", "tertiaryColor": "#fef3c7"}}}%%
flowchart TB
  subgraph ProductPlane["Product and operator plane"]
    Product["Product app / operator shell"]
    AppKit["AppKit<br/>northbound governed surfaces"]
  end

  subgraph OperationalPlane["Deterministic&nbsp;operational&nbsp;plane"]
    Mezz["Mezzanine<br/>packs, workflow, leases, decisions, evidence, projections"]
  end

  subgraph BrainPlane["Brain&nbsp;plane"]
    OB["OuterBrain<br/>semantic runtime and context assembly"]
    Citadel["Citadel<br/>authority, policy, continuity, Brain handoff"]
  end

  subgraph SpinePlane["Spine&nbsp;and&nbsp;lower&nbsp;truth&nbsp;plane"]
    Spine["Jido Integration<br/>durable intake, connector contracts, lower facts"]
    Hazmat["Execution Plane<br/>raw runtime mechanics and placement"]
  end

  subgraph ProofPlane["Proof&nbsp;and&nbsp;observability"]
    StackLab["StackLab<br/>assembled proofs and drills"]
    Trace["AITrace / telemetry<br/>trace joins and execution cinema"]
  end

  Product --> AppKit
  AppKit --> Mezz
  Mezz --> OB
  OB --> Citadel
  Citadel --> Spine
  Spine --> Hazmat

  Mezz -. "operator projections / reviews" .-> AppKit
  Spine -. "lower facts / receipts" .-> Mezz
  Hazmat -. "raw outcomes" .-> Spine
  StackLab -. "proves seams" .-> AppKit
  StackLab -. "fault drills" .-> Spine
  Trace -. "trace_id joins" .-> Product
  Trace -. "trace_id joins" .-> Hazmat

  classDef product fill:#e0f2fe,stroke:#0369a1,stroke-width:2px,color:#0f172a;
  classDef operational fill:#fef3c7,stroke:#b45309,stroke-width:2px,color:#0f172a;
  classDef brain fill:#f0fdf4,stroke:#15803d,stroke-width:2px,color:#0f172a;
  classDef spine fill:#f1f5f9,stroke:#475569,stroke-width:2px,color:#0f172a;
  classDef proof fill:#fae8ff,stroke:#a21caf,stroke-width:2px,color:#0f172a;

  class Product,AppKit product;
  class Mezz operational;
  class OB,Citadel brain;
  class Spine,Hazmat spine;
  class StackLab,Trace proof;

  style ProductPlane fill:#f8fafc,stroke:#0369a1,stroke-width:2px,color:#0f172a;
  style OperationalPlane fill:#fff7ed,stroke:#b45309,stroke-width:2px,color:#0f172a;
  style BrainPlane fill:#f0fdf4,stroke:#15803d,stroke-width:2px,color:#0f172a;
  style SpinePlane fill:#f8fafc,stroke:#475569,stroke-width:2px,color:#0f172a;
  style ProofPlane fill:#fdf4ff,stroke:#a21caf,stroke-width:2px,color:#0f172a;

This diagram is the simplest mental model:

  • AppKit keeps products honest.
  • Mezzanine keeps operations durable and reusable.
  • OuterBrain lets stochastic reasoning happen without owning durable truth.
  • Citadel turns semantic or typed intent into governed Brain packets.
  • Jido Integration accepts or rejects lower work and owns lower-fact truth.
  • Execution Plane runs dangerous mechanics without learning product meaning.

The spine-to-hazmat split is important here because raw execution is below the
governance and acceptance layers. Products and semantic runtimes do not scrape
it directly.

Monolith First, Physically Bounded From Day One

The initial deployment should be allowed to run as a monolith.

That does not mean the architecture should be a monolith internally. A useful
monolith for this system has physical boundaries inside the codebase:

  • separate repos and packages for owner domains
  • public DTOs instead of shared internal structs
  • explicit contract packets instead of PID or process identity leakage
  • no-bypass scanners to prevent product and AppKit drift into lower layers
  • idempotency keys and stable identifiers at every handoff
  • lease and authorization scopes on lower reads and stream attachment
  • audit and evidence records before derived projections
  • clear owner-of-truth rules for resync after restart

This lets the first deployment avoid premature distributed consensus while
still proving the system will survive distribution later.

%%{init: {"theme": "base", "themeVariables": {"fontFamily": "Inter, ui-sans-serif, system-ui", "primaryColor": "#f8fafc", "primaryTextColor": "#0f172a", "primaryBorderColor": "#475569", "lineColor": "#64748b", "secondaryColor": "#ecfeff", "tertiaryColor": "#fef9c3"}}}%%
flowchart TD
  subgraph Today["Initial&nbsp;deployment:&nbsp;one&nbsp;deployable&nbsp;assembly"]
    TApp["AppKit"]
    TMezz["Mezzanine"]
    TOB["OuterBrain"]
    TCit["Citadel"]
    TSpine["Spine"]
    THaz["Execution Plane"]

    TApp --> TMezz --> TOB --> TCit --> TSpine --> THaz
  end

  subgraph Later["Later&nbsp;deployment:&nbsp;cells&nbsp;and&nbsp;independent&nbsp;durability&nbsp;groups"]
    PCell["Product/API cell"]
    OCell["Operational workflow cell"]
    BCell["Brain semantic/authority cell"]
    SCell["Spine intake cell"]
    ECell["Execution cells"]

    PCell --> OCell
    OCell --> BCell
    BCell --> SCell
    SCell --> ECell
  end

  TApp -. "same DTOs, leases, ids, packets" .-> PCell
  TMezz -. "same workflow/evidence contracts" .-> OCell
  TOB -. "same context/provenance contracts" .-> BCell
  TCit -. "same authority handoff contracts" .-> BCell
  TSpine -. "same lower-fact contracts" .-> SCell
  THaz -. "same runtime contracts" .-> ECell

  classDef today fill:#e0f2fe,stroke:#0369a1,stroke-width:2px,color:#0f172a;
  classDef later fill:#f0fdf4,stroke:#15803d,stroke-width:2px,color:#0f172a;
  class TApp,TMezz,TOB,TCit,TSpine,THaz today;
  class PCell,OCell,BCell,SCell,ECell later;

  style Today fill:#f8fafc,stroke:#0369a1,stroke-width:2px,color:#0f172a;
  style Later fill:#f0fdf4,stroke:#15803d,stroke-width:2px,color:#0f172a;

The practical rule is: local function calls are fine during the monolith
phase, but the payloads must already be serializable contract payloads with
stable logical identity. If an internal handoff requires a process identifier,
shared mutable state, unscoped direct SQL, or a live in-memory object, it is
not a future distributed seam.

End-To-End Operating Model

A governed run should be explainable without naming implementation details.

  1. A product or operator enters through AppKit with a request context.
  2. AppKit validates the product boundary and calls the appropriate northbound
    surface.
  3. Mezzanine resolves installation, pack revision, workflow state, lease
    posture, decision gates, and configured context sources.
  4. If semantic reasoning is required, Mezzanine enters OuterBrain through the
    semantic surface with context-source declarations and runtime bindings.
  5. OuterBrain assembles a read-only context pack from trusted adapters,
    records provenance and diagnostics, and produces semantic output or a
    normalized semantic failure.
  6. Citadel receives typed or semantic intent, constructs deterministic Brain
    context, applies authority and policy, and emits a governed handoff.
  7. Jido Integration accepts or rejects the lower submission, narrows auth and
    runtime scope, records durable lower facts, and exposes bounded reads.
  8. Execution Plane performs raw mechanics and returns lower outcomes to the
    Spine.
  9. Mezzanine updates workflow, evidence, decision, projection, audit, and
    archival truth.
  10. AppKit renders product/operator state from northbound DTOs.
  11. Memory recall and promotion happen only at their allowed seams: read-only
    context admission before reasoning, and evidence-backed derived projection
    after durable outcomes.
sequenceDiagram
  autonumber
  participant Product as Product / Operator
  participant AppKit as AppKit
  participant Mezz as Mezzanine
  participant OB as OuterBrain
  participant Citadel as Citadel
  participant Spine as Jido Integration
  participant Hazmat as Execution Plane
  participant Memory as Governed Memory Surfaces

  Product->>AppKit: Request with tenant, actor, trace
  AppKit->>Mezz: Northbound work/control/query call
  Mezz->>Mezz: Resolve pack, workflow, leases, gates
  Mezz->>OB: Semantic task with trusted context declarations
  OB->>Memory: Read-only recall request with scope and provenance requirements
  Memory-->>OB: Context fragments with provenance/staleness
  OB->>OB: Build context pack and semantic journal
  OB->>Citadel: Semantic intent / normalized failure
  Citadel->>Citadel: Compile authority and planning context
  Citadel->>Spine: Governed lower handoff
  Spine->>Spine: Accept, reject, narrow, or fence
  Spine->>Hazmat: Runtime execution request
  Hazmat-->>Spine: Raw outcome and runtime facts
  Spine-->>Mezz: Lower receipts and bounded facts
  Mezz->>Mezz: Evidence, workflow, review, audit, projection
  Mezz->>Memory: Eligible promotion event with evidence refs
  Memory-->>Mezz: Derived projection receipt
  Mezz-->>AppKit: Operator/product DTOs
  AppKit-->>Product: Governed state and actions

The sequence is illustrative. Many product reads stop at AppKit and Mezzanine.
Many typed commands do not need semantic reasoning. Some operator actions are
local Mezzanine workflow signals. When a request crosses layers, the boundary
shape stays predictable.

Brain, Spine, And The Middle Layer

The architecture uses “Brain” and “Spine” as a durability and responsibility
split, not as branding.

The Brain is where intent is interpreted and governed:

  • OuterBrain handles semantic interpretation, context assembly, prompt
    construction, provider-neutral semantic failures, and semantic reply
    publication.
  • Citadel handles deterministic authority, structured ingress, host-local
    continuity, policy compilation, planning context, and Brain-to-Spine
    handoff.

The Spine is where accepted lower work becomes durable operational fact:

  • Jido Integration owns durable lower intake, connector contracts, auth and
    credential lifecycle, lower receipts, lower events, lower artifact refs,
    tenant-scoped readbacks, review projections, and derived-state anchors.

Mezzanine sits above and around both as the neutral operational layer:

  • It is not the semantic Brain.
  • It is not the lower Spine.
  • It is the durable product-neutral machinery for packs, installation
    activation, workflow runtime, execution ledger, lifecycle, decisions,
    reviews, evidence, projections, operator control, audit, archival, and
    lower-read leasing.

That middle layer is why the stack can support many products without each
product becoming a mini control plane. Products call AppKit. AppKit delegates
to Mezzanine. Mezzanine composes the Brain and Spine according to pack,
workflow, policy, tenant, and installation state.

The Hyperscalar Enterprise Shape

The long-term enterprise version of this architecture is cell-based.

An initial monolith can prove the contracts. A hyperscalar deployment can split
by region, tenant class, product family, data residency zone, execution risk,
or workload tier without changing the logical ownership model.

A plausible global shape is:

  • product/API cells close to users and enterprise control planes
  • regional Mezzanine cells for workflow, decision, evidence, and operator
    projections
  • regional or tenant-isolated Brain cells for semantic runtime and authority
    compilation
  • Spine intake cells with durable lower ledgers and connector/auth control
  • execution cells near data, sandboxes, GPUs, private networks, or customer
    regions
  • cold archival and incident-export systems for long-retention evidence
  • memory projection cells scoped by tenant, region, data class, and policy
  • proof cells that continuously run StackLab-style drills against staged and
    production-like seams

In that world, AppKit remains the product boundary, Mezzanine remains the
operational substrate, OuterBrain remains semantic, Citadel remains authority,
Jido Integration remains lower acceptance, and Execution Plane remains raw
runtime mechanics. The implementation may become distributed, but ownership
does not dissolve.

The enterprise value is not only scale. It is explainability under pressure.
When a global customer asks why an automated agent changed something, the
platform should be able to reconstruct:

  • the tenant and actor context
  • the installation and pack revision
  • the semantic context that was admitted
  • the authority and policy decisions
  • the lower execution receipts
  • the review or waiver state
  • the memory fragments that influenced the run
  • the evidence that allowed any memory promotion
  • the revocations or suppressions in force at the time
  • the final product/operator projection

That is the difference between an agent demo and an enterprise control plane.

Memory Thesis

Memory in a governed agentic platform is not a truth source. It is an
admissible context source with provenance, scope, staleness, and revocation.

The platform truth sources remain:

  • semantic-session facts and prompt/context hashes owned by OuterBrain
  • authority, policy, host context, and advisory planning records owned by
    Citadel
  • workflow, installation, lease, decision, audit, projection, archival, and
    control-room facts owned by Mezzanine
  • lower run, attempt, event, artifact, connector, and target facts owned by
    Jido Integration and Execution Plane

Memory may summarize those truths, index them, rank them, cluster them, or make
them retrievable, but it must not replace them.

A memory fragment is safe to use only when the consumer can answer:

  • Who is allowed to see this fragment?
  • What source record or evidence does it summarize?
  • Which policy or governance decision made it admissible?
  • Is it stale, revoked, quarantined, or redacted?
  • Is it advisory context, durable evidence, or an operator-approved derived
    projection?
  • Which component owns the underlying truth if a conflict appears?

The answer must be carried as structured data, not as tribal knowledge in an
adapter.

Sidecar-Only Memory Is Insufficient

A memory projection sidecar can be a compliant implementation detail, but it is
too narrow to define platform memory.

Valid sidecar constraints:

  • OuterBrain does not own provider memory, RAG engines, or model-native memory
    as platform truth.
  • OuterBrain context adapters are read-only contributors to context-pack
    assembly.
  • The Jido Integration contracts define a higher-order sidecar seam on the
    core/contracts seam.
  • Jido Integration’s DerivedStateAttachment exists precisely so sidecars can
    persist derived enrichments while anchoring them to SubjectRef,
    EvidenceRef, and GovernanceRef.
  • Memory-like sidecars must not copy lower-gateway truth and become a second
    control-plane ledger.

Limits of a sidecar-only model:

  • The stack does not define memory as a single sidecar-owned database. It
    defines memory-related seams across OuterBrain, Citadel, Mezzanine, and Jido
    Integration.
  • “Only persists derived state” is necessary for lower-fact sidecars, but it
    does not cover all memory classes. Citadel has advisory memory records.
    OuterBrain has semantic-session context packs and provenance. Mezzanine has
    context-source declarations and trusted adapter activation gates.
  • “Storage after action passes through Jido Integration” is too simple.
    Durable memory promotion may need evidence from OuterBrain semantic
    validation, Citadel authority, Mezzanine decisions or reviews, Jido lower
    outcomes, and operator suppression or revocation records.
  • A future memory system should not require every memory implementation to live
    under Jido Integration. It should require every memory implementation to
    satisfy the same evidence, governance, recall, and revocation contracts.

A memory projection sidecar is therefore one participant in the promotion
plane. It is not the platform’s memory architecture.

Stack Responsibilities

OuterBrain

OuterBrain is the provider-neutral semantic-runtime gateway above Citadel. It
owns durable semantic-session truth, prompt and strategy shaping, context-pack
construction, tool-manifest snapshots, semantic quality checkpoints,
provider-neutral semantic failure carriers, live semantic-session fencing, and
restart-safe reply publication.

The important memory-relevant contracts are:

  • OuterBrain.ContextAdapterReadOnly.v1
  • OuterBrain.SemanticContextProvenance.v1
  • OuterBrain.SemanticDuplicateSuppression.v1
  • OuterBrain.SemanticActivityNormalized.v1
  • OuterBrain.SemanticActivityPayloadBoundary.v1
  • Platform.SuppressionVisibility.v1
  • OuterBrain.Contracts.ReplyPublication
  • OuterBrain.Prompting.ContextPack
  • OuterBrain.Prompting.ContextFragment
  • OuterBrain.Prompting.ContextAdapterRegistry

OuterBrain’s memory stance is strict:

  • It can assemble context.
  • It can read from configured context adapters.
  • It can record prompt hashes, context hashes, provenance refs, diagnostics
    refs, claim-check refs, and bounded routing facts.
  • It can publish provisional and final semantic replies with deterministic
    suppression.
  • It must not treat raw provider memory, RAG state, provider-native bodies, raw
    prompts, raw context packs, raw artifacts, or workflow history as northbound
    DTO truth.
  • It must not grant context adapters mutation permissions during semantic
    evaluation.

In other words, OuterBrain is the recall orchestrator and semantic provenance
producer. It is not the durable memory store.

The context-pack path is especially important. ContextPack.build/3
takes semantic state, declared context sources, runtime bindings, and an
adapter registry. For each configured source, it sends a bounded request to the
registered adapter and expects a list of structured fragments. Each fragment
must include content, provenance, staleness, and metadata. The pack records the
source report, adapter key, fragment count, merge strategy, and degraded/error
state.

This is the platform’s implemented recall interface.

Citadel

Citadel is the host-local Brain kernel. It accepts structured ingress at the
kernel boundary, compiles authority and planning decisions, preserves host-local
session continuity, and projects Brain-authored packets toward the shared
Jido Integration contracts layer.

Citadel owns:

  • canonical Brain context construction
  • structured host ingress
  • authority packet construction
  • policy and planning context
  • host-local session continuity
  • runtime coordination
  • projection into invocation, review, trace, and derived-state seams

Citadel explicitly does not own durable lower run, attempt, review, approval,
artifact, credential, provider runtime, or raw natural-language truth.

Citadel currently contains a memory-relevant value:

Citadel.MemoryRecord

That record has:

  • memory_id
  • scope_ref
  • session_id
  • kind
  • summary
  • subject_links
  • evidence_links
  • expires_at
  • confidence
  • metadata

Citadel also exposes Citadel.Ports.Memory, an advisory memory seam with:

  • put_memory_record/1
  • get_memory_record/2
  • rank_memory_records/1

The port contract is deliberately advisory. A write can be
:stable_put_by_id or :best_effort. Reads are keyed lexically by
memory_id or ranked by scope, session, kind, and limit. This is useful for
host-local planning and summaries, but it is not a ledger.

Citadel therefore provides a second memory class:

  • OuterBrain memory is recall context for semantic reasoning.
  • Citadel memory is advisory planning context for the Brain kernel.

Both may point at the same source facts, but neither owns those facts.

Mezzanine

Mezzanine is the neutral business-semantics and workflow substrate. It owns the
platform machinery that decides which contexts can be activated, which lower
facts can be read, which workflows are durable, and which reviews or operator
actions are authoritative.

Memory-relevant Mezzanine seams include:

  • Mezzanine.Pack.ContextSourceSpec
  • Mezzanine.Authoring.Bundle
  • Mezzanine.ConfigRegistry
  • Mezzanine.WorkflowRuntime
  • Mezzanine.WorkflowRuntime.ActivitySideEffectIdempotency
  • Mezzanine.Leasing
  • Mezzanine.IntegrationBridge.ReadDispatcher
  • Mezzanine.ControlRoom.IncidentBundle
  • Mezzanine.ControlRoom.IncidentExportBundle
  • Mezzanine.ControlRoom.ForensicReplay
  • Mezzanine.Decisions
  • Mezzanine.Reviews
  • audit, archival, projection, and retry posture contracts

ContextSourceSpec is the pack-level declaration that a read-only source can
feed OuterBrain context assembly. It carries:

  • source_ref
  • description
  • binding_key
  • usage_phase
  • required?
  • timeout_ms
  • schema_ref
  • max_fragments
  • merge_strategy

The key point is that a product pack can declare memory-like context sources
without owning memory runtime code. The Mezzanine config registry validates
context adapter descriptors against trusted registry configuration before
activation. Unknown adapters, missing bindings, invalid descriptors, stale
installation revisions, invalid policy refs, and pack-authored platform
migrations are rejected before runtime activation.

That gives the platform a supply-chain and installation gate for memory
adapters.

Mezzanine also controls the lower-fact read path. Its integration bridge
builds a typed Jido.Integration.V2.TenantScope from authorized context,
checks execution lineage, rejects lower-id overrides, and calls the bounded
Jido Integration LowerFacts surface. Product code is expected to enter
through AppKit and carry Mezzanine read leases plus authorization scope.

For memory, this means lower facts can be recalled only through:

  • an authorized workflow or operator context
  • an execution lineage
  • a tenant and installation scope
  • lease and revocation checks
  • bounded lower-facts operations

Mezzanine is therefore the policy, lease, workflow, and evidence envelope
around memory. It should not become a vector store, but it should govern which
memory adapters are trusted, which facts can be read, and which derived memory
updates can be promoted.

Jido Integration

Jido Integration owns the lower acceptance gateway and public lower contracts
for connector discovery, invocation, auth lifecycle, runtime targeting, review
packet projection, lower facts, and substrate-facing reads.

Memory-relevant Jido Integration contracts include:

  • Jido.Integration.V2.SubjectRef
  • Jido.Integration.V2.EvidenceRef
  • Jido.Integration.V2.GovernanceRef
  • Jido.Integration.V2.DerivedStateAttachment
  • Jido.Integration.V2.ReviewProjection
  • Jido.Integration.V2.TenantScope
  • Jido.Integration.V2.LowerFacts
  • Jido.Integration.V2.SubstrateReadSlice
  • ExecutionEvent.v1
  • ExecutionOutcome.v1

The lower-facts surface is intentionally bounded:

  • submission receipts
  • run state
  • attempt state
  • event streams
  • artifact refs
  • scoped trace resolution

It does not assemble product projections, operator packets, memory summaries,
or review aggregates.

DerivedStateAttachment is the key memory-adjacent contract. It states that
higher-order repos may persist their own enrichments, memories, lineage, and
scores, but those records must remain anchored to node-local source truth
through explicit subject, evidence, and governance refs.

That is the right sidecar contract for a future memory projection service:

  • store repo-local memory records
  • anchor each record to SubjectRef, EvidenceRef, and GovernanceRef
  • avoid copying lower-gateway-owned truth
  • avoid becoming a second review, approval, run, or artifact ledger
  • expose derived state through its own bounded API

But Jido Integration should not be the semantic recall owner. Its role is lower
truth and sidecar anchoring.

AppKit

AppKit is the northbound application-surface and product-boundary enforcement
point. Product repos must use AppKit surfaces for governed writes, operator
reads, reviews, installation bootstrap, semantic assist, trace lookup, and
leased lower read access.

For memory integration, AppKit matters because it prevents product apps from
calling Mezzanine, Citadel, Jido Integration, or Execution Plane directly for
governed behavior. A product can author a pure Mezzanine.Pack model contract,
including context source declarations, but runtime behavior must flow through
AppKit and the lower owner seams.

That means memory is not allowed to become a product-local bypass path.

Execution Plane

Execution Plane owns lower runtime execution and effect projection mechanics.
It should not own semantic memory. It may produce lower events, artifacts,
process output, HTTP results, JSON-RPC results, and runtime state that later
become eligible for memory projection, but memory sidecars should consume those
through Jido Integration and Mezzanine, not through direct runtime scraping.

Ground Plane, StackLab, And AITrace

Ground Plane supplies generic lower contracts and projection helpers. It is
not the memory owner.

StackLab is the proving harness. It should prove that memory recall and memory
promotion do not bypass AppKit, Mezzanine leases, Citadel authority, OuterBrain
semantic boundaries, or Jido Integration tenant scope.

AITrace can be useful as an observability layer for reasoning spans, tool
calls, and trace visualization. It should be treated as trace evidence or
developer observability, not as memory truth.

Ownership Diagram

%%{init: {"theme": "base", "themeVariables": {"fontFamily": "Inter, ui-sans-serif, system-ui", "primaryColor": "#f8fafc", "primaryTextColor": "#0f172a", "primaryBorderColor": "#475569", "lineColor": "#475569", "secondaryColor": "#ecfeff", "tertiaryColor": "#fef9c3"}}}%%
flowchart TB
  Product["Product / operator UI"]
  AppKit["AppKit<br />northbound boundary"]
  Mezz["Mezzanine<br />workflow, packs, leases, evidence"]
  OB["OuterBrain<br />semantic context and provenance"]
  Citadel["Citadel<br />authority, planning, advisory memory"]
  JI["Jido Integration<br />lower facts and sidecar anchors"]
  EP["Execution Plane<br />runtime execution"]
  Sidecar["Memory projection sidecar<br />optional, derived state"]

  Product --> AppKit
  AppKit --> Mezz
  Mezz --> OB
  OB --> Citadel
  Citadel --> JI
  JI --> EP

  Mezz -. "trusted context source descriptors" .-> OB
  OB -. "read-only context fragments" .-> Citadel
  Mezz -. "tenant-scoped lower reads" .-> JI
  JI -. "SubjectRef / EvidenceRef / GovernanceRef" .-> Sidecar
  Sidecar -. "recall fragments with provenance" .-> OB

  classDef boundary fill:#e0f2fe,stroke:#0369a1,stroke-width:2px,color:#0f172a;
  classDef semantic fill:#f0fdf4,stroke:#15803d,stroke-width:2px,color:#0f172a;
  classDef governance fill:#fef3c7,stroke:#b45309,stroke-width:2px,color:#0f172a;
  classDef lower fill:#f1f5f9,stroke:#475569,stroke-width:2px,color:#0f172a;
  classDef sidecar fill:#fae8ff,stroke:#a21caf,stroke-width:2px,color:#0f172a;

  class Product,AppKit boundary;
  class OB semantic;
  class Citadel,Mezz governance;
  class JI,EP lower;
  class Sidecar sidecar;

Memory Classes

The word “memory” hides several different data classes. The platform should
name them separately.

1. Semantic-Session State

Owner: OuterBrain

Purpose:

  • preserve the state of one semantic interaction or semantic session
  • construct prompt/context packs
  • maintain commitments and unresolved questions
  • support deterministic reply publication and restart-safe recovery

Allowed to influence:

  • prompt assembly
  • semantic strategy
  • reply publication
  • semantic validation and reask paths

Not allowed to replace:

  • lower run facts
  • workflow facts
  • authority decisions
  • review decisions
  • product state

2. Advisory Planning Memory

Owner: Citadel, with backing implementation behind Citadel.Ports.Memory

Purpose:

  • expose host-local summaries, prior choices, preferences, lessons, or planning
    hints
  • give the Brain kernel an advisory recall surface
  • preserve a lexical memory_id for stable lookup

Allowed to influence:

  • planning
  • summarization
  • host-local continuity
  • policy-context enrichment when explicitly admitted

Not allowed to replace:

  • authority packets
  • execution governance
  • review packets
  • derived-state attachment
  • lower evidence

3. Pack-Declared Context Sources

Owner: Mezzanine for declaration and activation; OuterBrain for runtime
context assembly

Purpose:

  • declare memory-like context sources in pack manifests
  • bind source refs such as workspace_memory to trusted adapter keys
  • restrict usage phase, fragment count, timeout, schema, and merge strategy

Allowed to influence:

  • OuterBrain context-pack assembly
  • semantic recall during preprocessing, retrieval, or repair

Not allowed to replace:

  • adapter trust registry
  • runtime authorization
  • provenance checks

cont…

Brain/Spine Platform And Governed Memory Deep Dive 2 of 2

…cont

4. Lower-Fact Projections

Owner: Jido Integration and Execution Plane for lower facts; Mezzanine for
authorized read access

Purpose:

  • recall facts about runs, attempts, events, artifacts, submission receipts,
    and traces
  • keep lower execution truth behind tenant scope
  • provide durable evidence anchors for memory projections

Allowed to influence:

  • operator investigation
  • semantic recall when admitted through a context adapter
  • memory promotion decisions
  • forensic and incident reconstruction

Not allowed to replace:

  • higher-level policy or review decisions
  • semantic interpretation
  • advisory summaries

5. Derived Memory Projection

Owner: future memory sidecar or product-approved memory projection service,
anchored by Jido Integration and Mezzanine evidence

Purpose:

  • persist summaries, embeddings, rankings, links, scores, or facts derived from
    durable source records
  • provide efficient retrieval without becoming the source of truth
  • rebuild from source evidence when needed

Allowed to influence:

  • future recall when fragments carry provenance and scope
  • operator search and summarization
  • downstream recommendations when explicitly policy-approved

Not allowed to replace:

  • source records
  • approval state
  • audit logs
  • review decisions
  • revocation records

6. Operator And Review Memory

Owner: Mezzanine decisions/reviews/control-room surfaces, possibly projected
into a sidecar

Purpose:

  • carry reviewer rationale, waivers, suppressions, escalations, incident notes,
    and runbook links
  • inform later reasoning only when policy permits

Allowed to influence:

  • future review posture
  • operator-visible context
  • incident reconstruction

Not allowed to replace:

  • the decision ledger
  • review quorum rules
  • release-readiness evidence

Two-Plane Model

Recall Plane

The recall plane is read-only.

It takes memory from an external or sidecar source and admits it into semantic
reasoning as context fragments. It must answer these questions before a
fragment reaches a prompt:

  • Was the adapter trusted at installation or runtime activation time?
  • Was the read authorized for this tenant, installation, actor, session, and
    trace?
  • Which source refs and evidence refs back the fragment?
  • Which schema does the fragment claim?
  • Which redaction policy was applied?
  • What is the staleness class?
  • How many fragments can this source contribute?
  • Is the source optional or required?
  • Is the adapter definitely unable to mutate state during this read?

OuterBrain’s current context adapter and context fragment shape is the concrete
implementation example.

Promotion Plane

The promotion plane is write-capable, but it is not allowed to write from the
middle of model reasoning directly into future memory.

It takes durable source records and constructs memory candidates. A candidate
can become a memory projection only after policy, provenance, and review rules
allow it.

The promotion plane should answer:

  • Which durable record or records produced this memory candidate?
  • Which authority decision allowed the source action?
  • Which review or governance decision approved the promotion?
  • Is the candidate a summary, preference, operational lesson, user fact,
    artifact index, policy note, or incident note?
  • Is it tenant-bound, installation-bound, workspace-bound, project-bound,
    actor-bound, session-bound, or global?
  • What is the retention class?
  • What is the revocation path?
  • Can the projection be rebuilt from source truth?
  • What happens if the source truth is corrected or deleted?

Jido Integration’s DerivedStateAttachment is the concrete anchor for this
plane, but the promotion policy itself belongs above the sidecar, primarily in
Mezzanine and Citadel.

Recall Flow

sequenceDiagram
  participant Product as Product / AppKit
  participant Mezz as Mezzanine
  participant OB as OuterBrain
  participant Adapter as Context Adapter
  participant Citadel as Citadel
  participant JI as Jido Integration
  participant Sidecar as Memory Sidecar

  Product->>Mezz: start governed work with tenant and installation context
  Mezz->>Mezz: load compiled pack and context source specs
  Mezz->>OB: invoke semantic activity with context source bindings
  OB->>OB: build ContextAdapterReadOnly request
  OB->>Adapter: fetch_fragments(read-only request, runtime binding)
  Adapter->>Citadel: optional advisory memory read
  Adapter->>Mezz: optional authorized lower read request
  Mezz->>JI: LowerFacts read with TenantScope
  Adapter->>Sidecar: optional derived memory read by scoped query
  Sidecar-->>Adapter: fragments with source and evidence refs
  JI-->>Mezz: lower facts
  Citadel-->>Adapter: advisory records
  Adapter-->>OB: ContextFragment list with provenance and staleness
  OB->>OB: assemble context pack and context hash
  OB-->>Mezz: semantic result with provenance refs and routing facts

Important properties:

  • The adapter is not trusted because it is code. It is trusted because
    Mezzanine activation accepted its descriptor and OuterBrain calls it under a
    read-only contract.
  • Lower-fact reads do not happen by lower id alone. They require Mezzanine
    authorization and Jido Integration TenantScope.
  • A memory sidecar can contribute fragments, but those fragments must carry
    source and governance references.
  • OuterBrain records context hashes and provenance refs so the workflow can
    reason from compact references rather than raw prompt or raw context blobs.

Promotion Flow

sequenceDiagram
  participant OB as OuterBrain
  participant Citadel as Citadel
  participant Mezz as Mezzanine
  participant JI as Jido Integration
  participant Review as Review / Operator
  participant Sidecar as Memory Sidecar

  OB->>Mezz: semantic result with context hash, provenance, routing facts
  Citadel->>Mezz: authority and governance refs
  JI->>Mezz: lower outcome, events, artifacts, trace refs
  Mezz->>Mezz: correlate semantic, authority, workflow, lower, and audit refs
  Mezz->>Review: require review if policy says memory promotion is sensitive
  Review-->>Mezz: approve, deny, waive, or escalate
  Mezz->>Sidecar: publish MemoryProjectionEvent / DerivedStateAttachment
  Sidecar->>Sidecar: persist derived memory with rebuildable source refs
  Sidecar-->>Mezz: projection receipt and index refs
  Mezz->>OB: future recall can use sidecar only through read-only adapter

Important properties:

  • OuterBrain may emit semantic summaries or memory candidates, but it should
    not mutate durable memory directly during reasoning.
  • Citadel may provide advisory memory or authority context, but durable memory
    promotion still needs evidence and policy.
  • Mezzanine is the promotion coordinator because it can see workflow state,
    decisions, audit, leases, and installation revisions.
  • Jido Integration supplies lower source facts and derived-state anchors.
  • The sidecar owns indexes and derived state, not source truth.

OuterBrain, Citadel, And Mezzanine Symbiosis

The memory architecture makes sense only when these three layers are treated as
complementary.

OuterBrain As Semantic Recall Boundary

OuterBrain answers:

  • What context is admissible to this reasoning turn?
  • Which context sources contributed fragments?
  • What was the context hash?
  • Which prompt, provider, model, and normalizer were used?
  • Was the semantic output valid, degraded, quarantined, terminal, retryable, or
    review-worthy?
  • What should be published, suppressed, or retried?

OuterBrain must be close to the model because it owns prompt assembly and
semantic validation. It must not own memory truth because that would let the
semantic layer rewrite the evidence that later semantic layers consume.

Citadel As Authority And Advisory Context Boundary

Citadel answers:

  • Is the request authorized?
  • Which policy, authority packet, scope, resource profile, approval profile,
    egress profile, workspace profile, and boundary class apply?
  • Which host-local advisory memory records are relevant?
  • Which plan or invocation should be compiled from structured ingress?
  • Which derived-state or trace seams receive Brain-authored packet context?

Citadel is where advisory memory belongs because it is host-local, planning
oriented, and scoped by policy context. It should not become the durable memory
projection owner because it does not own lower outcomes, workflow decisions, or
review ledgers.

Mezzanine As Workflow, Lease, And Promotion Boundary

Mezzanine answers:

  • Which pack declared this context source?
  • Was the context adapter trusted at activation time?
  • Which installation revision and lease epoch are active?
  • Which workflow owns the durable execution?
  • Which lower facts can be read under this authorization scope?
  • Which review, waiver, escalation, suppression, or audit facts constrain the
    result?
  • Is a derived memory projection allowed?

Mezzanine should be the promotion coordinator because it can join semantic
facts, authority refs, lower facts, decisions, audit, archival, and operator
projections without making any one runtime the memory owner.

Symbiosis Summary

%%{init: {"theme": "base", "themeVariables": {"fontFamily": "Inter, ui-sans-serif, system-ui", "primaryColor": "#f8fafc", "primaryBorderColor": "#475569", "lineColor": "#334155", "clusterBkg": "#ffffff", "clusterBorder": "#cbd5e1"}}}%%
flowchart TD
  subgraph S["Semantic&nbsp;Plane"]
    OB1["OuterBrain<br />recall, context hash, semantic validation"]
  end

  subgraph A["Authority&nbsp;Plane"]
    C1["Citadel<br />authority, planning, advisory memory"]
  end

  subgraph W["Workflow&nbsp;Plane"]
    M1["Mezzanine<br />packs, leases, workflow, review, promotion"]
  end

  subgraph L["Lower&nbsp;Fact&nbsp;Plane"]
    J1["Jido Integration<br />lower facts, refs, derived anchors"]
    E1["Execution Plane<br />runtime effects"]
  end

  OB1 -- "semantic result refs" --> M1
  C1 -- "authority and governance refs" --> M1
  M1 -- "authorized lower read" --> J1
  J1 -- "lower evidence refs" --> M1
  J1 --> E1
  M1 -- "trusted context bindings" --> OB1
  C1 -- "advisory records" --> OB1

  classDef semantic fill:#dcfce7,stroke:#15803d,stroke-width:2px,color:#0f172a;
  classDef authority fill:#fef3c7,stroke:#b45309,stroke-width:2px,color:#0f172a;
  classDef workflow fill:#dbeafe,stroke:#2563eb,stroke-width:2px,color:#0f172a;
  classDef lower fill:#f1f5f9,stroke:#475569,stroke-width:2px,color:#0f172a;

  class OB1 semantic;
  class C1 authority;
  class M1 workflow;
  class J1,E1 lower;

  style S fill:#f0fdf4,stroke:#15803d,stroke-width:2px,color:#0f172a;
  style A fill:#fff7ed,stroke:#b45309,stroke-width:2px,color:#0f172a;
  style W fill:#eff6ff,stroke:#2563eb,stroke-width:2px,color:#0f172a;
  style L fill:#f8fafc,stroke:#475569,stroke-width:2px,color:#0f172a;

The rule of thumb:

  • OuterBrain decides how memory enters reasoning.
  • Citadel decides whether context and action are authorized.
  • Mezzanine decides whether evidence can be read, joined, reviewed, and
    promoted.
  • Jido Integration proves what actually happened at the lower boundary.

Proposed Standard: Governed Memory Context Protocol

This section generalizes the stack into a possible standard for memory
integration in any serious AI governance platform.

The standard is not “install a vector database.” The standard is a set of
seams that make memory safe, reviewable, revocable, and rebuildable.

Standard Goals

A compliant memory integration should:

  • keep recall reads separate from promotion writes
  • prevent model-internal loops from mutating future recall state directly
  • require explicit memory-source declarations
  • require read-only context adapters for semantic evaluation
  • carry provenance, staleness, redaction, and scope on every fragment
  • anchor promoted memory to durable subject, evidence, and governance refs
  • support revocation, suppression, deletion, and rebuild
  • allow sidecars and external stores without making them truth owners
  • support monolith and distributed implementations
  • make no-bypass enforcement testable

Required Components In A Generic Monolith

Even a monolith AI agent runtime needs these logical components if memory is
going to be governed:

  • Application boundary: accepts user and operator requests.
  • Authority engine: decides tenant, actor, policy, capability, and resource
    scope.
  • Semantic runtime: assembles prompts, calls models, validates semantic output,
    and emits semantic provenance.
  • Workflow/evidence ledger: records accepted work, durable outcomes, decisions,
    retries, and audit events.
  • Runtime/tool layer: executes tools, connectors, subprocesses, HTTP calls, or
    provider actions.
  • Context source registry: declares which memory/context adapters are trusted.
  • Recall adapter interface: read-only API that returns provenance-bearing
    fragments.
  • Memory projection coordinator: decides whether durable outcomes can become
    memory candidates or promoted memories.
  • Review engine: handles human or policy review for sensitive memory
    promotion.
  • Memory projection store: stores indexes, summaries, embeddings, rankings,
    and derived memory records.
  • Revocation and invalidation service: removes or suppresses memory when source
    truth is revoked, corrected, deleted, or policy changes.
  • Operator surface: explains why memory was recalled, promoted, suppressed, or
    denied.
  • Trace and audit system: reconstructs causality without exposing raw provider
    or tenant-sensitive payloads.

The nshkr stack distributes these components across repos. A monolith can keep
them in one codebase, but it still needs the same boundaries.

Standard Data Contracts

The following contract set defines the generic standard. The names are
illustrative, but the fields are the important part.

1. MemoryContextSourceDescriptor

Declares that a memory source may be used for recall.

Required fields:

  • source_ref
  • binding_key
  • adapter_ref
  • usage_phase
  • schema_ref
  • max_fragments
  • timeout_ms
  • merge_strategy
  • required
  • trusted_registry_ref
  • redaction_policy_ref
  • allowed_read_resources
  • denied_write_resources

Equivalent stack seams:

  • Mezzanine.Pack.ContextSourceSpec
  • Mezzanine authoring context adapter descriptors
  • OuterBrain.ContextAdapterReadOnly.v1

2. MemoryReadRequest

The request sent to a context adapter.

Required fields:

  • tenant_ref
  • installation_ref
  • workspace_ref
  • project_ref
  • environment_ref
  • actor_ref
  • session_ref
  • trace_id
  • semantic_ref
  • source_ref
  • binding_key
  • usage_phase
  • schema_ref
  • max_fragments
  • objective
  • unresolved_questions
  • commitments
  • refs
  • allowed_read_resources
  • mutation_permissions: []

Equivalent stack seams:

  • OuterBrain.Prompting.ContextPack source request
  • OuterBrain.ContextAdapterReadOnly.v1
  • Mezzanine lower-read authorization scope

3. MemoryFragment

A piece of recall context admitted into reasoning.

Required fields:

  • fragment_id
  • schema_ref
  • schema_version
  • content
  • provenance
  • staleness
  • redaction
  • scope
  • source_refs
  • evidence_refs
  • governance_refs
  • confidence
  • expires_at
  • metadata

Equivalent stack seams:

  • OuterBrain.Prompting.ContextFragment
  • Citadel.MemoryRecord
  • Jido.Integration.V2.EvidenceRef
  • Jido.Integration.V2.GovernanceRef

4. MemoryContextProvenance

Records how recall influenced semantic reasoning.

Required fields:

  • semantic_ref
  • provider_ref
  • model_ref
  • prompt_hash
  • context_hash
  • input_claim_check_ref
  • output_claim_check_ref
  • provenance_refs
  • normalizer_version
  • redaction_policy_ref
  • trace_id
  • idempotency_key

Equivalent stack seam:

  • OuterBrain.SemanticContextProvenance.v1

5. MemoryCandidate

A candidate memory produced after semantic validation, tool execution, lower
outcome, review, or operator annotation.

Required fields:

  • candidate_id
  • candidate_kind
  • summary
  • content_ref
  • source_subject_refs
  • source_evidence_refs
  • source_governance_refs
  • semantic_refs
  • authority_refs
  • workflow_refs
  • lower_refs
  • confidence
  • risk_band
  • retention_class
  • proposed_scope
  • redaction_policy_ref
  • idempotency_key

Equivalent stack seams:

  • future Mezzanine memory promotion command
  • OuterBrain normalized semantic result
  • Citadel authority and advisory memory refs
  • Jido Integration derived-state attachment inputs

6. MemoryPromotionDecision

The policy or human decision that accepts, denies, waives, escalates, or
expires a memory candidate.

Required fields:

  • decision_id
  • candidate_id
  • decision
  • reason
  • actor_ref
  • policy_ref
  • review_ref
  • quorum_ref
  • safe_action
  • idempotency_key
  • trace_id

Equivalent stack seams:

  • Mezzanine.Decisions
  • Mezzanine.Reviews
  • Jido.Integration.V2.GovernanceRef

7. MemoryProjectionEvent

The command/event sent to a memory sidecar after promotion is approved.

Required fields:

  • projection_event_id
  • projection_kind
  • subject_ref
  • evidence_refs
  • governance_refs
  • projection_payload
  • projection_hash
  • schema_ref
  • scope
  • retention_class
  • redaction_policy_ref
  • rebuild_policy
  • idempotency_key

Equivalent stack seam:

  • Jido.Integration.V2.DerivedStateAttachment

8. MemoryInvalidation

Revokes, suppresses, quarantines, expires, or deletes memory.

Required fields:

  • invalidation_id
  • memory_ref
  • source_ref
  • reason
  • scope
  • revocation_ref
  • cache_invalidation_ref
  • post_revocation_attempt_ref
  • operator_visibility
  • effective_at
  • trace_id

Equivalent stack seams:

  • Citadel lease revocation contracts
  • Mezzanine lease invalidation and operator actions
  • OuterBrain suppression visibility
  • Jido Integration lease revocation and lower truth integrity contracts

Standard Component Diagram

%%{init: {"theme": "base", "themeVariables": {"fontFamily": "Inter, ui-sans-serif, system-ui", "primaryColor": "#f8fafc", "primaryBorderColor": "#334155", "lineColor": "#475569", "clusterBkg": "#ffffff", "clusterBorder": "#cbd5e1"}}}%%
flowchart LR
  subgraph Mono["Generic&nbsp;Agent&nbsp;Runtime,&nbsp;Monolith&nbsp;Or&nbsp;Distributed"]
    App["Application Boundary"]
    Auth["Authority Engine"]
    Sem["Semantic Runtime"]
    WF["Workflow / Evidence Ledger"]
    Tool["Runtime / Tool Layer"]
    Reg["Context Source Registry"]
    Recall["Read-Only Recall Adapters"]
    Promote["Memory Promotion Coordinator"]
    Review["Review Engine"]
    Store["Memory Projection Store"]
    Revoke["Revocation / Invalidation"]
    Ops["Operator Surface"]
    Trace["Trace / Audit"]
  end

  App --> Auth
  Auth --> Sem
  Reg --> Sem
  Sem --> Recall
  Recall --> Store
  Sem --> WF
  WF --> Tool
  Tool --> WF
  WF --> Promote
  Auth --> Promote
  Review --> Promote
  Promote --> Store
  Revoke --> Store
  WF --> Trace
  Sem --> Trace
  Auth --> Trace
  Store --> Ops
  Trace --> Ops

  classDef app fill:#e0f2fe,stroke:#0369a1,stroke-width:2px,color:#0f172a;
  classDef gov fill:#fef3c7,stroke:#b45309,stroke-width:2px,color:#0f172a;
  classDef sem fill:#dcfce7,stroke:#15803d,stroke-width:2px,color:#0f172a;
  classDef data fill:#fae8ff,stroke:#a21caf,stroke-width:2px,color:#0f172a;
  classDef obs fill:#f1f5f9,stroke:#475569,stroke-width:2px,color:#0f172a;

  class App app;
  class Auth,WF,Review,Promote,Revoke gov;
  class Sem,Recall,Reg sem;
  class Store data;
  class Tool,Ops,Trace obs;

  style Mono fill:#f8fafc,stroke:#334155,stroke-width:2px,color:#0f172a;

Concrete Future Direction For This Stack

Direction 1: Treat Memory As A First-Class Pack Context Source

Mezzanine already has ContextSourceSpec. That should become the standard
configuration path for recall:

  • product packs declare context sources
  • authoring bundles include context adapter descriptors
  • the config registry validates those descriptors against a trusted registry
  • AppKit imports authoring bundles through the operator-only path
  • OuterBrain receives only activated runtime bindings
  • adapters return ContextFragment values with provenance and staleness

This lets products opt into memory without bypassing product boundaries.

Direction 2: Promote A Formal Memory Fragment Contract

OuterBrain’s ContextFragment is close to the right shape, but future memory
needs stronger fields:

  • source refs
  • evidence refs
  • governance refs
  • redaction status
  • retention class
  • expiry
  • confidence
  • recall reason
  • staleness class
  • revocation status

The current ContextFragment can remain the semantic-facing minimal shape, but
a memory-specific contract should normalize these fields before fragments enter
the context pack.

Direction 3: Keep Citadel Memory Advisory

Citadel.MemoryRecord should stay host-local and advisory.

It is useful for:

  • user preference summaries
  • local planning hints
  • host-session continuity
  • operator notes
  • policy-adjacent reminders

It should not be used as:

  • source truth
  • review truth
  • lower fact truth
  • cross-tenant vector memory
  • unreviewed long-term user fact storage

If Citadel memory needs to become durable and product-visible, promotion should
flow through Mezzanine and a sidecar projection, not by expanding
Citadel.Ports.Memory into a platform ledger.

Direction 4: Keep Memory Projection Sidecars Derived And Bounded

A future memory projection sidecar should likely:

  • depend only on the Jido Integration contracts seam plus any generic platform
    DTOs explicitly approved for memory
  • persist memory records, embeddings, rankings, clusters, and search indexes
  • require DerivedStateAttachment for every promoted record
  • keep source, evidence, and governance refs in the primary key or uniqueness
    policy
  • support idempotent rebuild from lower facts, review decisions, and semantic
    provenance
  • expose recall through a read-only adapter consumed by OuterBrain
  • expose operator search through AppKit or a Mezzanine/AppKit read surface
  • implement suppression, revocation, and deletion as first-class state

It should not:

  • call Execution Plane directly
  • ingest lower store rows without TenantScope
  • make review or approval decisions
  • mutate Citadel authority
  • mutate OuterBrain semantic-session truth
  • become the canonical transcript store

Direction 5: Add A Mezzanine Memory Promotion Coordinator

Mezzanine is the right place to coordinate promotion because it can join:

  • pack context source policy
  • installation revision and activation epoch
  • semantic result refs from OuterBrain
  • authority refs from Citadel
  • lower outcome refs from Jido Integration
  • review and decision state
  • audit and archival state
  • operator suppression and revocation state

A future coordinator could expose commands like:

  • propose_memory_candidate/1
  • approve_memory_candidate/2
  • deny_memory_candidate/2
  • publish_memory_projection/1
  • invalidate_memory_projection/1
  • rebuild_memory_projection/1

Those commands should emit compact refs and never raw provider payloads.

Direction 6: Add AppKit Memory Surfaces

Products should not directly call memory sidecars. AppKit should eventually
own the northbound product/operator surfaces:

  • list memory context sources for an installation
  • preview recalled fragments for a work item
  • explain why a fragment was admitted
  • approve or deny memory promotion when policy requires a human
  • suppress or revoke memory
  • export memory evidence for incident review

The AppKit surface should talk to Mezzanine, not to a sidecar directly.

Direction 7: Prove No-Bypass In StackLab

StackLab should prove:

  • product code cannot import memory sidecars directly for governed reads
  • OuterBrain adapters reject mutation permissions
  • Mezzanine rejects unknown context adapters before activation
  • Jido lower facts reject missing or wrong TenantScope
  • memory promotion without evidence refs is rejected
  • memory promotion without governance refs is rejected when policy requires
    approval
  • revoked memory cannot be recalled
  • stale installation revisions cannot activate memory adapters
  • sidecar records can be rebuilt from source refs

Suggested First Formal Memory Contract Set

This stack could converge on these concrete names:

  • Platform.MemoryContextSource.v1
  • Platform.MemoryReadRequest.v1
  • Platform.MemoryFragment.v1
  • Platform.MemoryRecallReport.v1
  • Platform.MemoryCandidate.v1
  • Platform.MemoryPromotionDecision.v1
  • Platform.MemoryProjectionEvent.v1
  • Platform.MemoryInvalidation.v1
  • Platform.MemoryProjectionReceipt.v1

Possible ownership:

  • Mezzanine owns MemoryContextSource, MemoryCandidate,
    MemoryPromotionDecision, MemoryProjectionEvent, MemoryInvalidation, and
    MemoryProjectionReceipt.
  • OuterBrain owns or mirrors MemoryReadRequest, MemoryFragment, and
    MemoryRecallReport at the semantic boundary.
  • Citadel maps Citadel.MemoryRecord into MemoryFragment when advisory
    memory is admitted to reasoning.
  • Jido Integration keeps DerivedStateAttachment, SubjectRef,
    EvidenceRef, GovernanceRef, and lower facts as the lower sidecar anchor.
  • AppKit owns northbound DTOs generated from the Mezzanine/OuterBrain memory
    contracts.

The key is not the exact module names. The key is that recall, promotion, and
invalidation become explicit contracts.

Review Checklist For Any Memory Adapter

Before a memory adapter is accepted, reviewers should ask:

  • Is this adapter read-only during OuterBrain semantic evaluation?
  • Does it declare denied write resources?
  • Does it reject mutation permission grants?
  • Is it registered in the trusted adapter registry before activation?
  • Does every fragment carry source provenance?
  • Does every fragment carry staleness?
  • Does the adapter respect tenant and installation scope?
  • Does it support redaction and revocation?
  • Does it return bounded fragment counts?
  • Does it fail closed when required context is missing?
  • Is the adapter output deterministic enough to audit by fragment ids and
    hashes?
  • Does the adapter avoid raw lower ids as public lookup keys?
  • Can operator surfaces explain why a fragment was recalled?

Review Checklist For Any Memory Projection Sidecar

Before a memory projection sidecar is accepted, reviewers should ask:

  • Does every stored memory record have a source subject ref?
  • Does every stored memory record have evidence refs?
  • Does every stored memory record have governance refs when policy requires
    them?
  • Can the record be rebuilt from source truth?
  • Does the sidecar avoid copying lower run, attempt, event, artifact, review,
    or approval truth as its own truth?
  • Does it use tenant-scoped lower reads?
  • Does it expose recall through a read-only adapter?
  • Does it expose operator writes only through Mezzanine/AppKit surfaces?
  • Does it handle source revocation, deletion, and correction?
  • Does it avoid direct Execution Plane access?
  • Does it avoid direct product bypass?
  • Does it maintain idempotent promotion keys?

Open Questions

Who Owns The First Memory-Specific Contract Package?

The likely answer is Mezzanine for promotion and installation policy, with
OuterBrain owning the semantic read boundary. But a small shared package might
be justified if the contracts need to be consumed by Citadel, Mezzanine,
OuterBrain, AppKit, and a future memory sidecar without creating cycles.

Should Citadel MemoryRecord Be Mapped Or Replaced?

Citadel.MemoryRecord is useful and already advisory. It may be enough to map
it into a generic MemoryFragment at the adapter boundary. Replacing it too
early could blur Citadel’s host-local role.

What Counts As A Memory Promotion?

Some records are harmless summaries. Others encode user preferences, operator
decisions, business facts, or safety-sensitive instructions. The platform needs
promotion classes:

  • low-risk automatic summary
  • operator-visible automatic projection
  • policy-reviewed projection
  • human-reviewed projection
  • prohibited projection

What Is The Revocation Model?

Memory revocation must handle:

  • user deletion requests
  • source artifact deletion
  • lower fact correction
  • policy changes
  • tenant offboarding
  • operator suppression
  • semantic integrity quarantine
  • expired retention

The revocation state should be visible to OuterBrain recall, Mezzanine
operator surfaces, and any sidecar projection.

How Are Embeddings Governed?

Embeddings are derived state. They can leak source content and should carry:

  • source evidence refs
  • model/version refs
  • redaction policy refs
  • retention class
  • scope
  • rebuild key
  • deletion key

The platform should not treat an embedding index as harmless just because it is
not human-readable.

How Are Conflicts Resolved?

Memory will conflict with source truth, current policy, or newer memory. The
standard should define precedence:

  1. Revocation and deletion state
  2. Current policy and authority
  3. Durable source truth
  4. Reviewed derived memory
  5. Advisory memory
  6. Unreviewed recall hints

OuterBrain should never resolve conflicts silently inside prompt assembly.
Conflicts should produce degraded context, review-required routing facts, or
operator-visible suppression.

Should Memory Candidates Be Produced By OuterBrain?

OuterBrain is close to semantic summarization and can produce useful candidate
summaries. It should be allowed to propose candidates, but not to promote them
directly. Promotion requires Mezzanine policy and evidence joins.

Should Memory Sidecars Read Mezzanine Or Jido Integration?

For lower execution evidence, the sidecar should read through Jido Integration
lower facts with tenant scope, usually under Mezzanine authorization. For
workflow decisions, reviews, and promotions, it should receive Mezzanine
projection events rather than scraping Mezzanine stores.

Where Does Operator Search Live?

Operator search over memory should probably surface through AppKit and
Mezzanine so products see governed DTOs. The sidecar can own the index and
search mechanics, but not the product-facing authority.

What Should Be In CI?

No-bypass scans should eventually assert:

  • product repos do not call memory sidecars directly for governed recall
  • OuterBrain does not import sidecar write APIs
  • memory sidecars do not import Execution Plane direct runtime APIs
  • adapters implement read-only behavior
  • promotion requires evidence refs
  • revocation is tested by negative recall

Practical Near-Term Architecture

The near-term architecture is:

  1. Use Mezzanine.Pack.ContextSourceSpec as the product-pack declaration for
    memory recall sources.
  2. Use Mezzanine authoring bundle validation to allow only trusted context
    adapter descriptors.
  3. Use OuterBrain.Prompting.ContextPack and
    OuterBrain.ContextAdapterReadOnly.v1 as the first recall runtime.
  4. Map Citadel.MemoryRecord into context fragments only through an adapter,
    preserving advisory status and scope.
  5. Use Mezzanine read leases and Jido.Integration.V2.TenantScope for any
    lower-fact-backed memory recall.
  6. Define a Mezzanine memory promotion coordinator before implementing a large
    memory sidecar.
  7. Let any memory projection sidecar implement derived-state storage and
    retrieval behind DerivedStateAttachment, not platform policy.
  8. Expose product/operator memory controls through AppKit.
  9. Prove the recall and promotion no-bypass paths in StackLab.

This gives the platform useful memory without granting memory systems hidden
authority.

Final Position

The platform position is:

Start as a monolith.
Keep the physical owner boundaries real.
Treat stochastic reasoning as volatile compute.
Let deterministic substrates own state transitions, evidence, review,
authority, replay, and revocation.
Scale later by splitting already-proven contract seams into cells.

The memory-specific failure mode is:

A semantic runtime reads and writes one memory sidecar around every outcome.

That pattern covers only a narrow implementation slice and does not define
platform memory.

The memory-specific design is:

Product/AppKit
  -> Mezzanine pack, lease, workflow, review, and promotion policy
      -> OuterBrain read-only semantic recall and context provenance
      -> Citadel authority and advisory planning memory
      -> Jido Integration lower facts and derived-state anchors
          -> optional memory sidecars that store rebuildable projections

Memory becomes safe when it is treated as governed context plus derived
projection, not as an agent-owned database. The stack already has the key
pieces. The missing memory-specific contracts should be formalized so
adapters and sidecars can participate without bypassing semantic, authority,
workflow, review, lower-fact, or product boundaries.

The same rationale drives the broader architecture. AppKit, Mezzanine,
OuterBrain, Citadel, Jido Integration, and Execution Plane are separate because
enterprise AI automation needs a place for each kind of truth, a way to prove
every crossing, and a path from local monolith to global cell deployment
without redefining what “truth” means in production.

Notes

  • Constructive criticism is welcomed, even if to say “overengineered slop”, at least that provides a useful data point.
  • The mermaid diagrams don’t seem to render here, and I am not for sure if the forum supports this. This link has the same contents as this post and shows the diagrams as rendered.