gtcode
Hi everyone,
I have been working on a large-scale, open-source platform for orchestrating and governing AI workloads on the BEAM (currently spanning various repositories under North-Shore-AI / nshkrdotcom).
Because AI agents are inherently non-deterministic and stateful, I wanted to build a system where governance precedes execution. Rather than letting LLMs directly execute code or API calls in a wild-west loop, every semantic turn, typed command, and external execution call passes through a strict, shared governance and durable-truth chain before it touches an outside system.
Much of the core idea, particularly the work in citadel came from work done by Mike Hostetler and his team. The jido_integration package was also created per Mike’s direction. I’m grateful to the Jido team for that opportunity.
In order to work on such a large scale endeavor, I’ve created a few helper projects to deal with cumbersome monorepos: blitz and weld. Blitz makes it easy to conduct monorepo-wide quality checks, and Weld will be used to assemble the various mix.exs projects in a monorepo into a single mix.exs project, combining the test suites and code, while also assembling shared documentation. Welding these monorepos will simplify publishing to Hex.
As the architecture progresses, I’d love to get some eyes on structural boundaries and OTP design philosophy from folks here who have distributed systems and BEAM expertise. I’m using this architecture to learn distributed systems design and implementation in practice. Constructive criticism is welcomed, especially since this is still early stage greenfield development.
High-Level Architecture
The platform is split into strict, structural ownership boundaries. No layer is allowed to bypass the layer below it.
[ Products / UIs / Workbenches ]
↓
[ Northbound Surface ]
* app_kit (Product-facing entry point)
* mezzanine (Universal business machine & operational-semantic engine)
↓
[ Brain Chain ] (Semantic → Typed → Governed)
* outer_brain (Semantic reasoning)
* citadel (Governance, policy, and intent shaping)
↓
[ The Spine ] (Durable Truth)
* jido_integration (Durable intake, auth, control plane, review truth)
↓
[ Execution Substrate ] (The "Hazmat Zone")
* execution_plane (Transport, placement, sandboxing, raw facts)
* Family Kits (REST, GraphQL, CLI/Subprocess, Python/Snakepit)
[ Foundational Substrate ] (Underlies everything)
* ground_plane (IDs, fences, leases, checkpoints, generic persistence)
Core BEAM / OTP Principles
To avoid turning this into an unmanageable “process soup,” I’ve adopted a few hard rules:
-
Strict Structural Ownership: The Execution Substrate handles transport and placement, but carries zero durable business meaning. The Spine owns durable truth. The Brain shapes intent.
-
Data & Contracts over Processes: Define stable data, pure compilers, reducers, and projectors first. Only wrap things in OTP processes (GenServers, Supervisors) where fault-tolerance, state recovery, or concurrency explicitly require them.
-
Traceability by Design: Because debugging AI is like forensic analysis on a dream, lineage and audit trails are first-class. Every action maps back to a governed decision.
Where I’d Love Your Input
While I am confident in the functional separation, distributed orchestration introduces specific challenges where I’d value expert insight from the community. LLM’s can certainly provide guidance on how to approach the design, tooling, and testing, but nothing beats feedback from a distributed systems engineer.
1. Distributed State & “The Spine”
jido_integration acts as the durable truth layer (intake, auth, control plane).
2. The Execution Substrate (“Hazmat Zone”)
The execution_plane isolates dangerous work (subprocess execution, Python bridges, API calls). I want to ensure that failures here cascade cleanly up to the Spine without taking down the governance layers
3. “Pure Core” vs OTP Overhead
I’m actively trying to keep business logic (in mezzanine and citadel) as pure functions, pushing side-effects to the edges. However, orchestrating multi-step LLM reasoning loops often requires suspending and resuming state.
4. Distributed Test Harness
stack_lab will be used to enable testing distributed systems on my local development machine and test environments, not as a general framework but specifically for this stack.
5. General Architectural Critique
Looking at the layer diagram above, do you spot any glaring anti-patterns or bottlenecks typical to BEAM distributed systems?
If you are interested in poking around the code, the core infrastructure lives across repos at nshkrdotcom. (Specific repos of interest might be execution_plane, citadel, and jido_integration).
While I am ultimately interested in ensuring the correctness of the code in granular detail, currently it’s expected that the code will have anti-patterns. These will be naturally resolved over time. So, I’m aware of this, but not looking for specific feedback on the minutia at this time, unless broadly relevant.
Here’s a whitepaper that describes the architecture and roadmap in more detail. The code on github is a bit out of date, pending completion/push of current phase efforts.
My goal is to nail down the big picture architecture, after which I’m willing to revamp/rebuild any library as needed. I’m willing to revisit the overall architectural boundaries if there’s a good reason. Thanks in advance for any insights, critiques, or war stories you’re willing to share!
Trending in Questions
Other Trending Topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixirconf-us
- #ai
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #hex
- #security
- #metaprogramming











Showing Posts 1 to 2- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
gtcode
Brain/Spine Platform And Governed Memory Deep Dive 1 of 2
Date: 2026-04-21
Status: technical architecture note
AI governance platforms need a deterministic control substrate around
stochastic reasoning. This stack separates product surfaces, operational
workflow truth, semantic reasoning, authority compilation, lower acceptance,
and raw execution mechanics so AI-generated work can become governable
enterprise state instead of opaque agent output.
Memory belongs in that same control model. It is not a convenience feature or
an implementation detail of retrieval. Memory is a future control input. If a
remembered fragment can influence a mutation, approval, plan, tool call,
customer-facing answer, or operator decision, then that fragment participates
in the control plane and must carry provenance, scope, staleness, revocation,
and ownership.
Primary repository surfaces:
Executive Thesis
This architecture is best understood as a monolith-compatible, physically
bounded control platform for agentic execution.
It is not a conventional monolith where every subsystem can reach into the same
database and call any internal module. It is also not a premature microservice
mesh. The intended first deployment can run as a single node or single
application assembly, but the boundaries are real: public DTOs, repo/package
ownership, no-bypass gates, lease scopes, idempotency keys, authority packets,
fences, claim-checks, audit facts, and projection contracts. The system pays
the boundary tax early so that later distribution is a deployment change and
capacity exercise, not an architectural rewrite.
The platform separates four concerns that are often collapsed in simpler agent
systems:
The named stack components map to those concerns:
workflows, leases, decisions, evidence, audit, projections, archival, and
operator/control-room state.
journals, strategy checkpoints, normalized semantic failures, and reply
publication.
authority, policy compilation, local continuity, planning context, and
Brain-to-Spine handoff.
connector contracts, credential and auth lifecycle, lower facts,
tenant-scoped substrate reads, review projections, and sidecar anchors.
process/HTTP/JSON-RPC lanes, attach/stream pressure, and sandbox-adjacent
execution concerns.
Memory integration falls out of this architecture. Memory should not be
modeled as one central database and should not be assigned wholesale to a
single sidecar. In this stack, memory integration is a governed pair of planes:
semantic reasoning.
durable outcomes into future recall surfaces.
The recall plane is where memory may influence a model. The promotion plane is
where outcomes become eligible to inform later runs. Those planes must be
separate because an AI system that can freely read and write its own memory
inside the same reasoning loop has created an unreviewed control path.
The stack already contains the beginning of that separation:
memory, RAG engines, and model-specific cognition as platform truth.
record shape.
adapter activation gates, workflow/evidence boundaries, and tenant-scoped
lower-read leases.
attachment seam for higher-order sidecars.
those lower owners.
That shape is more sophisticated than assigning memory wholesale to one
sidecar. A memory projection sidecar can be valid, but it should be one
implementation of the promotion plane, not the definition of memory integration
for the platform.
Why Serious Engineers Will Challenge This
The strongest objection to this architecture is correct: it is complex.
A battle-tested distributed systems engineer will ask why this is not just:
That simpler architecture is often the right answer for a small product. It is
not obviously the right answer for an AI governance platform where stochastic
model output can trigger external side effects, update durable enterprise
state, cross tenant boundaries, enter audit trails, or become context for later
automated action.
The platform should accept complexity only where it protects an invariant.
Every boundary in this stack needs to justify itself by preserving at least one
of these properties:
process what happened
affordances
invalidated
truth
If a boundary does not preserve one of those properties, it is probably
ceremony. If it does, it is not ceremony; it is the point of the system.
The core design wager is that AI agents should be treated as volatile,
non-deterministic compute nodes inside a deterministic governance substrate.
The model may reason, summarize, plan, and propose. The deterministic platform
owns identity, leases, state transitions, idempotency, review state, audit,
projection, revocation, and replay. That split is the reason the architecture
looks heavier than a normal agent loop.
The complexity ledger should be explicit:
This is also the standard for deleting complexity. If a seam cannot name an
invariant and a proof mechanism, it should be simplified or removed. The
architecture is defensible only if its boundaries are continuously executable,
not merely described.
Why Memory Needs Governance
Memory governance is not about making a vector database bureaucratic. It is
about recognizing what memory does in an agentic system.
In a conventional application, cached context is often an optimization. In an
AI execution platform, memory can become a hidden policy input. A remembered
preference can change a tool call. A recalled incident can bias a review. A
summarized customer conversation can influence a future deployment. A stale
fragment can cause a model to reapply an obsolete instruction. A cross-tenant
fragment can leak data. A provider-native memory feature can persist secrets
outside the platform’s audit and revocation model.
So the question is not “why govern memory?” The question is “when may a stored
or retrieved fact influence a future action?” The answer must be explicit.
A governed memory fragment needs to carry at least:
session produced it
policy context can see it
projection, operator annotation, or incident/forensic evidence
invalidated
truth
Without those fields, memory is an untyped side channel. A side channel that
influences future actions is a bypass.
Platform Plane Map
This diagram is the simplest mental model:
The spine-to-hazmat split is important here because raw execution is below the
governance and acceptance layers. Products and semantic runtimes do not scrape
it directly.
Monolith First, Physically Bounded From Day One
The initial deployment should be allowed to run as a monolith.
That does not mean the architecture should be a monolith internally. A useful
monolith for this system has physical boundaries inside the codebase:
This lets the first deployment avoid premature distributed consensus while
still proving the system will survive distribution later.
The practical rule is: local function calls are fine during the monolith
phase, but the payloads must already be serializable contract payloads with
stable logical identity. If an internal handoff requires a process identifier,
shared mutable state, unscoped direct SQL, or a live in-memory object, it is
not a future distributed seam.
End-To-End Operating Model
A governed run should be explainable without naming implementation details.
surface.
posture, decision gates, and configured context sources.
semantic surface with context-source declarations and runtime bindings.
records provenance and diagnostics, and produces semantic output or a
normalized semantic failure.
context, applies authority and policy, and emits a governed handoff.
runtime scope, records durable lower facts, and exposes bounded reads.
Spine.
archival truth.
context admission before reasoning, and evidence-backed derived projection
after durable outcomes.
The sequence is illustrative. Many product reads stop at AppKit and Mezzanine.
Many typed commands do not need semantic reasoning. Some operator actions are
local Mezzanine workflow signals. When a request crosses layers, the boundary
shape stays predictable.
Brain, Spine, And The Middle Layer
The architecture uses “Brain” and “Spine” as a durability and responsibility
split, not as branding.
The Brain is where intent is interpreted and governed:
construction, provider-neutral semantic failures, and semantic reply
publication.
continuity, policy compilation, planning context, and Brain-to-Spine
handoff.
The Spine is where accepted lower work becomes durable operational fact:
credential lifecycle, lower receipts, lower events, lower artifact refs,
tenant-scoped readbacks, review projections, and derived-state anchors.
Mezzanine sits above and around both as the neutral operational layer:
activation, workflow runtime, execution ledger, lifecycle, decisions,
reviews, evidence, projections, operator control, audit, archival, and
lower-read leasing.
That middle layer is why the stack can support many products without each
product becoming a mini control plane. Products call AppKit. AppKit delegates
to Mezzanine. Mezzanine composes the Brain and Spine according to pack,
workflow, policy, tenant, and installation state.
The Hyperscalar Enterprise Shape
The long-term enterprise version of this architecture is cell-based.
An initial monolith can prove the contracts. A hyperscalar deployment can split
by region, tenant class, product family, data residency zone, execution risk,
or workload tier without changing the logical ownership model.
A plausible global shape is:
projections
compilation
regions
production-like seams
In that world, AppKit remains the product boundary, Mezzanine remains the
operational substrate, OuterBrain remains semantic, Citadel remains authority,
Jido Integration remains lower acceptance, and Execution Plane remains raw
runtime mechanics. The implementation may become distributed, but ownership
does not dissolve.
The enterprise value is not only scale. It is explainability under pressure.
When a global customer asks why an automated agent changed something, the
platform should be able to reconstruct:
That is the difference between an agent demo and an enterprise control plane.
Memory Thesis
Memory in a governed agentic platform is not a truth source. It is an
admissible context source with provenance, scope, staleness, and revocation.
The platform truth sources remain:
Citadel
control-room facts owned by Mezzanine
Jido Integration and Execution Plane
Memory may summarize those truths, index them, rank them, cluster them, or make
them retrievable, but it must not replace them.
A memory fragment is safe to use only when the consumer can answer:
projection?
The answer must be carried as structured data, not as tribal knowledge in an
adapter.
Sidecar-Only Memory Is Insufficient
A memory projection sidecar can be a compliant implementation detail, but it is
too narrow to define platform memory.
Valid sidecar constraints:
as platform truth.
assembly.
core/contractsseam.DerivedStateAttachmentexists precisely so sidecars canpersist derived enrichments while anchoring them to
SubjectRef,EvidenceRef, andGovernanceRef.control-plane ledger.
Limits of a sidecar-only model:
defines memory-related seams across OuterBrain, Citadel, Mezzanine, and Jido
Integration.
does not cover all memory classes. Citadel has advisory memory records.
OuterBrain has semantic-session context packs and provenance. Mezzanine has
context-source declarations and trusted adapter activation gates.
Durable memory promotion may need evidence from OuterBrain semantic
validation, Citadel authority, Mezzanine decisions or reviews, Jido lower
outcomes, and operator suppression or revocation records.
under Jido Integration. It should require every memory implementation to
satisfy the same evidence, governance, recall, and revocation contracts.
A memory projection sidecar is therefore one participant in the promotion
plane. It is not the platform’s memory architecture.
Stack Responsibilities
OuterBrain
OuterBrain is the provider-neutral semantic-runtime gateway above Citadel. It
owns durable semantic-session truth, prompt and strategy shaping, context-pack
construction, tool-manifest snapshots, semantic quality checkpoints,
provider-neutral semantic failure carriers, live semantic-session fencing, and
restart-safe reply publication.
The important memory-relevant contracts are:
OuterBrain.ContextAdapterReadOnly.v1OuterBrain.SemanticContextProvenance.v1OuterBrain.SemanticDuplicateSuppression.v1OuterBrain.SemanticActivityNormalized.v1OuterBrain.SemanticActivityPayloadBoundary.v1Platform.SuppressionVisibility.v1OuterBrain.Contracts.ReplyPublicationOuterBrain.Prompting.ContextPackOuterBrain.Prompting.ContextFragmentOuterBrain.Prompting.ContextAdapterRegistryOuterBrain’s memory stance is strict:
refs, claim-check refs, and bounded routing facts.
suppression.
prompts, raw context packs, raw artifacts, or workflow history as northbound
DTO truth.
evaluation.
In other words, OuterBrain is the recall orchestrator and semantic provenance
producer. It is not the durable memory store.
The context-pack path is especially important.
ContextPack.build/3takes semantic state, declared context sources, runtime bindings, and an
adapter registry. For each configured source, it sends a bounded request to the
registered adapter and expects a list of structured fragments. Each fragment
must include content, provenance, staleness, and metadata. The pack records the
source report, adapter key, fragment count, merge strategy, and degraded/error
state.
This is the platform’s implemented recall interface.
Citadel
Citadel is the host-local Brain kernel. It accepts structured ingress at the
kernel boundary, compiles authority and planning decisions, preserves host-local
session continuity, and projects Brain-authored packets toward the shared
Jido Integration contracts layer.
Citadel owns:
Citadel explicitly does not own durable lower run, attempt, review, approval,
artifact, credential, provider runtime, or raw natural-language truth.
Citadel currently contains a memory-relevant value:
Citadel.MemoryRecordThat record has:
memory_idscope_refsession_idkindsummarysubject_linksevidence_linksexpires_atconfidencemetadataCitadel also exposes
Citadel.Ports.Memory, an advisory memory seam with:put_memory_record/1get_memory_record/2rank_memory_records/1The port contract is deliberately advisory. A write can be
:stable_put_by_idor:best_effort. Reads are keyed lexically bymemory_idor ranked by scope, session, kind, and limit. This is useful forhost-local planning and summaries, but it is not a ledger.
Citadel therefore provides a second memory class:
Both may point at the same source facts, but neither owns those facts.
Mezzanine
Mezzanine is the neutral business-semantics and workflow substrate. It owns the
platform machinery that decides which contexts can be activated, which lower
facts can be read, which workflows are durable, and which reviews or operator
actions are authoritative.
Memory-relevant Mezzanine seams include:
Mezzanine.Pack.ContextSourceSpecMezzanine.Authoring.BundleMezzanine.ConfigRegistryMezzanine.WorkflowRuntimeMezzanine.WorkflowRuntime.ActivitySideEffectIdempotencyMezzanine.LeasingMezzanine.IntegrationBridge.ReadDispatcherMezzanine.ControlRoom.IncidentBundleMezzanine.ControlRoom.IncidentExportBundleMezzanine.ControlRoom.ForensicReplayMezzanine.DecisionsMezzanine.ReviewsContextSourceSpecis the pack-level declaration that a read-only source canfeed OuterBrain context assembly. It carries:
source_refdescriptionbinding_keyusage_phaserequired?timeout_msschema_refmax_fragmentsmerge_strategyThe key point is that a product pack can declare memory-like context sources
without owning memory runtime code. The Mezzanine config registry validates
context adapter descriptors against trusted registry configuration before
activation. Unknown adapters, missing bindings, invalid descriptors, stale
installation revisions, invalid policy refs, and pack-authored platform
migrations are rejected before runtime activation.
That gives the platform a supply-chain and installation gate for memory
adapters.
Mezzanine also controls the lower-fact read path. Its integration bridge
builds a typed
Jido.Integration.V2.TenantScopefrom authorized context,checks execution lineage, rejects lower-id overrides, and calls the bounded
Jido Integration
LowerFactssurface. Product code is expected to enterthrough AppKit and carry Mezzanine read leases plus authorization scope.
For memory, this means lower facts can be recalled only through:
Mezzanine is therefore the policy, lease, workflow, and evidence envelope
around memory. It should not become a vector store, but it should govern which
memory adapters are trusted, which facts can be read, and which derived memory
updates can be promoted.
Jido Integration
Jido Integration owns the lower acceptance gateway and public lower contracts
for connector discovery, invocation, auth lifecycle, runtime targeting, review
packet projection, lower facts, and substrate-facing reads.
Memory-relevant Jido Integration contracts include:
Jido.Integration.V2.SubjectRefJido.Integration.V2.EvidenceRefJido.Integration.V2.GovernanceRefJido.Integration.V2.DerivedStateAttachmentJido.Integration.V2.ReviewProjectionJido.Integration.V2.TenantScopeJido.Integration.V2.LowerFactsJido.Integration.V2.SubstrateReadSliceExecutionEvent.v1ExecutionOutcome.v1The lower-facts surface is intentionally bounded:
It does not assemble product projections, operator packets, memory summaries,
or review aggregates.
DerivedStateAttachmentis the key memory-adjacent contract. It states thathigher-order repos may persist their own enrichments, memories, lineage, and
scores, but those records must remain anchored to node-local source truth
through explicit subject, evidence, and governance refs.
That is the right sidecar contract for a future memory projection service:
SubjectRef,EvidenceRef, andGovernanceRefBut Jido Integration should not be the semantic recall owner. Its role is lower
truth and sidecar anchoring.
AppKit
AppKit is the northbound application-surface and product-boundary enforcement
point. Product repos must use AppKit surfaces for governed writes, operator
reads, reviews, installation bootstrap, semantic assist, trace lookup, and
leased lower read access.
For memory integration, AppKit matters because it prevents product apps from
calling Mezzanine, Citadel, Jido Integration, or Execution Plane directly for
governed behavior. A product can author a pure
Mezzanine.Packmodel contract,including context source declarations, but runtime behavior must flow through
AppKit and the lower owner seams.
That means memory is not allowed to become a product-local bypass path.
Execution Plane
Execution Plane owns lower runtime execution and effect projection mechanics.
It should not own semantic memory. It may produce lower events, artifacts,
process output, HTTP results, JSON-RPC results, and runtime state that later
become eligible for memory projection, but memory sidecars should consume those
through Jido Integration and Mezzanine, not through direct runtime scraping.
Ground Plane, StackLab, And AITrace
Ground Plane supplies generic lower contracts and projection helpers. It is
not the memory owner.
StackLab is the proving harness. It should prove that memory recall and memory
promotion do not bypass AppKit, Mezzanine leases, Citadel authority, OuterBrain
semantic boundaries, or Jido Integration tenant scope.
AITrace can be useful as an observability layer for reasoning spans, tool
calls, and trace visualization. It should be treated as trace evidence or
developer observability, not as memory truth.
Ownership Diagram
Memory Classes
The word “memory” hides several different data classes. The platform should
name them separately.
1. Semantic-Session State
Owner: OuterBrain
Purpose:
Allowed to influence:
Not allowed to replace:
2. Advisory Planning Memory
Owner: Citadel, with backing implementation behind
Citadel.Ports.MemoryPurpose:
hints
memory_idfor stable lookupAllowed to influence:
Not allowed to replace:
3. Pack-Declared Context Sources
Owner: Mezzanine for declaration and activation; OuterBrain for runtime
context assembly
Purpose:
workspace_memoryto trusted adapter keysAllowed to influence:
Not allowed to replace:
cont…
gtcode
Brain/Spine Platform And Governed Memory Deep Dive 2 of 2
…cont
4. Lower-Fact Projections
Owner: Jido Integration and Execution Plane for lower facts; Mezzanine for
authorized read access
Purpose:
and traces
Allowed to influence:
Not allowed to replace:
5. Derived Memory Projection
Owner: future memory sidecar or product-approved memory projection service,
anchored by Jido Integration and Mezzanine evidence
Purpose:
durable source records
Allowed to influence:
Not allowed to replace:
6. Operator And Review Memory
Owner: Mezzanine decisions/reviews/control-room surfaces, possibly projected
into a sidecar
Purpose:
and runbook links
Allowed to influence:
Not allowed to replace:
Two-Plane Model
Recall Plane
The recall plane is read-only.
It takes memory from an external or sidecar source and admits it into semantic
reasoning as context fragments. It must answer these questions before a
fragment reaches a prompt:
trace?
OuterBrain’s current context adapter and context fragment shape is the concrete
implementation example.
Promotion Plane
The promotion plane is write-capable, but it is not allowed to write from the
middle of model reasoning directly into future memory.
It takes durable source records and constructs memory candidates. A candidate
can become a memory projection only after policy, provenance, and review rules
allow it.
The promotion plane should answer:
artifact index, policy note, or incident note?
actor-bound, session-bound, or global?
Jido Integration’s
DerivedStateAttachmentis the concrete anchor for thisplane, but the promotion policy itself belongs above the sidecar, primarily in
Mezzanine and Citadel.
Recall Flow
Important properties:
Mezzanine activation accepted its descriptor and OuterBrain calls it under a
read-only contract.
authorization and Jido Integration
TenantScope.source and governance references.
reason from compact references rather than raw prompt or raw context blobs.
Promotion Flow
Important properties:
not mutate durable memory directly during reasoning.
promotion still needs evidence and policy.
decisions, audit, leases, and installation revisions.
OuterBrain, Citadel, And Mezzanine Symbiosis
The memory architecture makes sense only when these three layers are treated as
complementary.
OuterBrain As Semantic Recall Boundary
OuterBrain answers:
review-worthy?
OuterBrain must be close to the model because it owns prompt assembly and
semantic validation. It must not own memory truth because that would let the
semantic layer rewrite the evidence that later semantic layers consume.
Citadel As Authority And Advisory Context Boundary
Citadel answers:
egress profile, workspace profile, and boundary class apply?
Citadel is where advisory memory belongs because it is host-local, planning
oriented, and scoped by policy context. It should not become the durable memory
projection owner because it does not own lower outcomes, workflow decisions, or
review ledgers.
Mezzanine As Workflow, Lease, And Promotion Boundary
Mezzanine answers:
result?
Mezzanine should be the promotion coordinator because it can join semantic
facts, authority refs, lower facts, decisions, audit, archival, and operator
projections without making any one runtime the memory owner.
Symbiosis Summary
The rule of thumb:
promoted.
Proposed Standard: Governed Memory Context Protocol
This section generalizes the stack into a possible standard for memory
integration in any serious AI governance platform.
The standard is not “install a vector database.” The standard is a set of
seams that make memory safe, reviewable, revocable, and rebuildable.
Standard Goals
A compliant memory integration should:
Required Components In A Generic Monolith
Even a monolith AI agent runtime needs these logical components if memory is
going to be governed:
scope.
and emits semantic provenance.
retries, and audit events.
provider actions.
fragments.
memory candidates or promoted memories.
promotion.
and derived memory records.
truth is revoked, corrected, deleted, or policy changes.
denied.
or tenant-sensitive payloads.
The nshkr stack distributes these components across repos. A monolith can keep
them in one codebase, but it still needs the same boundaries.
Standard Data Contracts
The following contract set defines the generic standard. The names are
illustrative, but the fields are the important part.
1.
MemoryContextSourceDescriptorDeclares that a memory source may be used for recall.
Required fields:
source_refbinding_keyadapter_refusage_phaseschema_refmax_fragmentstimeout_msmerge_strategyrequiredtrusted_registry_refredaction_policy_refallowed_read_resourcesdenied_write_resourcesEquivalent stack seams:
Mezzanine.Pack.ContextSourceSpecOuterBrain.ContextAdapterReadOnly.v12.
MemoryReadRequestThe request sent to a context adapter.
Required fields:
tenant_refinstallation_refworkspace_refproject_refenvironment_refactor_refsession_reftrace_idsemantic_refsource_refbinding_keyusage_phaseschema_refmax_fragmentsobjectiveunresolved_questionscommitmentsrefsallowed_read_resourcesmutation_permissions: []Equivalent stack seams:
OuterBrain.Prompting.ContextPacksource requestOuterBrain.ContextAdapterReadOnly.v13.
MemoryFragmentA piece of recall context admitted into reasoning.
Required fields:
fragment_idschema_refschema_versioncontentprovenancestalenessredactionscopesource_refsevidence_refsgovernance_refsconfidenceexpires_atmetadataEquivalent stack seams:
OuterBrain.Prompting.ContextFragmentCitadel.MemoryRecordJido.Integration.V2.EvidenceRefJido.Integration.V2.GovernanceRef4.
MemoryContextProvenanceRecords how recall influenced semantic reasoning.
Required fields:
semantic_refprovider_refmodel_refprompt_hashcontext_hashinput_claim_check_refoutput_claim_check_refprovenance_refsnormalizer_versionredaction_policy_reftrace_ididempotency_keyEquivalent stack seam:
OuterBrain.SemanticContextProvenance.v15.
MemoryCandidateA candidate memory produced after semantic validation, tool execution, lower
outcome, review, or operator annotation.
Required fields:
candidate_idcandidate_kindsummarycontent_refsource_subject_refssource_evidence_refssource_governance_refssemantic_refsauthority_refsworkflow_refslower_refsconfidencerisk_bandretention_classproposed_scoperedaction_policy_refidempotency_keyEquivalent stack seams:
6.
MemoryPromotionDecisionThe policy or human decision that accepts, denies, waives, escalates, or
expires a memory candidate.
Required fields:
decision_idcandidate_iddecisionreasonactor_refpolicy_refreview_refquorum_refsafe_actionidempotency_keytrace_idEquivalent stack seams:
Mezzanine.DecisionsMezzanine.ReviewsJido.Integration.V2.GovernanceRef7.
MemoryProjectionEventThe command/event sent to a memory sidecar after promotion is approved.
Required fields:
projection_event_idprojection_kindsubject_refevidence_refsgovernance_refsprojection_payloadprojection_hashschema_refscoperetention_classredaction_policy_refrebuild_policyidempotency_keyEquivalent stack seam:
Jido.Integration.V2.DerivedStateAttachment8.
MemoryInvalidationRevokes, suppresses, quarantines, expires, or deletes memory.
Required fields:
invalidation_idmemory_refsource_refreasonscoperevocation_refcache_invalidation_refpost_revocation_attempt_refoperator_visibilityeffective_attrace_idEquivalent stack seams:
Standard Component Diagram
Concrete Future Direction For This Stack
Direction 1: Treat Memory As A First-Class Pack Context Source
Mezzanine already has
ContextSourceSpec. That should become the standardconfiguration path for recall:
ContextFragmentvalues with provenance and stalenessThis lets products opt into memory without bypassing product boundaries.
Direction 2: Promote A Formal Memory Fragment Contract
OuterBrain’s
ContextFragmentis close to the right shape, but future memoryneeds stronger fields:
The current
ContextFragmentcan remain the semantic-facing minimal shape, buta memory-specific contract should normalize these fields before fragments enter
the context pack.
Direction 3: Keep Citadel Memory Advisory
Citadel.MemoryRecordshould stay host-local and advisory.It is useful for:
It should not be used as:
If Citadel memory needs to become durable and product-visible, promotion should
flow through Mezzanine and a sidecar projection, not by expanding
Citadel.Ports.Memoryinto a platform ledger.Direction 4: Keep Memory Projection Sidecars Derived And Bounded
A future memory projection sidecar should likely:
DTOs explicitly approved for memory
DerivedStateAttachmentfor every promoted recordpolicy
provenance
It should not:
TenantScopeDirection 5: Add A Mezzanine Memory Promotion Coordinator
Mezzanine is the right place to coordinate promotion because it can join:
A future coordinator could expose commands like:
propose_memory_candidate/1approve_memory_candidate/2deny_memory_candidate/2publish_memory_projection/1invalidate_memory_projection/1rebuild_memory_projection/1Those commands should emit compact refs and never raw provider payloads.
Direction 6: Add AppKit Memory Surfaces
Products should not directly call memory sidecars. AppKit should eventually
own the northbound product/operator surfaces:
The AppKit surface should talk to Mezzanine, not to a sidecar directly.
Direction 7: Prove No-Bypass In StackLab
StackLab should prove:
TenantScopeapproval
Suggested First Formal Memory Contract Set
This stack could converge on these concrete names:
Platform.MemoryContextSource.v1Platform.MemoryReadRequest.v1Platform.MemoryFragment.v1Platform.MemoryRecallReport.v1Platform.MemoryCandidate.v1Platform.MemoryPromotionDecision.v1Platform.MemoryProjectionEvent.v1Platform.MemoryInvalidation.v1Platform.MemoryProjectionReceipt.v1Possible ownership:
MemoryContextSource,MemoryCandidate,MemoryPromotionDecision,MemoryProjectionEvent,MemoryInvalidation, andMemoryProjectionReceipt.MemoryReadRequest,MemoryFragment, andMemoryRecallReportat the semantic boundary.Citadel.MemoryRecordintoMemoryFragmentwhen advisorymemory is admitted to reasoning.
DerivedStateAttachment,SubjectRef,EvidenceRef,GovernanceRef, and lower facts as the lower sidecar anchor.contracts.
The key is not the exact module names. The key is that recall, promotion, and
invalidation become explicit contracts.
Review Checklist For Any Memory Adapter
Before a memory adapter is accepted, reviewers should ask:
hashes?
Review Checklist For Any Memory Projection Sidecar
Before a memory projection sidecar is accepted, reviewers should ask:
them?
or approval truth as its own truth?
Open Questions
Who Owns The First Memory-Specific Contract Package?
The likely answer is Mezzanine for promotion and installation policy, with
OuterBrain owning the semantic read boundary. But a small shared package might
be justified if the contracts need to be consumed by Citadel, Mezzanine,
OuterBrain, AppKit, and a future memory sidecar without creating cycles.
Should Citadel
MemoryRecordBe Mapped Or Replaced?Citadel.MemoryRecordis useful and already advisory. It may be enough to mapit into a generic
MemoryFragmentat the adapter boundary. Replacing it tooearly could blur Citadel’s host-local role.
What Counts As A Memory Promotion?
Some records are harmless summaries. Others encode user preferences, operator
decisions, business facts, or safety-sensitive instructions. The platform needs
promotion classes:
What Is The Revocation Model?
Memory revocation must handle:
The revocation state should be visible to OuterBrain recall, Mezzanine
operator surfaces, and any sidecar projection.
How Are Embeddings Governed?
Embeddings are derived state. They can leak source content and should carry:
The platform should not treat an embedding index as harmless just because it is
not human-readable.
How Are Conflicts Resolved?
Memory will conflict with source truth, current policy, or newer memory. The
standard should define precedence:
OuterBrain should never resolve conflicts silently inside prompt assembly.
Conflicts should produce degraded context, review-required routing facts, or
operator-visible suppression.
Should Memory Candidates Be Produced By OuterBrain?
OuterBrain is close to semantic summarization and can produce useful candidate
summaries. It should be allowed to propose candidates, but not to promote them
directly. Promotion requires Mezzanine policy and evidence joins.
Should Memory Sidecars Read Mezzanine Or Jido Integration?
For lower execution evidence, the sidecar should read through Jido Integration
lower facts with tenant scope, usually under Mezzanine authorization. For
workflow decisions, reviews, and promotions, it should receive Mezzanine
projection events rather than scraping Mezzanine stores.
Where Does Operator Search Live?
Operator search over memory should probably surface through AppKit and
Mezzanine so products see governed DTOs. The sidecar can own the index and
search mechanics, but not the product-facing authority.
What Should Be In CI?
No-bypass scans should eventually assert:
Practical Near-Term Architecture
The near-term architecture is:
Mezzanine.Pack.ContextSourceSpecas the product-pack declaration formemory recall sources.
adapter descriptors.
OuterBrain.Prompting.ContextPackandOuterBrain.ContextAdapterReadOnly.v1as the first recall runtime.Citadel.MemoryRecordinto context fragments only through an adapter,preserving advisory status and scope.
Jido.Integration.V2.TenantScopefor anylower-fact-backed memory recall.
memory sidecar.
retrieval behind
DerivedStateAttachment, not platform policy.This gives the platform useful memory without granting memory systems hidden
authority.
Final Position
The platform position is:
The memory-specific failure mode is:
That pattern covers only a narrow implementation slice and does not define
platform memory.
The memory-specific design is:
Memory becomes safe when it is treated as governed context plus derived
projection, not as an agent-owned database. The stack already has the key
pieces. The missing memory-specific contracts should be formalized so
adapters and sidecars can participate without bypassing semantic, authority,
workflow, review, lower-fact, or product boundaries.
The same rationale drives the broader architecture. AppKit, Mezzanine,
OuterBrain, Citadel, Jido Integration, and Execution Plane are separate because
enterprise AI automation needs a place for each kind of truth, a way to prove
every crossing, and a path from local monolith to global cell deployment
without redefining what “truth” means in production.
Notes