Jiyi - Durable, vector-backed memory service for autonomous agents

Jiyi (记忆, jìyì — “personal memory” in Chinese) is an Elixir service that provides structured, retrievable memory for LLM-based agents. It stores three kinds of memory; episodic events, semantic facts, and per-session working state — backed by PostgreSQL with pgvector embeddings, and exposes both an HTTP API and an MCP server so agents can write and retrieve context across turns and sessions.

This was a thought I had and opted to materialize it, free to learn on new approaches and will be thrilled to get feedback.

Core features:

  • Three memory types: episodic events (timestamped observations), semantic facts (subject/predicate/object triples with validity windows), and working memory (per-session key/value state)
  • Procedural memory via git-backed playbook files, loaded at assembly time
  • Retrieval pipeline that fan-outs across stores, ranks by trust tier × recency × relevance, and respects a configurable token budget
  • Three visibility scopes: agent_private, session_shared, org_shared
  • Trust-tier clamping at the auth layer — per-agent keys and MCP session tokens cannot self-assert human_asserted provenance
  • A very simple skeleton for anomaly detection with the aim to isolate, quarantine and remove compositional injection attempts surgically rather than wiping the whole context.
  • MCP provider abstraction — swap server or client adapter via env var without touching business logic.

Requires Elixir ~> 1.17, OTP ~> 27, and PostgreSQL 16+ with the pgvector extension.


5 Likes

Update:

Jiyi also implements an optional local embedding server:

It loads BAAI/bge-base-en-v1.5 via Bumblebee + Nx + EXLA and serves POST /embed from inside the BEAM VM. Every semantic and episodic write gets a 768-dimensional vector, and writes fail fast if embedding generation breaks instead of silently storing NULL vectors.

It is opt-in because transformers are heavy: ~400 MB model download on first start, ~500 MB–1 GB RAM, and CPU-bound inference.