Hey folks,
I’m building Zed (not editor) a declarative BEAM deploy tool for FreeBSD and illumos that uses ZFS user properties as the state store (com.zed:version=1.4.2) and zfs rollback as the rollback mechanism.
No K8s, no etcd, no external state. It’s Apache 2.0, early days, ~2000 lines of Elixir so far: GitHub - borodark/zed: Declarative BEAM deployment on FreeBSD/illumos. ZFS properties as state store. No etcd, no YAML. · GitHub
Phase 1–4 are done (DSL, convergence engine, FreeBSD jails, multi-host via Erlang distribution). I’m now on Phase 5 and adding secrets support. The overall pipeline is pretty clear — DSL declares sources, a resolver runs on the target at converge time, resolved values land in a per-app 0600 env file that the release reads via System.get_env/1 in runtime.exs. Full design doc here: docs/SECRETS_DESIGN.md.
The reason I’m posting here is that I’ve got two forks in the road where I don’t want to pick solo, because the choices compound. Both affect what a real deploy looks like, and I’d rather hear from people who’ve done this in anger than commit to an answer and regret it.
The DSL shape (for context)
The motivating example is a shape I suspect many of you have run into: a BEAM app with several external integrations under one deploy — multiple broker-account credential pairs, a license key, a Phoenix secret_key_base, a distribution cookie.
app :broker_bot do
dataset "apps/broker_bot"
version "1.5.0"
cookie {:env, "BEAM_COOKIE"}
secrets do
broker_a_key {:env, "BROKER_A_API_KEY"}
broker_a_secret {:env, "BROKER_A_API_SECRET"}
broker_b_token {:file, "/var/run/secrets/broker_b.token"}
license_key {:env, "LICENSE_KEY"}
secret_key_base {:env, "SECRET_KEY_BASE"}
end
end
{:env, ...} and {:file, ...} are the MVP sources. The questions are about what else should ship in MVP, and what should stay out.
Fork 1 — age-encrypted files in-repo: MVP or Phase 5.1?
The idea: support {:file, "secrets/broker_a.age", mode: :age} so encrypted secrets can live in the git repo alongside the DSL. The target-side resolver shells out to age -d -i <keyfile> to decrypt. This is the standard NixOS / agenix / sops-age pattern.
Case for MVP: without it, a first real deploy that involves several credential pairs (e.g., a multi-account broker integration with a structured accounts.config) still relies on out-of-band secret delivery — operator SCPs a file, or a bootstrap script seeds it. That defeats half the point of declarative deploys.
Case for Phase 5.1: adds a binary dependency (age CLI, or vendoring the Rust age crate via Rustler). Introduces a whole new concern — where does the keyfile live on the target, who owns it, how is it rotated. Risk of shipping a bad first version of something that’s security-critical.
My current lean: defer to 5.1. MVP keeps secret files out of git entirely; Option A in the design doc is “opaque file on target, ship path only.” But that means the first real customer deploy still has a plaintext credentials file living on a host with no replication story.
What would change my mind: if someone here has shipped age-decryption in an Elixir deploy tool and has a reusable pattern, or if someone can point at a failure mode I’m not seeing with shell-out to age.
Fork 2 — should {:zfs_prop, "com.zed:key"} be a secret source at all?
The appeal: ZFS user properties travel with zfs send/receive. If a secret lives in a property, replicating the dataset replicates the secret for free. No separate secret-replication pipeline, no sops-per-host keys.
The problem: zfs get all is readable by anyone with access to the dataset, which on a multi-user FreeBSD box is broader than you’d like. Property values end up in history, backup tooling, and any zpool status-adjacent introspection path.
Option A: reject {:zfs_prop, ...} from the secrets block entirely. Keep it as a source for non-sensitive config only (node_name, version, feature flags).
Option B: allow it with a big compile-time warning and an opt-in allow_zfs_prop_secrets: true on the deploy. Trust the operator to know their threat model.
My current lean: Option A, hard reject. The convenience isn’t worth the footgun — “replicates for free” is exactly the kind of seductive default that bites at 3 AM.
What would change my mind: a use case where the operator genuinely controls all dataset access (single-user homelab, say) and wants the replication story without adding a second mechanism. Probably a real thing, but I’d rather make those users opt into something else than weaken the default.
Specific things I’d love input on
- If you’ve shipped secrets in an Elixir deploy (sys.config from envs,
{:system, ...}in config, sops, agenix-style flows) — what bit you? What would you want a new tool to not repeat? - Anyone shipped age in production, in an Elixir context? Was it a shell-out or a Rust NIF? Rotation story?
- For the zfs_prop question — is anyone actually doing this? Is there a homelab crowd I’m discounting?
- Am I missing a source kind that should be in MVP? (I’ve deliberately left off 1Password CLI, HashiCorp Vault, AWS Secrets Manager — they feel out of scope for a zero-infra tool, but push back if you think one belongs.)
Full design doc with threat model, module layout, and wire-up seams: zed/docs/SECRETS_DESIGN.md at main · borodark/zed · GitHub
Happy to move this to a GitHub Discussion if that’s a better venue — figured ElixirForum would get the most eyes for the “what would a seasoned BEAM person do” angle.
Cheers,
Igor
