Yeah. I wanted to be proactively battle testing it against some real use-cases.
Excited to fill the main gap: recover of long running/stale steps. Once that’s in place, and it’s using Jido journals for persisting and reconstructing runs I’d say the work on the alpha should be settled.
Quick update on Rift:
I’m moving toward a full end-to-end demo flow:
Recent progress: Rift is now shaping up as a properly embeddable Phoenix/LiveView app, with host-defined case types/forms and separately mountable operator/originator routes. Next pieces are durable case persistence, wiring submissions into workflow starts, then operator decisions.
After the happy path works, I want to stress-test duplicate cases, stale operator actions, unresolved cases, failed side effects, and similar runtime edge cases.
More advanced examples could include an AI agent triaging tickets, labeling them, or validating included documents before operator review. The use cases still feel pretty wide open.
This also leaves the door open for human in the loop on auto started workflows (cron).
This is nice , it could serve by itself as simple incident or ticketing system with activity timeline showing the actions/comments/…etc
Yeah. Glad things are fitting together. Ticketing seems a very common theme specially for internal human in the loop.
Small architecture update on the durable delivery / long-running step recovery side.
Earlier I framed the remaining runtime gap mostly as Squid Mesh needing heartbeat-backed recovery for claimed long-running work. I still think that recovery path matters, but I’m trying to keep the layers cleaner:
I’m currently evaluating Bedrock / bedrock_job_queue for that lower delivery layer, because the interesting property is not just “another queue”. The more interesting property is having workflow state and queued work participate in the same transactional substrate, so a workflow transition and the job dispatch for the next step can commit together.
That would keep Squid Mesh focused on workflow semantics instead of becoming a queue/runtime mechanics project.
Still exploratory, and the public API should stay Squid Mesh-native. But the direction I’m leaning toward is:
Jido
|> SquidMesh
|> IntentLayer # if useful as intent-domain layer
|> Bedrock.JobQueue
|> Bedrock/FDB
Curious if this boundary makes sense to folks who have built durable workflow systems: should heartbeat/lease recovery live entirely in the delivery substrate, with the workflow engine only interpreting durable outcomes?
For more, check the Discord channel
Small follow-up on the execution layer side.
I’ve been spiking bedrock_job_queue as an alternative executor backend for Squid Mesh. The initial motivation was to see whether Bedrock could cover the parts a workflow runtime eventually cares about beyond “enqueue a job”: delayed delivery, retries, dead-lettering, worker claims, lease extension/heartbeat, and stale worker recovery.
So far the fit looks promising.
The most interesting part is that Bedrock’s lease model maps very naturally to workflow execution. In a workflow engine, a runnable step is not just a background job. The runtime eventually needs to reason about whether work was claimed, whether the worker is still alive, whether the lease expired, whether the step completed, and whether failure should retry or become terminal.
The spike currently covers:
There were some integration rough edges, mostly around using newer Bedrock Job Queue changes and understanding the low-level keyspace/lease APIs, but nothing that felt like the wrong abstraction.
The bigger takeaway for me is that Bedrock seems like a strong match for clustered workflow runtimes because the storage layer is already oriented around distributed coordination. Squid Mesh still has its current persistence/runtime path, but this spike is pushing the architecture toward clearer boundaries:
That separation feels important if Squid Mesh is going to support different execution backends
without baking one job system into the workflow model.
i.e.
+-----------------------------------------------------+
| Squid Mesh |
+-----------------------------------------------------+
| Public API: start_run / inspect_run / explain_run |
+-----------------------------------------------------+
|
v
+-----------------------------------------------------+
| Squid Mesh Runtime |
+-----------------------------------------------------+
| Plans work, applies results, retries, pauses, |
| cancels, completes, inspects, and explains |
+-----------------------------------------------------+
|
v
+-----------------------------------------------------+
| Jido Journals |
+-----------------------------------------------------+
| Durable workflow facts: runs, attempts, claims, |
| heartbeats, completions, failures, terminal state |
+-----------------------------------------------------+
| ^
v |
+----------------------------+ +----------------------------+
| SquidMesh.Executor | | SquidMesh.Executor.Leases |
+----------------------------+ +----------------------------+
| enqueue jobs / cron / delay| | claim / heartbeat / finish |
+----------------------------+ +----------------------------+
| |
+-------------+------------+
|
v
+----------------------------------------------------+
| Backend Adapter |
+----------------------------------------------------+
| Queue: enqueue, delay, cron delivery |
| Lease: claim, heartbeat, expiry, complete/fail |
+----------------------------------------------------+
|
v
+----------------------------------------------------+
| Backend Storage |
+----------------------------------------------------+
| Jobs, leases, worker liveness, delivery metadata |
+----------------------------------------------------+
I don’t think this means relational DBs are a bad fit for workflow engines in general. Postgres + Oban is still a very strong default (we’d just need to implement the leasing behavior).
The distinction I’m seeing is between workflow facts and workflow execution coordination. Workflow facts/history/inspection fit relational storage well. But the executor side starts looking less relational once you care about claims, leases, heartbeat extension, stale worker recovery, delayed visibility, and multi-node ownership.
That is where Bedrock feels like a better-shaped tool: the queue/lease layer is naturally a distributed coordination problem, not just a table of jobs.
New release is up!
Squid Mesh has moved further into the journal-backed runtime, and the public surface now reflects that direction more clearly.
The broader direction is to keep the runtime readable and inspectable first, then build toward read-only visual workflow preview, and eventually visual editing on top of the same model.
TLDR: Runtime it’s fully switched over to Jido, you can plug Bedrock as executor and have leasing as a bonus!
Only one feature left for later: expand existing workflows ( Add dynamic workflow graph expansion after the Jido-native core · Issue #141 · dark-trench/squid_mesh · GitHub )
Will re-visit project status and future goals to add more features in the roadmap. I’ve been contacted by an early stage project that is interested on using squid-mesh but needs authored workflows and visual editing so i’ll see what’s the sweet spot to give them the primitives to support that as i harden the runtime and prove it’s solidly orchestrating workflows.
cc @dimitarvp
I’m still super busy but I’m approaching an important project milestone and I’ll be able to check SquidMesh again. 2-3 weeks.
I love the visual additions btw. They pair naturally with human-in-the-loop mechanics. Amazing work figuring that out and pursuing it.
And dynamically adding to workflows? Awesome!
The only thing I am entirely unclear on is being based on Jido and Bedrock. I have sizeable notes on Jido; will have to read them again and ask Claude to help me research Bedrock.
I’ve been reading all your forum announcements btw. Not just putting a 
on them.
Jido journal as the durable runtime, it’s mostly an append-only projection/ read model to give the historical facts of the run. You don’t need to learn Jido, the public api will not expose any Jido primitive unless you want.
on the bedrock, you also can use other executor. The only downside is that bedrock is the only one exposing leasing apis. Maybe at some point I could build my own relational executor but not something I am planning for the near future.
thanks for the support!
Good video to learn more about Bedrock https://www.youtube.com/watch?v=GNOJYKSLvws
96 minutes sounds very rough for my inhuman schedule but I’ll try hard to carve some time, thank you.
You can listen while washing the dishes - that’s the only time i listen to podcasts 
I already watched DDH (author of Ruby on Rails) 6h interview with Lex Fridman - but of course by splitting in my parts. Each during a gym session
I am too efficient, I go through the dishes in 15 minutes. 
And I hate cooking but oh well. Anyway, I’ll get to it. The time is very in question but I am sold on the idea and the final result so I’ll find the time and energy.
…or I’ll link a few spare laptops I have in a cluster and run Ollama with a local Whisper-like model to transcribe videos for me.
My gym sessions are between 35 minutes (heavy) and 100-ish minutes (longer breaks but exercising muscles and tendons with more reps and more series but light weights). And of course, cardio.
But yeah, I’ll find a way. I love the “black cock” in the hands of the guy btw.
Dynamic child workflows is merged to main branch GitHub - dark-trench/squid_mesh: Squid Mesh is a Jido-native durable workflow layer with a friendly Elixir DSL for human-in-the-loop and agentic workflows · GitHub!
Also, did a complete overhaul to the public docs including some livebook examples
Hey there. Since I can’t yet find the time, I’d like to draw your attention to the links below, so you and your agents can compare SquidMesh with two Rust and one Python durable workflow engines:
absurd) GitHub - tensorzero/durable: Durable execution in Postgres · GitHubI plan to use those and feed them to my instance of Claude to compare in a battle royale of workflow engines that I started to maintain (privately for now). If you have the time I’m very curios to hear why does your + your agents researches say.
Haha love the “your agents” part. Thanks for pointing me those libs! Will check them out!
© Copyright Elixir Forum | Terms | Privacy & Cookies
