A bit more context on why Squid Mesh uses both Oban and Jido.

Oban is the durable execution backbone. Squid Mesh owns workflow state, retries, replay, cancellation, inspection, and step progression, but it still needs a strong execution layer underneath for queueing, scheduling, redelivery, and background execution across restarts and deploys. I didn’t want Squid Mesh to re-implement a job runtime when Oban is already very good at that layer.

Jido is currently used mostly at the step/action boundary. Right now that means custom workflow steps can be expressed as Jido actions instead of every app inventing its own step contract. That part is useful already, but I also think Jido could play a bigger role over time if Squid Mesh grows more agent-oriented execution patterns, richer action libraries, or more AI-heavy workflow steps. So today Oban is the more foundational dependency, while Jido is more about the execution contract and future direction.

On “why not just use Oban workflows?”: I think that’s a fair question. If a team already has Oban Pro and is happy building directly at that layer, Squid Mesh may be unnecessary. The reason this project exists is to provide a higher-level workflow runtime for application code: workflows as a first-class app concept, with a DSL, run/step/attempt history, replay, cancellation, cron-backed triggers, and a small public API around them. So the goal isn’t to compete with Oban as a job system, but to sit one layer above it.

If helpful, a simple way to describe the split is:

• Oban handles durable execution
• Jido handles custom step actions
• Squid Mesh handles workflow semantics

Next on the roadmap is making the runtime less linear and more workflow-shaped: dependency-based steps, parallel execution, branching/conditions, clearer step input/output mapping, and better inspection for non-linear runs. Jido will stay at the custom step boundary for now, while the main focus is making Squid Mesh itself a stronger workflow runtime.

Congratulations on the release!

This is a super interesting area for me and I have made a sizable research over the weekend so most things are still fresh in my head. What it would take me to use SquidMesh and what it does not have right now:

• Human-in-the-loop state. You have a state for waiting that’s basically delegating to Oban’s scheduled_at. For my work I need a halt and manual unblocking facilities.
• No compensation / undo / rollback per each step. To me the Saga pattern is an absolute requirement.
• Related to the previous: distinction between compensation and undo (Ash’s Reactor taught me this): on the one hand you might have a partial success and then error and the “compensation” step might “finish the job” (happens for f.ex. big file read operations with streaming) and “undo” is what it says: “this has failed, we don’t want to retry, let’s just undo whatever it did”.
• Some steps should have :irreversible undo mechanics. F.ex. posting to a 3rd party ledger in the cloud. (Though that’s not the best example because accounting operations can be voided.)
• Access to currently accumulated state a la Ecto.Multi. I have plenty of production workflows where f.ex. a customer got a tentative KYC result from a 3rd party service and that means a downstream step in the workflow must not execute in this case and be demoted to an Oban job that checks every 24h until KYC state resolves properly and only then finish a certain step – but the workflow should still run to completion otherwise (just that this one particular step becomes a future background job).
• No parallel workflows or dynamic adding of steps a la Oban Pro. Not crucial for me at the moment but I know I’ll need it.

IMO you should compare against Sage, Reactor, FlowStone and Runic by @MrDoops. I liked Runic the most but ultimately decided not to go for it because I can’t force a single Repo.transaction (or Repo.transact) across sub-steps i.e. while doing fan-out – Jido has the same limitation AFAIR. Runic did come super close for me though and in the end I decided to roll my own durable workflow execution engine because I’d not use most of Runic and will have to use some of its hooks to emulate what I need (which is not a bad thing; I just did not want to do it).

One interesting thing that my research kind of surfaced: I was wondering whether I can’t just use somebody’s super developed agentic framework but it turned out that:

• Agentic frameworks more or less optimize for long-running stateful agents hooked to them, which means: routing of signals (messages) and emitting directives. That very often means that the workflow must “freeze” on idling and be “thawed” when the work resumes. And the 2-3 frameworks I checked (Jido included) maintain in-memory state which is acceptable for them because most sessions don’t last super long and because they are all readable on-resume. This does not work well for f.ex. financial saga workflows. Every single step must be auditable.
• Workflow engines mostly optmize for short-lived multi-step transactions with non-negotiable transaction durability i.e. every step is persisted before and after execution, every termination state must be known, compensation / undo, auditable cleaning paths (so compensation and undo must also log/store stuff). The main unit of work is an operation that must be mostly reversible, mostly atomic, and fully observable / auditable.

I know your thing is new and it can’t do everything. Don’t take it as a cold shower type of criticism, please. I am making use of the interesting coincidence that this is posted literal 3 days after I finished a fairly detailed research on the topic. Use or don’t use it. We all have our own requirements. To me durability, full compensation/undo support, explicit marking of steps as irreversible, ability to force a single Repo.transaction across a step + its sub-steps and, well, support for fan-out are all non-negotiable for what I do. To you it might be other factors.

Thanks, this is extremely useful feedback!

A lot of what you listed maps pretty closely to the gaps I already felt were still there, but you made the boundary much clearer for me.

The biggest things I’m taking from your comment are:

• human-in-the-loop as a first-class runtime concern, not just a delayed wait
• compensation / undo / rollback semantics as a core workflow primitive
• explicit irreversibility for certain steps
• proper fan-out / parallel execution
• richer access to accumulated workflow state and more flexible data flow
• stronger transactional thinking around sub-steps and durable auditing

I also think your distinction between agent frameworks and workflow engines is right. The more I work on Squid Mesh, the more it feels like the real target is not “agent runtime with workflows attached”, but “durable workflow runtime that can happen to execute agentic steps when needed”.

The comparison set you suggested also makes sense to me:

• Sage feels strongest on compensation and saga semantics
• Reactor seems ahead on undo / compensation modeling and transaction-oriented orchestration
• FlowStone feels stronger on lineage, dependencies, and approval-style orchestration, though in a more asset/data-oriented direction
• Runic seems especially interesting for DAG/dataflow and parallel graph execution

So from where I’m sitting right now, Squid Mesh is still underpowered compared to that group in some important areas, especially:

• saga/compensation semantics
• HITL / unblockable waiting states
• fan-out / joins
• richer graph/dataflow behavior

That’s not discouraging to me, it actually helps. It gives me a better sense of which problems are real enough to be worth targeting instead of just adding features because they sound workflow-ish.

I’m very willing to implement a lot of what you suggested if I can find the right shape for it. What I’d especially appreciate, if you’re open to it, is help on the maturation process itself:

• which of these concerns should become first-class primitives first
• which are essential for a serious durable workflow runtime vs just nice-to-haves
• where you think Squid Mesh should intentionally stop instead of trying to absorb every idea from Sage / Reactor / Runic / FlowStone
• whether the “application workflow runtime” positioning is actually the right lane for it

So really, thank you. This is exactly the kind of feedback I was hoping for.

Here’s how I’m framing Squid Mesh right now:

An embeddable workflow runtime with a clean DSL, where each step is easy to read and reason about.

That’s the core constraint I’m optimizing for:

• workflows that feel native inside Phoenix / OTP apps
• durable execution on top of the existing Repo + Oban
• a simple, host-app-facing runtime API
• definitions that stay readable, not buried in orchestration plumbing

So the approach is pretty deliberate:

Start with “embeddable runtime + readable DSL”, then let real workflow use cases pull new primitives into existence.

Not trying to front-load every orchestration feature upfront.

If that direction doesn’t hold up in practice, there’s no point reinventing what’s already out there. But so far, it feels like a gap worth exploring.

Thinking of this a little bit and comparing against my own needs currently:

• Compensation and generally Saga semantics;
• Compensate vs. undo distinction;
• Have steps that are :irreversible – better example would be a DB DELETE operation. Whether that means that during a chain undo the engine shuts down and refuses to proceed by saying “I have stumbled upon an irreversible step and refuse to finish the full undo chain” or just returns a partial success result saying “These are the steps that I have undone but these 2 steps are irreversible so nothing could be done about them”… that’s an open question. I’d probably go for the latter;
• The accumulated state;
• HITL mechanics, at least “pause when you get this signal / message” and “resume on receiving this payload or via an explicit command” (the latter is to just un-pause something that does not actually rely on a human so it might be unwelcome and a scope creep, now that I think of it).

What can wait:

• Fan-out / parallel. Most business processes don’t need this.
• Any DAG or conditional branching primitives: I am only mentioning this because my research showed me some people need this and others encode it in workflow engines. I admit I have zero interest in it but I can see how it can be useful (f.ex. what I mentioned earlier about KYC outcomes and where do we go from there).
• Dynamic step injection a la ObanPro’s grafting (I suppose). Very low-prio for me. I don’t think too many people generally need this but I have had cases where it was needed and I recognize it’s useful.

To me your idea to position SquidMesh as a workflow engine that can happen to also execute some agentic flows is the right thing to do – there are literal thousands of agentic frameworks out there and IMO you don’t want to compete with them. I’d normally go for Sage but it does not have persistence, and I cannot get Oban Pro for every project I work on (stakeholders don’t approve the expense). So your engine can be somewhere in the middle of Sage/Runic and Oban Pro, it seems to me.

Alright. I’ve put some of your ideas to my roadmap: Milestones · dark-trench/squidie · GitHub

Milestones order:

1. Graph Workflows
2. Operational Workflows
3. Durable Recovery
4. Saga Semantics
5. Maturity And Positioning

Released squid_mesh 0.1.0-alpha.2.

This update adds a few important workflow/runtime pieces:

• dependency-based workflows with after: […]
• explicit error routing with transition(…, on: :error, …)
• explicit step input / output mapping
• improved inspection for non-linear runs with graph-aware step history
• a round of runtime hardening around dependency scheduling and concurrency

Also expanded the example host app and smoke coverage to exercise these paths end to end.

Hex: squid_mesh | Hex
Docs: squid_mesh v0.1.0-alpha.2 — Documentation
Release notes: Release v0.1.0-alpha.2 · dark-trench/squidie · GitHub

Next milestone is focused on human-in-the-loop workflow primitives.

I published squid_mesh v0.1.0-alpha.3.

This release adds human-in-the-loop workflow support:

• :pause steps can move a run into a durable paused state and resume through
  SquidMesh.unblock_run/2.
• approval_step/2 adds an explicit approve/reject contract for manual review gates, with
  SquidMesh.approve_run/3 and SquidMesh.reject_run/3.
• inspect_run(..., include_history: true) now includes audit events for pause, resume,
  approval, and rejection.
• Paused and approval runs persist their resume targets and output mapping, so already-paused
  runs keep the same behavior across restarts and deploys.

The release also hardens duplicate delivery, cancellation, retry, dispatch-failure, and stale-running-step behavior. Stale running step reclaim is now opt-in; by default, duplicate/redelivered jobs skip already-running steps instead of starting another attempt after a timeout.

Install:

{:squid_mesh, "~> 0.1.0-alpha.3"}

Release notes: Release v0.1.0-alpha.3 · dark-trench/squidie · GitHub

The production-readiness warning still applies. External side-effect steps should use application-owned idempotency keys or another duplicate-safety strategy.

Awesome, love the progress.

Forgot to ask you earlier: do you plan to add an “explanation” function of sorts like Journey does? Basically it can tell you “this has failed because X” or “this did not resume because Y”.

EDIT: talking about Journey.Tools.introspect/1.