mccraigmccraig

mccraigmccraig

Skuld - an effectful approach to deterministic testing, N+1 batching, and serialisable workflows

Skuld started out as a limited exploratory project, and then it snowballed…

Most Elixir apps have a layer of orchestration code bridging whatever pure model has been extracted with side-effecting components - “fetch user, check permissions, load subscription, hit an API, compute a price, write an invoice.” This code is tangled with databases, HTTP clients, and randomness. It’s hard to test, hard to refactor, and oftentimes impossible to property-test.

Skuld lets you write that code as pure descriptions of side effects. Handlers decide what those descriptions mean. Same code runs with real I/O in production, in-memory maps in tests, and properties you can verify with stream_data.

Here’s what that looks like:

Automatic N+1 batching. This reads like sequential code but fetch_user and fetch_orders run concurrently, and all the AccountQueries functions get batched into bulk queries. Stream 10 users with concurrency: 4 and FiberPool batches all deffetch calls into round-trips of [4, 4, 2]. No manual loader, no explicit batching:

defquery build_account_summary(user_id, month) do
  user <- AccountQueries.fetch_user(user_id)
  orders <- AccountQueries.fetch_orders(user_id, month)
  details <- Query.map(Enum.map(orders, & &1.id), &AccountQueries.fetch_order_details/1)
  build_account_summary(user, orders, details)
end

user_ids
|> Brook.from_enum()
|> Brook.map(&build_account_summary(&1, "2026-01"), concurrency: 4)
|> ...

Pausable state machines. A computation pauses at each Yield.yield, waits for input, resumes with full effect context. The same code wrapped in an AsyncCoroutine pauses for the user in a LiveView handle_info or, wrapped in a plain Coroutine, runs as three Coroutine.run calls in tests:

defcomp checkout do
  cart <- Yield.yield(:get_cart)
  {:ok, inventory} <- Inventory.check_stock(cart.items)
  payment <- Yield.yield(:get_payment)
  {:ok, order} <- Orders.place(cart, payment)
  ...
end

Durable computation. The same checkout state machine, above, but now serialised: pause mid-flight, save its entire execution history as JSON, resume after a restart. Every effect invocation is captured in a serialisable log. SerializableCoroutine.run(json, sc, cart) replays recorded effects and continues where it left off:

sc = SerializableCoroutine.new(checkout, fn comp -> ... end)
suspended = SerializableCoroutine.run(sc)
json = SerializableCoroutine.serialize(SerializableCoroutine.get_log(suspended))
# => EffectLogEntry{data: :get_cart, value: nil, state: :started}

SerializableCoroutine.run(json, sc, %{items: [...]})
# => EffectLogEntry{data: :get_cart, value: %{items: [...]}, state: :executed}
# => EffectLogEntry{data: :get_payment, value: nil, state: :started}

Under the hood there are algebraic effects, but you don’t need to know that. Writing domain code with State, Reader, Yield, Port (for external dispatch), and Repo (for database operations) reads naturally once you’ve seen the examples.

The library bundles effects for state management, cooperative concurrency (FiberPool, Channel, Brook), value generation (Fresh, Random), error handling (Throw, Bracket), external integration (Port, Repo, hexagonal architecture), and serialisable workflows (EffectLogger, SerializableCoroutine). Full docs at hexdocs.pm/skuld.

Add {:skuld, "~> 0.28"} to your deps.

I’d love to hear what you think - so far I’ve had reactions varying from solid interest to “I would burn that with fire”!

https://github.com/mccraigmccraig/skuld

Most Liked

mccraigmccraig

mccraigmccraig

I could have talked about backpressure… that works very nicely with Brook … but I thought the automatic batching/N+1 elimination was more fun!

Where Next?

Popular in Announcing Top

Hal9000
Here is my first stab at this. README pasted below. https://github.com/Hal9000/elixir_random Comments and critiques are welcome. Thank...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 36654 110
New
Qqwy
Hello everyone, I wrote a small library today called MapDiff. It returns a map listing the (smallest amount of) changes to get from map...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 54006 488
New
dominicletz
Hi, I thought I had posted my library before but seems I hadn’t. The project is still in early stages but it’s growing and so I think it...
New
tfwright
After working on it for a couple of months and using it in production for most of that time, today I’ve released LiveAdmin, a LiveView ba...
New
woylie
Flop is an Elixir library that applies filtering, ordering and pagination parameters to your Ecto queries. offset-based pagination with...
New

Other popular topics Top

Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 49084 226
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New

We're in Beta

About us Mission Statement