To me those 3 things are actually two. I see the token approach as a means to clean up complex code with spread out dependencies/inputs into code, which can be pipelined very well.

For with vs. pipelines I have the soft rule of the pure functional core using pipelines and with being used at the outside where you not only have to deal with data, but also side-effects.

I see it mostly as a solution for “gathering” errors. So if, for example, you need to return more than 1 error (like in Ecto) then token passing is better as it makes accumulation simpler.

…and because context was already taken by Phoenix

A paper (PDF) describes the

Context Object - A Design Pattern for Efficient Information Sharing across Multiple System Layers

which was submitted for

The 12th Pattern Languages of Programs (PLoP) 2005

conference

This pattern provides an efficient and application transparent way of sharing information between different layers in a software system.

So calling it a (processing) context (structure) instead of a token is entirely justified (and because of bounded context I call it a Phoenix context anyway).

• many parts of your system have to talk about the same thing in different contexts

The majority of the talk is about designing/implementing a context-driven processing “pipeline”.

Architecting Flow in Elixir:

2. Should I Use with or |> for Architecting Flow in Elixir Programs?

Better Control Flow Using the “with” Macro

Robust Data Processing Pipeline with Elixir and Flow - Laszlo Bacsi - ElixirConf EU 2018 (Slides)

You’re typically designing around a context type when an earlier stage is creating information which is used by a later stage while intermediate stages completely ignore that information. If your function’s first parameter is a tuple or map that contains information that is returned as is and is never inspected then that first parameter/return value may be a context type.

Having complete control over your own function signatures it is quite easy to implement them to accept and return the context type so that these functions just work with the pipe operator (|>).

However that also couples those functions to the context type - so it’s not something one does to functions that are reused in different situations as general functions don’t make allowances for accepting and passing “context information”.

So with/1 is often used when these more general/reusable functions are stitched into a flow (typically for errors, though you can flow “out-of-band” data inside tuples around functions that don’t use it - personally I think “adaptor functions” are clearer).