While exploring with Oban I was made aware of some of the risks involved with serialization, in particular with serializing anonymous functions (see Risks of serializing structs and captured function). The related/belated follow-up thought that occurred to me was “is it safe to send streams to separate nodes for execution?” Streams are essentially captured functions, right? So isn’t there a risk that if the stream is created on one node and then sent to another node for execution, then the stream might fail to execute because maybe the receiving node doesn’t have the same modules available or maybe it has been updated slightly causing the function identifier to change? Or does this issue only come up if the thing is serialized?

My specific use case is dealing with data processing – in a nutshell, one process prepares streams and sends them to another process where they are run with specific boundaries for concurrency and rate-limiting enforced. It seems to work fine when running on a single node, but I wanted a reality check… could this all fall apart if 2 nodes were involved or if a hot-code update were applied?

Thanks for any thoughts on the matter.