Abstractions and reuse

:+1: I can try to address things better now that I understand the core of your conundrum.

How very FP of you. but … (see below)

This was effectively my suggestion above of where to start just because it’s easy and straightforward. But you are absolutely right about the test concurrency angle. It wasn’t clear to me whether you needed this level of config at an app level or multiple combinations of plugins in a single app.

I don’t think protocols are ideal here either.

I’ve been trying to internalize wisdom I’ve received, the most key pieces being Joe Armstrong’s “Concurrency Oriented Programming” idea and Franceso Caesarini saying “processes should be used for each unit of concurrency in your system”. So I’ll offer a new option.

Not me :grin: . A great thing about concurrency-orientented thinking is that there is an overlap with the good parts of OOP.

Option 4: make the snippet above a process (either GenServer or Task probably) if it’s proper to say “poll publication” is a unit of concurrency in your system. The plugins (as MFA) are state in the process. This solves your testing concurrency requirement.

There’s also an Option 4a that I’ve been using that I am quite pleased with. You could refactor to it if/when it makes sense or start here. Use a struct to contain the context of the processing by taking inspiration in how Ecto.Query, Ecto.Changeset and Plug.Conn work. Your concrete plugin steps are stored in the struct when it is initialized then the struct is threaded through your logic (vasty changing your type signatures :slightly_frowning_face:). This allows you to decouple the processing logic from the concurrent process model. I use the separation to throttle outgoing 3rd party API calls. This approach is covered well in this talk.