I’m a big fan of shallow/wide learning. I recommend learning just enough about them to know, when a specific problem comes up, whether or not they are a good fit for it; then coming back to the tool when you’ve decided you need it and need to learn more.

This is, of course, easier said than done, and when to “call it quits” when studying something is heavily dependent on your personal learning style.

Personally, I feel confident enough in classifying a concept as “sufficiently understood for evaluation later” when I’ve read enough about:

1. How it is meant to be used
2. What it is good at
3. What it is bad at

It’s normally this last point that is difficult to surface. Very few libraries spend a lot of time polishing a description of what they suck at, so I really like to hone in on things like post-mortems and the ‘x’ in “we migrated from x to y” blog posts, to learn from other’s mistakes before making them myself. (In fact, I’m often more ready to adopt a tool that has a lot of stories of switching off of it—it makes me feel more confident in knowing when it is a poor choice, so I get to make better ones!)

I’m confident enough in my abilities and the excellent quality of documentation in this community to trust that if I can recognize the utility and trade-offs of a tool, I can navigate the implementation details when I decide it is time to reach for it.

You can improve this by starting the Tasks under DynamicSupervisor, with restart: :temporary, and monitor the Task’s pid instead of linking to get notified when the process finishes or crashes (and what was the exit value/reason).

Further, if you need retries, you can use a different restart strategy on the supervisor and introduce an additional process, whose job is to start and monitor the Tasks, and provide responses in case these finally crash after N restarts and the job can’t be completed.

Then, if you want to limit the concurrency, you can introduce max_children for the DynamicSupervisor, and then queue up the calls somewhere (the new process from above paragraph can do it) if they’re coming in and the capacity is exceeded, and you just built yourself an in-memory queue system with a pool of N workers, retries and error handling.

Alternatively there’s Task.Supervisor implementing most of that already within the abstractions of Task.