I’m currently using ConsumerSupervisors to launch 1 task per event but the consumer currently makes a direct insert to the db causing serialization error. I even put in a sleep timer in the consumer to not launch a new task giving the old task time to complete but I still get the error.
Is there a way to tell the supervisor not to launch a new task till it finishes the current one? I can only see which_children in the doc but I think it might be taking it to the extreme to check the children and their status.
Thanks in advance! Cheers!
The minimum amount of concurrent children per producer is specified by
:min_demand and the maximum is given by
Given your question, you seem to want to configure both
1 - though that seems to defeat the intent behind
ConsumerSupervisor - so it is entirely possible that I’m misunderstanding your question.
Thanks for the reply!
I was wondering if there is another way to somehow bypass the db transaction error and make it still concurrent. =( I was wondering if I should remove the statements from the transaction but I think it might still cause an error. I had to do a transaction in order to re-create an upsert for redshift. (stage data -> delete from source -> insert from stage data)