Passing Correlation ID in async tasks

Phoenix Forum Questions / Help
,
1

I have an API which sits right in the middle of our Architecture, i.e. it is being called by another microservice as well as it is calling multiple upstream systems. Now in any given system of this sorts, correlation for the same call across systems is very very important in terms of identifying what happened to a call, especially when trying to debug or doing root cause analysis.

While I can use Logger meta data for this, i.e. setting the correlation as part of logger meta data which is available until that process lasts, now that is both a boon and a curse in my case, “Let me explain” :slight_smile:

We are using async tasks (Task.async) internally in some of the endpoints in the API to parallelly call independent upstream services, now since as per the nature of async task, it creates a separate process for every parallel task, I am not able to get Logger meta data ( remember from above Logger metadata is per process, now you see the curse part lol ) :frowning:

Is there any way I could attach metadata to those Async task, except from adding the information as a parameter to the Calling functions ( Really don’t want to do that )

Any help will be appreciated.

2

Yeah, one way to do it is to set up a Registry as a K/V store. At the time when you save the logger metadata, also run:

Registry.register(MyCorrelationRegistry, make_ref(), [<whatever-metadata-you-want>])

When you launch the task, run the following:

metadata = :"$callers"
|> Process.get([])
|> Enum.find_value(fn parent ->
  case Registry.select(MyCorrelationRegistry, [{{:_, :"$1", :"$2"}, [{:==, :"$1", parent}], [:"$2"]}]) do
    [c] -> c
    _ -> nil
  end
end)
|> List.first

note that if something goes wrong (e.g. tasks not rooted to a process that registered itself) metadata will be nil; also note that this should work for nested Tasks.

2 Likes