Phillipp

Phillipp

Help cleaning up this data transformation code

Hey,

I got a list of structs which I need to convert to a map (or similar useful data structure) that I can then further use to create some Prometheus metrics.

Here is what I got. The code is quite dirty in my opinion and maybe there is a smarter way to do it. I wrote it last night at 1am just to get it done, don’t judge me :smiley:

Input data:

trackers = [
  %{
    pause: true,
    pause_timeout_in_ms: 72276156,
    position: %{"lat" => 11, "lng" => 1},
    response: nil,
    timeout_in_ms: 309722,
    user_id: "5cb6fb2c2071c963d21c517f"
  },
  %{
    pause: false,
    pause_timeout_in_ms: false,
    position: %{"lat" => 11, "lng" => 1},
    response: %{
      "status" => "ok",
      "vehicle_position" => %{"vehicle" => %{"tracking" => nil}}
    },
    timeout_in_ms: 1747683,
    user_id: "5cb6fb2d2071c963d21c5180"
  }
]

Then I do the following:

trackers
    |> Enum.map(
         fn x ->
           state = case x[:pause] do
             true -> "paused"
             false -> "active"
           end
           health = case x[:response] do
             nil -> "unhealthy"
             _ -> "healthy"
           end
           %{state: state, health: health}
         end
       )
    |> Enum.reduce(
         %{
           "active" => %{
             "healthy" => 0,
             "unhealthy" => 0
           },
           "paused" => %{
             "healthy" => 0,
             "unhealthy" => 0
           }
         },
         fn (x, acc) ->
           put_in(acc[x[:state]][x[:health]], get_in(acc, [x[:state], x[:health]]) + 1)
         end
       )

(Ignore the inline functions, gonna clean it up after the implementation is set)

Which gives me the following output:

%{
  "active" => %{"healthy" => 1, "unhealthy" => 0},
  "paused" => %{"healthy" => 0, "unhealthy" => 1}
}

Which I then use the following way:

series = for {state, health} <- extract_labeled_data(trackers), # extract_labeled_data is the cove above
    {health, value} <- health do
  {[state: state, health: health], value}
end

Prometheus.Model.gauge_metrics(series)

To generate Prometheus metrics like:

app_gateway_tracking_sessions_count{state="active",health="healthy"} 1
app_gateway_tracking_sessions_count{state="active",health="unhealthy"} 0
app_gateway_tracking_sessions_count{state="paused",health="healthy"} 0
app_gateway_tracking_sessions_count{state="paused",health="unhealthy"} 1

I am sure there are some things that can be improved to get from the input data to the final metrics.

Marked As Solved

mudasobwa

mudasobwa

Creator of Cure

Actually, Enum.into/3 would suffice:

|> Enum.into(%{
    [state: "active", health: "healthy"] => 0,
    [state: "active", health: "unhealthy"] => 0,
    [state: "paused", health: "healthy"] => 0,
    [state: "paused", health: "unhealthy"] => 0
  }, fn {{state, health}, v} -> {[state: state, health: health], Enum.count(v)} end)

Also Liked

mudasobwa

mudasobwa

Creator of Cure

The first step could probably be simplified to:

input
|> Enum.group_by(& {
  (if &1.pause, do: "paused", else: "active"), 
  (if is_nil(&1.response), do: "unhealthy", else: "healthy")
})

Now one might Enum.reduce/3 if zeroes are indeed required, otherwise Enum.map/2 would work:

... |> Enum.map(fn {{state, health}, v} ->
  {[state: state, health: health], Enum.count(v)}
end)
#⇒ [
#     {[state: :active, health: :healthy], 1},
#     {[state: :paused, health: :unhealthy], 1}
#  ]
mudasobwa

mudasobwa

Creator of Cure

At least you might be slightly more DRY:

Last Post!

mudasobwa

mudasobwa

Creator of Cure

At least you might be slightly more DRY:

Where Next?

Popular in Questions Top

JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

Other popular topics Top

jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
274 42576 114
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

We're in Beta

About us Mission Statement