Phillipp

Phillipp

Help cleaning up this data transformation code

Hey,

I got a list of structs which I need to convert to a map (or similar useful data structure) that I can then further use to create some Prometheus metrics.

Here is what I got. The code is quite dirty in my opinion and maybe there is a smarter way to do it. I wrote it last night at 1am just to get it done, don’t judge me :smiley:

Input data:

trackers = [
  %{
    pause: true,
    pause_timeout_in_ms: 72276156,
    position: %{"lat" => 11, "lng" => 1},
    response: nil,
    timeout_in_ms: 309722,
    user_id: "5cb6fb2c2071c963d21c517f"
  },
  %{
    pause: false,
    pause_timeout_in_ms: false,
    position: %{"lat" => 11, "lng" => 1},
    response: %{
      "status" => "ok",
      "vehicle_position" => %{"vehicle" => %{"tracking" => nil}}
    },
    timeout_in_ms: 1747683,
    user_id: "5cb6fb2d2071c963d21c5180"
  }
]

Then I do the following:

trackers
    |> Enum.map(
         fn x ->
           state = case x[:pause] do
             true -> "paused"
             false -> "active"
           end
           health = case x[:response] do
             nil -> "unhealthy"
             _ -> "healthy"
           end
           %{state: state, health: health}
         end
       )
    |> Enum.reduce(
         %{
           "active" => %{
             "healthy" => 0,
             "unhealthy" => 0
           },
           "paused" => %{
             "healthy" => 0,
             "unhealthy" => 0
           }
         },
         fn (x, acc) ->
           put_in(acc[x[:state]][x[:health]], get_in(acc, [x[:state], x[:health]]) + 1)
         end
       )

(Ignore the inline functions, gonna clean it up after the implementation is set)

Which gives me the following output:

%{
  "active" => %{"healthy" => 1, "unhealthy" => 0},
  "paused" => %{"healthy" => 0, "unhealthy" => 1}
}

Which I then use the following way:

series = for {state, health} <- extract_labeled_data(trackers), # extract_labeled_data is the cove above
    {health, value} <- health do
  {[state: state, health: health], value}
end

Prometheus.Model.gauge_metrics(series)

To generate Prometheus metrics like:

app_gateway_tracking_sessions_count{state="active",health="healthy"} 1
app_gateway_tracking_sessions_count{state="active",health="unhealthy"} 0
app_gateway_tracking_sessions_count{state="paused",health="healthy"} 0
app_gateway_tracking_sessions_count{state="paused",health="unhealthy"} 1

I am sure there are some things that can be improved to get from the input data to the final metrics.

Marked As Solved

mudasobwa

mudasobwa

Creator of Cure

Actually, Enum.into/3 would suffice:

|> Enum.into(%{
    [state: "active", health: "healthy"] => 0,
    [state: "active", health: "unhealthy"] => 0,
    [state: "paused", health: "healthy"] => 0,
    [state: "paused", health: "unhealthy"] => 0
  }, fn {{state, health}, v} -> {[state: state, health: health], Enum.count(v)} end)

Also Liked

mudasobwa

mudasobwa

Creator of Cure

The first step could probably be simplified to:

input
|> Enum.group_by(& {
  (if &1.pause, do: "paused", else: "active"), 
  (if is_nil(&1.response), do: "unhealthy", else: "healthy")
})

Now one might Enum.reduce/3 if zeroes are indeed required, otherwise Enum.map/2 would work:

... |> Enum.map(fn {{state, health}, v} ->
  {[state: state, health: health], Enum.count(v)}
end)
#⇒ [
#     {[state: :active, health: :healthy], 1},
#     {[state: :paused, health: :unhealthy], 1}
#  ]
mudasobwa

mudasobwa

Creator of Cure

At least you might be slightly more DRY:

Last Post!

mudasobwa

mudasobwa

Creator of Cure

At least you might be slightly more DRY:

Where Next?

Popular in Questions Top

JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a &gt; b) do {:ok, "a"} end if (a &lt; b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New

Other popular topics Top

electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
274 42576 114
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New

We're in Beta

About us Mission Statement