Phillipp
Help cleaning up this data transformation code
Hey,
I got a list of structs which I need to convert to a map (or similar useful data structure) that I can then further use to create some Prometheus metrics.
Here is what I got. The code is quite dirty in my opinion and maybe there is a smarter way to do it. I wrote it last night at 1am just to get it done, don’t judge me ![]()
Input data:
trackers = [
%{
pause: true,
pause_timeout_in_ms: 72276156,
position: %{"lat" => 11, "lng" => 1},
response: nil,
timeout_in_ms: 309722,
user_id: "5cb6fb2c2071c963d21c517f"
},
%{
pause: false,
pause_timeout_in_ms: false,
position: %{"lat" => 11, "lng" => 1},
response: %{
"status" => "ok",
"vehicle_position" => %{"vehicle" => %{"tracking" => nil}}
},
timeout_in_ms: 1747683,
user_id: "5cb6fb2d2071c963d21c5180"
}
]
Then I do the following:
trackers
|> Enum.map(
fn x ->
state = case x[:pause] do
true -> "paused"
false -> "active"
end
health = case x[:response] do
nil -> "unhealthy"
_ -> "healthy"
end
%{state: state, health: health}
end
)
|> Enum.reduce(
%{
"active" => %{
"healthy" => 0,
"unhealthy" => 0
},
"paused" => %{
"healthy" => 0,
"unhealthy" => 0
}
},
fn (x, acc) ->
put_in(acc[x[:state]][x[:health]], get_in(acc, [x[:state], x[:health]]) + 1)
end
)
(Ignore the inline functions, gonna clean it up after the implementation is set)
Which gives me the following output:
%{
"active" => %{"healthy" => 1, "unhealthy" => 0},
"paused" => %{"healthy" => 0, "unhealthy" => 1}
}
Which I then use the following way:
series = for {state, health} <- extract_labeled_data(trackers), # extract_labeled_data is the cove above
{health, value} <- health do
{[state: state, health: health], value}
end
Prometheus.Model.gauge_metrics(series)
To generate Prometheus metrics like:
app_gateway_tracking_sessions_count{state="active",health="healthy"} 1
app_gateway_tracking_sessions_count{state="active",health="unhealthy"} 0
app_gateway_tracking_sessions_count{state="paused",health="healthy"} 0
app_gateway_tracking_sessions_count{state="paused",health="unhealthy"} 1
I am sure there are some things that can be improved to get from the input data to the final metrics.
Marked As Solved
mudasobwa
Actually, Enum.into/3 would suffice:
|> Enum.into(%{
[state: "active", health: "healthy"] => 0,
[state: "active", health: "unhealthy"] => 0,
[state: "paused", health: "healthy"] => 0,
[state: "paused", health: "unhealthy"] => 0
}, fn {{state, health}, v} -> {[state: state, health: health], Enum.count(v)} end)
Also Liked
mudasobwa
The first step could probably be simplified to:
input
|> Enum.group_by(& {
(if &1.pause, do: "paused", else: "active"),
(if is_nil(&1.response), do: "unhealthy", else: "healthy")
})
Now one might Enum.reduce/3 if zeroes are indeed required, otherwise Enum.map/2 would work:
... |> Enum.map(fn {{state, health}, v} ->
{[state: state, health: health], Enum.count(v)}
end)
#⇒ [
# {[state: :active, health: :healthy], 1},
# {[state: :paused, health: :unhealthy], 1}
# ]
mudasobwa
Last Post!
mudasobwa
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex









