wfgilman

wfgilman

Enum Running Total Calculations with chunk

I have a data set for which I need to calculate a running total for sub-segments. My data set looks like this:

trip_id, station_code, departure_time, station_sequence
1, A, 08:00:00, 1
1, B, 08:05:00, 2
1, C, 08:07:00, 3
1, D, 08:12:00, 4
2, Z, 08:30:00, 1
2, Y, 08:37:00, 2
2, X, 08:41:00, 3

For each record in this data set, I want to add another field, called trip_duration_min. This will be the total time of the trip since the first stop sequence.

trip_id, station_code, departure_time, station_sequence, trip_duration_min
1, A, 08:00:00, 1, 0
1, B, 08:05:00, 2, 5
1, C, 08:07:00, 3, 7
1, D, 08:12:00, 4, 12
2, Z, 08:30:00, 1, 0
2, Y, 08:37:00, 2, 7
2, X, 08:41:00, 3, 11

I’m wondering what is the best way to calculate this value using Stream or Enum. My thought was to use Stream.chunk_by to break up the data set into individual enumerables for each trip_id, then use Stream.scan to accumulate the total time in minutes of each trip. However, I’m not sure what the best way is to reference the prior record in an enumerable.

Any thoughts?

Marked As Solved

wfgilman

wfgilman

Here’s what I did. I put the data set into a list of maps using Stream.map. It looks like this:

[%{trip_id: 1, station_code: "A", departure_time: "08:00:00", station_sequence: 1}, 
 %{trip_id: 1, station_code: "B", departure_time: "08:05:00", station_sequence: 2},
 %{...}]

I then output the Stream to a list:

|> Enum.to_list

I used pattern matching to calculate the trip_duration_min values. Here’s my function.

def put_trip_duration_min([], _, _, acc), do: acc
def put_trip_duration_min([%{departure_time: departure_time,
                               station_sequence: station_sequence} = schedule|t],
                               prior_departure_time, trip_duration_min, acc) do
  trip_duration_min =
    case station_sequence do
      1 ->
        0
      _ ->
        duration(departure_time, prior_departure_time, trip_duration_min, :minutes)
    end

  acc = [Map.put(schedule, :trip_duration_min, trip_duration_min)|acc]
  put_trip_duration_min(t, departure_time, trip_duration_min, acc]
end

defp duration(time, prior_time, duration, units) do
  time = Time.from_iso8601!(time) |> Timex.Duration.from_time
  prior_time = Time.from_iso8601!(prior_time) |> Timex.Duration.from_time
  duration + Timex.Duration.diff(time, prior_time, units)
end

I then call the function and reverse the list back to it’s original order at the end!

|> put_trip_duration_min(nil, 0, [])
|> Enum.reverse

I hope this is helpful for someone else down the road :+1:

Also Liked

schaary

schaary

I would use stream to go through the file and pattern match the line content.

Where Next?

Popular in Questions Top

chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New

Other popular topics Top

AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52341 488
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 39297 209
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New

We're in Beta

About us Mission Statement