How to collapse/group records by id within a list of maps

Questions / Help
,
1

Hello! I am really really new to Elixir, getting hands dirty with existing code base.

I do have a transactions object - a list of maps that is returned by an API call. Now, let’s assume this object is done this way:

[{"id":"xyz", "amount": 45.00, "description":"Una mattina mi son svegliato", "dateTime":"2022-10-06T00:00:00.000"},
{"id":"abc", "amount": 74.00, "description":"Vacationes En Chile", "dateTime":"2022-10-05T15:30:00.000"},
{"id":"xyz", "amount": 2.00, "description":"oh bella ciao", "dateTime":"2022-10-06T00:00:00.000"}
 ]

Currently, the array is streamed with stuff being done in .filter and .map method, something like this:

defp fetch_account_transactions(consent, token, account, start_from) do
    with {:ok, transactions} <-
           do_fetch_account_transactions(consent, token, account, []) do
      {
        :ok,
        transactions
        |> Stream.filter(fn data ->
          data["status"] == "BOOKED" && data["transactionMutability"] != "Mutable"
        end)
        |> Stream.map(&data_to_transaction(account, consent, &1))
      }
    end
  end

What I’d like to do is applying a transformation to transactions before it’s being streamed, such that, if something is true, I group maps in that list such that the final transactions object that gets filtered and mapped is the following:

[{"id":"xyz", "amount": 47.00, "description":"Una mattina mi son svegliato oh bella ciao", "dateTime":"2022-10-06T00:00:00.000"},
{"id":"abc", "amount": 74.00, "description":"Vacationes En Chile", "dateTime":"2022-10-05T15:30:00.000"}
 ]

that is - grouping by “id” field in maps and applying transformations such as sum(amount), min(dateTime) and concat(descriptions).

So grateful in advance for your support!

1 Like
2

Here is a way to do it ! (there is not a single solution)
Let me know if this it, or if there something you don’t understand

list = [
  %{id: "xyz", amount: 45.00, description: "Una mattina mi son svegliato"},
  %{id: "abc", amount: 74.00, description: "Vacationes En Chile"},
  %{id: "xyz", amount: 2.00, description: "oh bella ciao"}
]
 
for {id, entries} <- Enum.group_by(list, & &1.id) do
  for %{amount: amount, description: desc} <- entries, reduce: %{amount: 0, description: ""} do
    %{amount: amount_acc, description: desc_acc} -> 
      %{id: id, amount: amount + amount_acc, description: desc_acc <> " " <> desc}
  end
end
3

Thanks a lot @cblavier ! What if I want to generalize the map-reduce operation to other N fields in the map? Do I have to declare all of the fields one by one or i can somehow say “for all fields in the map, get me the first value when reducing”. Thanks!

4

You can merge with a multi clause function that will handle map keys with different merge strategies : sum for amounts, concatenation for descriptions and first value for other keys

for {id, entries} <- Enum.group_by(list, & &1.id) do
  for entry <- entries, reduce: %{id: id, amount: 0, description: ""} do
    acc ->
      Map.merge(acc, entry, fn 
        :amount, amount_1, amount_2 -> amount_1 + amount_2
        :description, desc_1, desc_2 -> "#{desc_1} #{desc_2}"
        _key, _v1 = nil, v2 -> v2
        _key, v1, _v2 -> v1
      end) 
  end
end
1 Like