Common data transform patterns - please add yours!

9mm

So I’m really loving elixir. BY FAR the most excruciating piece of learning a functional language for me is having to “transform” all my input data so that it includes ALL desired output data. In a mutable language like ruby you can have loops and iterators that just update variables (maps, counters, etc) outside the scope. This makes it super easy to wrangle complex data structures… but in Elixir it’s taking me quite awhile to deal with all this.

I wanted to make a little “database” in this thread that I can constantly come back to and reference of all the best ways you might do simple/medium/advanced data transformations. The advanced ones are too many to include in this thread but I feel like there are a lot of simple/medium ones that you can count on 2 hands that you use over and over and over.

I will keep adding to this thread but here are a few to start off…

A. Transform all values inside a map

input: %{a: "hello", b: "world"}
output: %{a: "HELLO", b: "WORLD"}

Answers:

x |> Enum.reduce(%{}, fn {k, v}, acc -> Map.put(acc, k, String.upcase(v)) end)

for({k, v} <- x, into: %{}, do: {k, String.upcase(v)})

:maps.map(fn (_k,v) -> String.upcase(v) end, i)

A2. Transform value inside a map for a specific key

input: %{a: "hello", b: "world"}
output: %{a: "hello", b: "WORLD"}

A3. Transform value for specific key inside a nested map

input: %{person: %{first: "Jose", last: "valim"}, age: 100}
input: %{person: %{first: "JOSE", last: "VALIM"}, age: 100}

Answers:

x |> update_in([:person, :last], &String.upcase(&1))

B. Transform all values inside a list of map

input: [%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz}]
output: [%{a: "HELLO", b: "WORLD"}, %{a: "WORLD", b: "BUZZ}]

B2. Transform one value inside a list of map for a specific key

input: [%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz}]
output: [%{a: "hello", b: "WORLD"}, %{a: "fizz", b: "BUZZ}]

Answers:

x |> Enum.map(fn m -> Map.update(m, :b, nil, &String.upcase(&1)) end)

C. Push to a list in a map

input: %{list: []}
output: %{list: [1]}

Answers:

x |> update_in([:list], &[1 | &1])

I will add more as I go, feel free to add your own

21 comments

#lists #maps

5 2338 21

2018-12-03 18:19:12 UTC

Showing Posts 1 to 10

Show Best Posts
Show All (oldest first)
Show All (newest first)

kokolegorille

My small contrib for A

iex> i = %{a: "hello", b: "world"}
iex> o = %{i | a: "HELLO", b: "WORLD"}

# or

iex> o = i |> Enum.reduce(%{}, fn {k, v}, acc -> Map.put(acc, k, String.upcase(v)) end)

Post #1

9mm

Amazing, I already like that more than Enum.map + Enum.into blah blah blah. thanks!

Post #2

peerreynders

input = %{list: []}
output = update_in(input, [:list], &[1 | &1])
IO.inspect(output)

iex(1)> input = %{list: []}
%{list: []}
iex(2)> output = update_in(input, [:list], &[1 | &1])
%{list: [1]}
iex(3)> IO.inspect(output)
%{list: [1]}
%{list: [1]}
iex(4)>

Post #3

peerreynders

input = %{person: %{first: "Jose", last: "valim"}, age: 100}
output = update_in(input, [:person, :last], &String.upcase(&1))
IO.inspect(output)

iex(1)> input = %{person: %{first: "Jose", last: "valim"}, age: 100}
%{age: 100, person: %{first: "Jose", last: "valim"}}
iex(2)> output = update_in(input, [:person, :last], &String.upcase(&1))
%{age: 100, person: %{first: "Jose", last: "VALIM"}}
iex(3)> IO.inspect(output)
%{age: 100, person: %{first: "Jose", last: "VALIM"}}
%{age: 100, person: %{first: "Jose", last: "VALIM"}}
iex(4)>

Post #4

peerreynders

input = %{a: "hello", b: "world"}
output = Map.update(input, :b, "THERE", &String.upcase(&1))
IO.inspect(output)

iex(1)> input = %{a: "hello", b: "world"}
%{a: "hello", b: "world"}
iex(2)> output = Map.update(input, :b, "THERE", &String.upcase(&1))
%{a: "hello", b: "WORLD"}
iex(3)> IO.inspect(output)
%{a: "hello", b: "WORLD"}
%{a: "hello", b: "WORLD"}
iex(4)>

Post #5

kokolegorille

For B2…

iex> input = [%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz"}]
iex> input |> Enum.map(fn m -> Map.update(m, :b, nil, &String.upcase(&1)) end)

Post #6

peerreynders

input = %{a: "hello", b: "world"}
output = for({k, v} <- input, do: {k, String.upcase(v)}) |> Map.new()

iex(1)> input = %{a: "hello", b: "world"}
%{a: "hello", b: "world"}
iex(2)> output = for({k, v} <- input, do: {k, String.upcase(v)}) |> Map.new()
%{a: "HELLO", b: "WORLD"}
iex(3)>

Post #7

kokolegorille

You might also use the into parameters…

iex> output = for({k, v} <- input, into: %{}, do: {k, String.upcase(v)})

Post #8

peerreynders

input = [%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz"}]
output = for(m <- input, do: for({k, v} <- m, into: %{}, do: {k, String.upcase(v)}))

iex(1)> input = [%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz"}]
[%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz"}]
iex(2)> output = for(m <- input, do: for({k, v} <- m, into: %{}, do: {k, String.upcase(v)}))
[%{a: "HELLO", b: "WORLD"}, %{a: "FIZZ", b: "BUZZ"}]
iex(3)>

into is OK as long as you stay with Keyword while you are still transforming the data - i.e. not constantly creating Maps only to turn them back to keywords again.

input = [%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz"}]
output = Enum.map(input, &(Enum.map(&1, fn {k, v} -> {k, String.upcase(v)} end) |> Map.new()))

iex(1)> input = [%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz"}]
[%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz"}]
iex(2)> output = Enum.map(input, &(Enum.map(&1, fn {k, v} -> {k, String.upcase(v)} end) |> Map.new()))
[%{a: "HELLO", b: "WORLD"}, %{a: "FIZZ", b: "BUZZ"}]
iex(3)>

Post #9

peerreynders

input = [%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz"}]
output = for(m <- input, do: update_in(m, [:b], &String.upcase(&1)))

iex(1)> input = [%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz"}]
[%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz"}]
iex(2)> output = for(m <- input, do: update_in(m, [:b], &String.upcase(&1)))
[%{a: "hello", b: "WORLD"}, %{a: "fizz", b: "BUZZ"}]
iex(3)>

input = [%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz"}]
output = input |> Enum.map(fn m -> update_in(m, [:b], &String.upcase(&1)) end)

iex(1)> input = [%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz"}]
[%{a: "hello", b: "world"}, %{a: "fizz", b: "buzz"}]
iex(2)> output = input |> Enum.map(fn m -> update_in(m, [:b], &String.upcase(&1)) end)
[%{a: "hello", b: "WORLD"}, %{a: "fizz", b: "BUZZ"}]
iex(3)>

Post #10

Load more posts (11 remaining)