Map, filter and reduce

Hi there,

I want to get the average height of a given list of people. Here is the associated code :

``````    people = [%{name: 'Mary', height: 160},
%{name: 'Paul', height: 180},
%{name: 'Hugo'}]

heights = Enum.map(people, fn person -> Map.get(person, :height) end)
heights = Enum.filter(heights, fn h -> h != nil end)

if length(heights) > 0, do: (
Enum.reduce(heights, 0, fn (h, total) -> total + h end) / length(heights)
)
``````

Is there a better way to do this? I would like to know if there is a way to filter a list of maps by a given key (here, the height property).

Thanks,
Didier

2 Likes

Iâd go for roughly the following as an `avg` function:

``````def avg(list)
list
|> Enum.reduce({0, 0}, &avg/2)
|> avg_finalize
end

defp avg(x, {sum, count}), do: {sum + x, count + 1}
defp avg_finalize({sum, count}), do: sum / count
``````

The cool thing about this is, that you only need to iterate a single time over the input list.

Currently it would fail on non-numeric input, but you can easily define appropriate guards. Adding one which does simply ignore `nil`s should be easy.

And as a small rule of thumb, when you start chaining from one `Enum`-function into another, you might think about using `Stream` instead to reduce the number of iterations over the complete input.

But this really depends on the size of your input, if your input is short enough and the `Enum`-chains also, then `Stream` might take much more time.

5 Likes

Applying @NobbZâs approach to your specific situation, Iâd be looking at something like:

``````people = [%{name: 'Mary', height: 160},
%{name: 'Paul', height: 180},
%{name: 'Hugo'}]

{total_height, count_people_with_height} =
Enum.reduce(people, {0,0}, fn %{height: height}, {sum, count} -> {sum + height, count + 1}
_, {sum, count} -> {sum, count}
end)

IO.puts total_height / count_people_with_height
``````
5 Likes

You are almost at the stage where it is easier to write the loop directly and not use `Enum`.

11 Likes

You say âloopâ and that makes me wonder. I donât see a good way to do it in one pass with a list comprehension. I can do it with recursion, but I prefer `Enum.reduce`

``````defmodule Test do
def avg(maps), do: avg(maps, {0,0})
def avg([], {total, count}) when count > 0, do: total / count
def avg([%{height: height} | tail], {total, count}), do: avg(tail, {total + height, count + 1})
def avg([_ | tail], acc), do: avg(tail, acc)
end

people = [%{name: 'Mary', height: 160},
%{name: 'Paul', height: 180},
%{name: 'Hugo'}]
IO.puts Test.avg(people)
``````
2 Likes

My professor for functional programming used to say, that âloopâ is just another word for recursion

6 Likes

Thanks for your replies! I really appreciate ! Iâm a totally noob with functional programming and elixir. This is far different from traditional OOPâŠ

2 Likes

Hi,
I am at very nascent stage in elixir , was wondering how can we achieve below case scenario in elixir

``````var doctors=[
{ doctorNumber: "#9",  playedBy: "Christopher Eccleston", yearsPlayed: 1 },
{ doctorNumber: "#10", playedBy: "David Tennant",         yearsPlayed: 6 },
{ doctorNumber: "#11", playedBy: "Matt Smith",            yearsPlayed: 4 },
{ doctorNumber: "#12", playedBy: "Peter Capaldi",         yearsPlayed: 1 }
]
``````

I only require to get only two columns out of three with new column names !

``````doctors = doctors.map(function(doctor) {
return { // return what new object will look like
doctorNumber:doctor.number,
playedBy: doctor.actor,

};
});``````
1 Like

Assuming that above is javascript, the equivalent in Elixir would be:

``````doctors=[
%{ doctorNumber: "#9", playedBy: "Christopher Eccleston", yearsPlayed: 1 },
%{ doctorNumber: "#10", playedBy: "David Tennant", yearsPlayed: 6 },
%{ doctorNumber: "#11", playedBy: "Matt Smith", yearsPlayed: 4 },
%{ doctorNumber: "#12", playedBy: "Peter Capaldi", yearsPlayed: 1 }
]
``````

So getting just a list of maps containing all of the `dockerNumber`'s and `playedBy`'s only would be as a direct translation of your next javascript (assuming you meant to swap the keys and values since they were backwards):

``````doctors = Enum.map(doctors, fn doctor -> %{
number: doctor.doctorNumber,
actor: doctor.playedBy
} end)
``````

As seen here:

``````iex> doctors=[
%{ doctorNumber: "#9", playedBy: "Christopher Eccleston", yearsPlayed: 1 },
%{ doctorNumber: "#10", playedBy: "David Tennant", yearsPlayed: 6 },
%{ doctorNumber: "#11", playedBy: "Matt Smith", yearsPlayed: 4 },
%{ doctorNumber: "#12", playedBy: "Peter Capaldi", yearsPlayed: 1 }
]
[%{doctorNumber: "#9", playedBy: "Christopher Eccleston", yearsPlayed: 1},
%{doctorNumber: "#10", playedBy: "David Tennant", yearsPlayed: 6},
%{doctorNumber: "#11", playedBy: "Matt Smith", yearsPlayed: 4},
%{doctorNumber: "#12", playedBy: "Peter Capaldi", yearsPlayed: 1}]
iex> doctors = Enum.map(doctors, fn doctor -> %{
number: doctor.doctorNumber,
actor: doctor.playedBy
} end)
[%{actor: "Christopher Eccleston", number: "#9"},
%{actor: "David Tennant", number: "#10"},
%{actor: "Matt Smith", number: "#11"},
%{actor: "Peter Capaldi", number: "#12"}]
``````

There are other more traditionally elixirây ways, but for such a simple example this is a simple output.

3 Likes

Woah, thanks a lot ,excellent ! can we also club filter with existing map, just curious though !

``````doctors = doctors.filter(function(doctor) {
return doctor.begin > 2000; // if truthy then keep item
}).map(function(doctor) {
return { // return what new object will look like
doctorNumber: "#" + doctor.number,
playedBy: doctor.actor,
yearsPlayed: doctor.end - doctor.begin + 1
};
});``````
1 Like

The âtraditionalâ Elixir way of writing that is in one of two forms, either via Enum piping:

``````doctors
|> Enum.filter(fn doctor -> doctor.begin > 2000 end)
|> Enum.map(fn doctor -> %{
doctorNumber: "#" <> to_string(doctor.number),
playedBy: doctor.actor,
yearsPlayed: doctor.end - doctor.begin + 1
} end)
``````

Or via a `for comprehension`:

``````for
%{doctorNumber: doctorNumber, playedBy: playedBy} <- doctors, # First grab each doctor out of the list (you can also match out parts here too, but it is numerous in this example so keeping it short)
doctor.begin > 2000, # Then filter on a condition
do: %{ # Then return the result after filtering, this example can be arbitrarily complex and remains easier to read when it does
doctorNumber: "#" <> to_string(doctor.number),
playedBy: doctor.actor,
yearsPlayed: doctor.end - doctor.begin + 1
}
``````

Or something like that.

EDIT: Also, this forum is Discourse, it supports normal markdown, so using code fences like:

```elixir
def Some elixir code
```
```javascript
var Some javascript code
```

Gets turned into this:

``````def Some elixir code
``````
``````var Some javascript code
``````

With syntax coloring and all.

2 Likes

Thank you , sure moving forword will take the markdown factor into account.
and also edited my queries as per markdowns

1 Like

Awesome! The formatting looks perfect!

Please ask if you have any other questions or conversions, I like the challenges.

3 Likes

Hi ,
I have a query, I would like to make dynamic Sql statements for multiple where clauses in elixir and pass it across to raw queries.
Please note i use sql.format in my nodejs code to prevent âSQL injectionâ
my existing javascript code looks likes this

``````var interns = [{
"name": "donnie"
}, {
"age": 16
}, {
"gender": "male"
}]

var selector = ""
interns.forEach(function(item, index) {

var searchvalue = item[Object.keys(item)];
var searchkey = Object.keys(item)[0];

if (selector == "") {
selector = "and " + (searchkey) + " LIKE " + "'%" + (searchvalue) + "%'";
} else {
selector = selector + " and  " + (searchkey) + " LIKE " + "'%" + (searchvalue) + "%'";
}

})
console.log(selector)
``````

, how can we achieve the same in elixir .
my progress is kinda stuck in how to trigger else clause and append to z variable

``````user_params=[%{"age" => 19}, %{"name" => "matt213"}, %{"sex" => "male"}]

sqlwhere=1;
z="";
for norm <- user_params do

for {k, x} <- norm do
IO.inspect  sqlwhere
if sqlwhere <=1  do

intern=" and #{k} = '#{x}'"
x = "and #{k}="
y = "'#{x}'"
z = x <> " " <> y

IO.inspect  z
sqlwhere=sqlwhere+1;

else
IO.inspect  "unable to get here !"
#     j = "#{k}"
# l = "#{x}"
# k = j <> " " <> l
#    IO.inspect k

#list=list+"#{list} where #{k} = '#{x}'"
end
end

end``````
1 Like

Using strings for building SQL is risky. Fortunately with ecto, we donât have to do it:

``````user_params = [%{"age" => 19}, %{"name" => "matt213"}, %{"sex" => "male"}]
user_params
|> Enum.map(&Enum.to_list/1)
|> Enum.reduce(Foo, fn [{key, value}], query ->
field = String.to_existing_atom(key)
search = "%#{value}%"
from q in query, where: like(field(q, ^field), ^search)
end
``````

where `Foo` is the schema you want to search. I would also probably employ some more advanced mechanism for filtering allowed fields than `String.to_existing_atom/1`, but for an example itâs good enough.

3 Likes

Hi,

Apologise for earlier post of in process codebase and case scenarios this is my updated codebase;

``````ui = [%{"age" =&gt; 19}, %{"fullname" =&gt; "todd"}]
lists=ui
|&gt; Enum.map(&Enum.to_list/1)
|&gt; Enum.reduce(User, fn [{key, value}], query -&gt;
field = String.to_existing_atom(key)
search = "%#{value}%"
Repo.all from(q in query, where: like(field(q, ^field), ^search), select: {q.fullname, q.age})
end)
IO.inspect  lists
``````

However , I am getting below exception for the same . could you please provide any pointers on same

``````** (exit) an exception was raised:
** (Protocol.UndefinedError) protocol Ecto.Queryable not implemented for [{"
Todd", 19}]
(ecto) lib/ecto/queryable.ex:1: Ecto.Queryable.impl_for!/1
(ecto) lib/ecto/queryable.ex:9: Ecto.Queryable.to_query/1``````
1 Like

You donât want to do `Repo.all` inside the `Enum.reduce` callback. The idea is that `reduce` is reducing over a single query, building it up with each iteration. The return value of the Enum.reduce is the query that you want to execute.

``````user_params = [%{"age" => 19}, %{"name" => "matt213"}, %{"sex" => "male"}]
query =
user_params
|> Enum.map(&Enum.to_list/1)
|> Enum.reduce(Foo, fn [{key, value}], query ->
field = String.to_existing_atom(key)
search = "%#{value}%"
from q in query, where: like(field(q, ^field), ^search)
end

users = Repo.all(query)
``````

Or simply:

``````users =
user_params
|> Enum.map(&Enum.to_list/1)
|> Enum.reduce(Foo, fn [{key, value}], query ->
field = String.to_existing_atom(key)
search = "%#{value}%"
from q in query, where: like(field(q, ^field), ^search)
end
|> Repo.all
``````
3 Likes

Affirmative , now it seems quite clear the functional usage of repo, thanks !
one more query what modification is required in code if the incoming user_params look like this

`user_params = %{"age" => 19, "fullname" => "matt213", "sex" => "male"}`

1 Like

This is a simple enough change I think itâs best if you gave it a shot on your own. If you have something like

``````user_params = %{"age" => 19, "fullname" => "matt213", "sex" => "male"}
``````

what does doing

``````user_params |> Enum.map(fn thing -> IO.inspect(thing) end)
``````

print?

What does this tell you about the shape of each item when you map over a map, and how might that fit into your Enum.reduce call?

1 Like

hi ,
below is my modified codebase , it throws below error

** (exit) an exception was raised:
** (Protocol.UndefinedError) protocol Ecto.Queryable not implemented for
[#Ecto.Query from u in HelloPhoenix.User, where: like(u.age, ^"%19%")>, #Ecto.Query
from u in HelloPhoenix.User, where: like(u.name, ^"%matt213%")>]

Modified Codebase

``````      query=user_params
|> Enum.map(&Enum.to_list/1)
|> Enum.reduce(User, fn thing, query ->

for {key, value} <- thing do

field = String.to_existing_atom(key)
search = "%#{value}%"
from q in query, where: like(field(q, ^field), ^search)

end

end)
user = Repo.all(query)

IO.inspect user``````
1 Like