Looping through multiple Enumerables the "Elixir Way"

Questions / Help
1

I have a scenario within my app where I need to loop over a few different associations to get to the data I need to process but the way I’m currently doing it feels a little sloppy. I’m curious if I could get some help refactoring this code and maybe learn better ways to handle this particular problem and I could be going about this completely wrong.

CODE:

seasons = Seasons.active_seasons |> Repo.preload([teams: [predictions: [question: :answer]]])

for season <- seasons do
  for team <- season.teams do
    for prediction <- team.predictions do
      start_answer_processing(prediction.question)
    end
  end
end

As you can see I’m using comprehensions to get to the meat of what I’m looking for. I understand that this is not performant and that is okay in this situation as this processing is going to happen in the background and UI doesn’t have to wait for this to complete.

This is a fairly common scenario and I’m curious for information to better handle this processing. Is using comprehension for this incorrect? Is there a better way to do this? If so, why is it better? Maybe this is just fine? Thanks for any help!

2

You don’t need to nest comprehension list :slight_smile:

iex> for a <- 1..2, b <- 1..3, do: a * b
[1, 2, 3, 2, 4, 6]

UPDATE: oh, You are cumulating result… sorry

3

Can you show us what is inside in this function?

4

TBH, really nothing :). All the setup is just getting to the point where I have the data I need when I call that function.

5

You might also write this… (not tested)

iex> seasons 
|> Enum.map(fn season -> 
  Enum.map(season.teams, fn team -> 
    Enum.map(team.predictions, fn prediction -> 
      start_answer_processing(prediction.question) 
    end) 
  end) 
end)
6

I also realize I can at least make look a little prettier like this:

for season <- seasons,
    team <- season.teams,
    prediction <- team.predictions,
    do: start_answer_processing(prediction.question)
7

Is there any difference between the two in practice?

8

I am not really expert about programming but everything depends on that function. Maybe cache, or maybe you dont need to call that function for every item. There can be alot possibilities to depends on situation

9

Truth, everything does depend on the situation. Thanks for the response, part of this was just looking for some convo around this particular bit of code.

10 
for season <- seasons,
    team <- season.teams,
    prediction <- team.predictions,
    do: start_answer_processing(prediction.question)

I like this version because it clearly separates how You get the collection, and what You apply on it.

1 Like
11

for is a macro and basically compiles to a bunch of Enum functions.

1 Like
12

Yea, me too. I think this is likely the most readable way to do this.

13

Oh interesting. Good to know.

14

Oh, interesting… I thought it would translate to Erlang list-comprehension.

15

I took another look at the docs, and yes, its a specialform, not a macro, so it can compile into anything. This again might mean, that for is a bit faster than Enum, as we do not have to dispatch on many different data types which all might implement the protocol, but we have something specialised to lists.

2 Likes
16

I have to add, that I’d probably try it this way, as I prefer piping over indenting:

seasons
|> Enum.flat_map(& &1.teams)
|> Enum.flat_map(& &1.predictions)
|> Enum.flat_map(& &1.question)
|> Enum.map(&start_answer_processing/1)

I’d use this variant (or a Stream equivalent) until it prooves to be a bottleneck.

2 Likes
17

Maybe even use get_in/3, e.g.:

seasons
|> get_in([Access.all(), :teams, Access.all(), :predictions, Access.all(), :question])
|> Enum.map(&start_answer_processing/1)

I didn’t test, or look carefully. It’s just to give you an idea. You may need to tweak the path in get_in/3 or even add Enum.unique/1 or List.flatten/1 before Enum.map/2.

Cheers

18

I think elixir’s list comprehensions are Enum.reduces for everything other than binaries. Otherwise they wouldn’t be able to support maps, for example.

1 Like