Missing something with Stream to Enum

Questions / Help
,
1

Missing something when going through the Little Elixir book. Made my own function to grab a csv and make weather requests.

The result is the result of the first operation (a correct temperature response from a service, followed by the cities list. I expected a list of outputs only. Any pointers?

def get_cities_weather(csv_path) do
  alias NimbleCSV.RFC4180, as: CSV

  cities = csv_path
  |> File.stream!
  |> CSV.parse_stream
  |> Stream.map(fn [city, city_ascii, lat, lng, pop, country, iso2, iso3, province] ->
      %{name: city_ascii, lat: lat, lng: lng}
    end)

  cities
  |> Enum.map_every(500, fn city ->
    Metex.Worker.temperature_of(city)
  end)

end

Produces:

[{:ok, 40.8}, %{lat: "34.5167011", lng: "65.25000063", name: "Chaghcharan"},
 %{lat: "31.58299802", lng: "64.35999955", name: "Lashkar Gah"},
 %{lat: "31.11200108", lng: "61.88699752", name: "Zaranj"},
 %{lat: "32.63329815", lng: "65.86669865", name: "Tarin Kowt"},
 %{lat: "32.85000016", lng: "68.41670453", name: "Zareh Sharan"}...]
1 Like
2

map_every(enumerable, nth, fun)

map_every(t, non_neg_integer, (element -> any)) :: list

Returns a list of results of invoking fun on every nth item of enumerable, starting with the first element.

So you have to wait another 495 elements after the ellipsis, before you will get a processed item again.

Perhaps you want plain old Enum.map/2 or Stream.map/2 instead?

1 Like
3

I don’t know what exactly you want to achieve, but there’s Enum.into if you want to convert these maps into lists and - most of all - there’s Enum.to_list for converting stream into list.

4

OH! It’s just returning the original input if it skips it… I was not thinking of that. In my head it was going to even return every 500th item… Thanks for pointing it out.

5

If you want that behaviour, you probably want Enum.take_every/2 or Stream.take_every/2.

It could look like this then:

def get_cities_weather(csv_path) do
  alias NimbleCSV.RFC4180, as: CSV

  csv_path
  |> File.stream!
  |> CSV.parse_stream
  |> Stream.map(fn [city, city_ascii, lat, lng, pop, country, iso2, iso3, province] ->
      %{name: city_ascii, lat: lat, lng: lng}
    end)
  |> Stream.take_every(500)
  |> Enum.map(fn city ->
    Metex.Worker.temperature_of(city)
  end)
end
6

take_every was what I wanted, thanks!

Question. If I did Stream.take_every it’s lazier right? It won’t actually do anything until I Enum.map in the next step?

7

Yupp. A stream does nothing until you either Stream.run/1 it or use it as an Enumerable.

8

or use it as an Enumerable

Minor nitpick: Enumerables are neither lazy nor eager. Stream deals with Enumerables lazily, and Enum deals with them eagerly.

So a stream does nothing until it is consumed. It can be consumed by Stream.run or any of the Enum functions that eagerly consume their inputs.

2 Likes
9

But an important one! I thought if enumerable is the right word here quite a moment. Names are hard especially if you have to deal an remember the relations between similar names.

2 Likes