Follow up to my last question about for comprehensions. (Would it have been better to post in that topic? Not sure how the community prefers followup questions to be broken up.)
I rewrote the comprehension as Stream calls. Except for the fact that the data actually being filtered is plenty small enough to do entirely in-memory without lazy streams, would this be idiomatic Elixir code?
Application.started_applications()
|> Stream.filter(fn({app, desc, _vsn}) -> String.starts_with?(to_string(app), query) and not List.starts_with?(desc, ~c"ERTS") end)
|> Stream.map(fn({app, _desc, vsn}) -> {app, vsn} end)
|> Stream.take(5)
No. Use stream if you know you need it. Otherwise you’re just wasting cpu cycles for computation, which could be done quicker in non-lazy fashion.
Application.started_applications()
|> Stream.filter(fn({app, desc, _vsn}) -> String.starts_with?(to_string(app), query) and not List.starts_with?(desc, ~c"ERTS") end)
|> Stream.map(fn({app, _desc, vsn}) -> {app, vsn} end)
|> Stream.take(5)
When you switch to non-lazy operations you’d want to first take the 5 items and then map over just those.
2 Likes
Yes of course. That’s why I said:
Except for the fact that the data actually being filtered is plenty small enough to do entirely in-memory without lazy streams
I’m interested specifically in the syntactic structure of the operations. Assuming I do actually need to use streams, is this how the code would be written? For example, I noticed Elixir has a more concise lambda syntax with ampersands. However I’m pretty sure it’s not usable in this case? Maybe there’s another format for the code that I’m not aware of which should be used?
the & syntax is not trivially usable in your case. You can force it, but it’s ugly. Aside from the Stream thing, it looks idoiomatic. I would say the only thing tha rankles me is that that first fun is on one line.
1 Like
As @LostKobrakai correctly pointed out, you’d want Stream.take(5) as the first stream processing function – that’s how your code will never take more than 5 pieces of data to process in the first place.
IMO your code is idiomatic Elixir as it is, whatever that means exactly (we all have slightly varying definitions of it).
Do your best to make sure you’re not being too fancy: Stream starts becoming useful when you have at least 100 records to process. Anywhere from 10 to 500 records is generally okay to pipe through Enum functions even if the output of the previous step gets copied as an input to the next. For such data sizes that’s not a problem.
But if you expect your input data to be bigger and thus making a new list (and discarding the previous one) at each processing step becomes expensive then this might be a problem. In those cases all lazy processing is a no-brainer (File.stream!, Repo.stream!, the Stream module functions etc).
With Stream it doesn’t matter. With Enum it does.
