laurazard
Lazy cartesian product
Hi all,
This is my first post in this forum, and although I did look, please forgive me if it’s a duplicate (or something very simple, I’m quite new at elixir).
Onto the question: I have two streams which I create with File.Stream!. I want to create the Cartesian product of the two streams, but only until I have 10000 items (the full Cartesian product will be over double that). What I would like to really do is do this lazily, but I can’t seem to come up with an easy way to do that.
Currently, what I’m doing is this
for a <- File.stream!("a.txt"),
b <- File.stream!("b.txt"),
do: (
a = String.trim(a)
b = String.trim(b)
{a, b}
)
However, this seems to work eagerly. Of note is that I also need to apply some transformations to the elements (as exemplified by the String.trim).
Any idea as to how I could accomplish this in a lazy way, maybe with better use of Streams, or zipping them, or some such?
Thank you so much!
Marked As Solved
josevalim
Welcome @laurazard!
A regular for-comprehension is equivalent to a chain of flat_map with a map at the end. So this comprehension:
iex(2)> for x <- 1..10, y <- 1..10, do: x * y
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 3, 6, 9, 12,
15, 18, 21, 24, 27, 30, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 5, 10, 15, 20,
25, 30, 35, 40, 45, 50, ...]
Is equivalent to:
iex(3)> Enum.flat_map(1..10, fn x -> Enum.map(1..10, fn y -> x * y end) end)
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 3, 6, 9, 12,
15, 18, 21, 24, 27, 30, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 5, 10, 15, 20,
25, 30, 35, 40, 45, 50, ...]
Which means you can make it a stream with:
iex(3)> Stream.flat_map(1..10, fn x -> Stream.map(1..10, fn y -> x * y end) end)
You can use a similar approach in your code.![]()
Also Liked
laurazard
Hi @josevalim,
Thank you for the comprehensive answer! This is exactly what I need ![]()
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #javascript
- #code-sync
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance









