laurazard

laurazard

Lazy cartesian product

Hi all,

This is my first post in this forum, and although I did look, please forgive me if it’s a duplicate (or something very simple, I’m quite new at elixir).

Onto the question: I have two streams which I create with File.Stream!. I want to create the Cartesian product of the two streams, but only until I have 10000 items (the full Cartesian product will be over double that). What I would like to really do is do this lazily, but I can’t seem to come up with an easy way to do that.

Currently, what I’m doing is this

for a <- File.stream!("a.txt"),
  b <- File.stream!("b.txt"),
  do: (
    a = String.trim(a)
    b = String.trim(b)
    {a, b}
  )

However, this seems to work eagerly. Of note is that I also need to apply some transformations to the elements (as exemplified by the String.trim).

Any idea as to how I could accomplish this in a lazy way, maybe with better use of Streams, or zipping them, or some such?

Thank you so much!

Marked As Solved

josevalim

josevalim

Creator of Elixir

Welcome @laurazard!

A regular for-comprehension is equivalent to a chain of flat_map with a map at the end. So this comprehension:

iex(2)> for x <- 1..10, y <- 1..10, do: x * y
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 3, 6, 9, 12,
 15, 18, 21, 24, 27, 30, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 5, 10, 15, 20,
 25, 30, 35, 40, 45, 50, ...]

Is equivalent to:

iex(3)> Enum.flat_map(1..10, fn x -> Enum.map(1..10, fn y -> x * y end) end)
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 3, 6, 9, 12,
 15, 18, 21, 24, 27, 30, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 5, 10, 15, 20,
 25, 30, 35, 40, 45, 50, ...]

Which means you can make it a stream with:

iex(3)> Stream.flat_map(1..10, fn x -> Stream.map(1..10, fn y -> x * y end) end)

You can use a similar approach in your code.:slight_smile:

13
Post #2

Also Liked

laurazard

laurazard

Hi @josevalim,

Thank you for the comprehensive answer! This is exactly what I need :grinning_face_with_smiling_eyes:

Where Next?

Popular in Questions Top

qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lists...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
jaysoifer
Is there a way to rollback a specific migration and only that one (“skipping” all the other ones)? Would mix ecto.rollback -v 200809061...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New

Other popular topics Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
274 41539 114
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 53690 245
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
dogweather
I wrote this comment on r/haskell, and it’s not popular there. :wink: But I think I’m on to something… Haskell reminds me of Java, and e...
New

We're in Beta

About us Mission Statement