orestis

Python generators equivalent?

So Python has a nice concept of generators - functions that become iterators when called:

def step(x, s):
    while True:
        yield x
        x = x + s

This is called like so:

s = step(1, 2)
s.next() # 1
s.next() # 3
s.next() # 5

And it implements the iterator protocol so you can pass it wherever an iterator is expected.

What’s the idiomatic way in Elixir to do this? We can get partway there with:

s = Stream.unfold(1, fn(x) -> {x, x+2} end)
Enum.take(s, 3) # [1, 3, 5]

But obviously this is not stateful. To get stateful, a process would be needed… But even if we don’t maintain state, I couldn’t find an easy way to do this:

{[1, 3], s} = Stream.split(s, 2)
{[5], s} = Stream.split(s, 1)

Using Enum.split blocks, probably trying to reach the end of the Stream.

Thoughts?

14 comments

#generators

1 6029 14

2019-11-22 18:26:05 UTC

Most Liked

Qqwy

TypeCheck Core Team

The concept of a Generator is not something that is unique to Python. Someone has taken the time in the past to write a version (using message passing) in Erlang (which was also translated to Elixir) on RosettaCode. Interestingly, as the Wikipedia Article mentions, Haskell’s lazily evaluated functions are also generators and as such the Stream that is built-in in Elixir indeed is also a generator.

Of course, in an immutable+pure language, you cannot call the same function with the same value over and over nextVal(mygen); nextVal(mygen); nextVal(mygen); and expect different results (unless you ‘cheat’ by using message passing). Indeed, when you’re working with for instance a Random Number Generator in Haskell, you have two options:

Call a function which takes a RNG, and returns a {randomNumber, newRNG} tuple. This is basically similar to what Enumerable does internally, which @benwilson512 talked about as well.
Specify that you want an infinite stream of random numbers, and take as many numbers from that stream as you need. In this case, the generator is ‘consumed’; you’ve turned in an infinite stream of numbers.

In most functional languages (including Elixir), depending on the specific situation, one of these two ways is used. The third one, ‘cheating’ using message-passing, is often too much overhead for what you want to use the generator for. Exceptions of course do exist: A GenServer that returns guaranteed-unique identifiers, for instance.

I do wonder if we can implement a generator based on fixpoint combinators as well, by the way…

Post #10

orestis

Can’t we all be friends?

Every new language you learn impacts your style. Some concepts tend to stick, so naturally I want to explore how they could be replicated or what the alternatives are.

Post #9

NobbZ

Well, at least for Enum.split/2s strictness there has been a discussion recently:

This also tackles some small bits of your “state” problem.

In elixir I do usually not expect side effects to happen, so next(s) which returns 1 on the first call and 2 on the second without rebinding s would be counterintuitive for me.

If you need a central authority handing out free ids, work items, whatever, I do think that is what GenStage has been developed for.

If you really need something as your python thingy (which involves far too much magic for me, I’d expect it to loop forever) then you need to implement it yourself. Some protocol or behaviour and a handfull of macros, and you are ready to release it on hex.

Post #2

Last Post!

rugyoga

After posting, I realised that my Elixir is ambiguous. It should be:

def solve(n, queens, r) when r == n, do: [{Enum.reverse(queens)}]
def solve(n, queens, r) do
0..(n-1)
|> Stream.filter(fn f → safe?(queens, {f, r}) end)
|> Stream.flat_map(fn f → solve(n, [{f, r} | queens], r+1) end)
end

Post #15