Is it possible to have lazy evaluation on list comprehensions?

[Fl4m3Ph03n1x]

Wow, so many comments and so much goodness ! Thank you all for participating !

@krstfk

Against my better judgement, I’m gonna have a few remarks.

I am grateful for your remarks. I am glad you went against your better judgement. I am here to have an honest discussion, sometimes that means I will be wrong, sometimes that means I will hear things I don’t like to. Sometimes we will just agree to disagree. The important thing is to do things respectfully, which you are.

Now, I am fairly aware of how Monads work. I understand that beneath the hood its a bunch of flatMaps and Maps.

As for books, I am reading something (that I consider) more hands on approach:

• Grokking Functional Programming
• Professor Frisby’s mostly adequate guide to functional programming

I did learn and tried Ocaml a few years ago, but I found these concepts easier to grasp in the books I mentioned.
I also follow Bartosz Milewski, the guy is a legend to me. I just didn’t have the proper time to go through his entire catalog yet (it is very long).

Also, yes, I am absolutely using the cats library. Good catch, I should have mentioned it.

Overall, I am thankful for your input. It truly gives me the impression I am moving in the right direction. I just need the community’s help to tune things into something more “Elixirish”.

@JEG2

Honestly, I don’t feel this is great code in any language.

I confess that I don’t fully understand what it’s doing, but it looks to be primarily about finding meetings everyone could attend. I wouldn’t want code that does that related to IO operations if I could help it.

Well, this code is supposed to be the function you call just before you “run” the code. As in, this is supposed to be the outer layer of you application:

program = schedule(....)
program.run()

(or something like this).

Because it is the outer most layer, it has to deal with things like IO requests that are fliquery and unreliable. This piece of code tries to make it clear by having an IO type in the specs.

For me this is important because I want to know in my code which functions I can trust, and which ones can blow in my face causing horrible pain at 4 AM.

@JEG2

Are we talking about the list comprehension inside the functions you posted or a hypothetical list comprehension operating on the returned result of the posted functions?

Right now, we can have an Option type and Either type Monads using Elixir’s list comprehensions. The code I show goes a little bit into “imagination land” as I also add an IO Monad and simply say “it just works, you can totally trust me”.

So for now, I want to focus on the list comprehensions inside of my functions.

@JEG2

I remain mostly convinced that a Stream is the Elixir answer to this question. Streams are lazy, can represent infinite data structures like a generated list of next possible meetings, and could generate no results when needed.

I am not saying Streams are a poor solution. However, it is my current understanding that I cannot have a function return a Stream(integer) because Stream’s take anything and return anything:

• Typespec for Enumerable - #2 by alfert

If you don’t care what a Stream does, this is perfectly fine. But in my case I want dialyzer to be of some help, so I need to let dialyzer know what thing a Stream has.

@JEG2

Gary Bernhardt’s Functional Core, Imperative Shell video shows off what I believe to be a better and more idiomatic Elixir design for the above goals.

I checked all of his videos and even did a detour using his architecture model some time ago. I really really liked it, but at the end the one thing that didn’t work for me was when I needed to go the shell and back from it several times in a function. It made code quite convoluted. I also based myself in this book:

• Unit Testing (Chapters 6 and 7)

Which I believe makes a good evaluation of the architecture.

In the end, I was unable to find a solution to my problems using that architecture (a function that goes back and forth). It may be I have miss learned or not fully grasped some of it’s concepts, I’ll admit. So I am turning myself into other solutions that promise “a fix”.

@JEG2

It’s possible I just don’t understand the code or questions yet though!

My aim here is to have functions with a sound type system and datastructures that make it mathematically possible to know for a fact my code will work. By work I mean “the ability to avoid a result that was not predicted and therefore produces an unexpected output”.

You can think of it as a tool to achieve and implement Paranoic Telemetry while having dialyzer help you and having code mathematically provable (or as close to it as you can get).

To this effect, I believe Monads are a good possible solution. I believe (for now) that marking functions as impure (with an IO type) makes that easier for my fleshy human brain.

I hope this long explanation somewhat makes sense

@msimonborg

Can’t you do this?

program = fn -> 1 end
program2 = fn -> 2 end

program3 = fn -> program.() + program2.() end

the specs won’t be what you’re looking for though.

For me, this solution has 1 issue: it is not easily composable, meaning I have to manually envelop everything in functions.
The other issue (specs) you got on your own

With this in mind, this solution would work. But I doubt anyone would like to read it. Thus, I am inquiring IO, as a datastructure that does all of this dirty work under the hood for me. At least this is the idea.

@msimonborg

I think I’m finally grasping what you might be trying to do. I have to say the code looks a little like gobbledygook.

All good questions.

• What is supposed to be happening? : Given a list of attendees to a meeting and a duration for said meeting, find a common time slot in their calendars for the meeting.
• What happens to the variable existingMeetings? + where does the meeting variable come from?: Not gonna lie, you found a typo/bug in the book’s code. I did not see this coming. Fixed code:

(in scala)

def schedule(attendees: List[String], lengthHours: Int): IO[Option[MeetingTime]] = {
  for {
    existingMeetings <- scheduledMeetings(attendees) # gets a list of all meetings for both attendees
    possibleMeeting = possibleMeetings(existingMeetings, 8, 16, lengthHours).headOption
    _ <- possibleMeeting match {
        case Some(meeting) => createMeeting(attendees, possibleMeeting)
        case None => IO.unit
      }
  } yield possibleMeeting
}

• Apologies for the use of camelCase

To give some more context:

• scheduledMeetings does some IO. This IO should be lazilly evaluated as well, meaning, it should not run until the main funciton schedule runs.
• possibleMeetings is a pure function.

The challenge here:

• not execute scheduledMeetings until schedule runs
• possibleMeetings must understand something that has not yet happened and work with it
• schedule must return something that has not yet happened and typespecs have to understand if it will fail or not.

It is my understanding, this is not possible with Elixir, mainly because of the interaction between scheduledMeetings and possibleMeetings. It would be like Stream(integer) + integer, if Stream could be typed.

Overall, your solutions are quite genius and your attention to detail impeccable. I fear however, this still proves the point I concluded earlier, that lazy evaluation in Elixir with a type to represent it (be it IO or Stream) is not possible with list comprehensions (only using Macros that transform everything into a function beneath).

@al2o3cr The main idea of the discussion is to use list comprehensions. I understand this is possible using Macros like you did. Although this is truly a testament to how expandable and remarkable Elixir can be, I also agree with you this is not idiomatic. I also do not see how it dialyzer would be happy :S

@stevensonmt

You can have type specs to help with code organization but you can’t rely on the compiler to help you enforce those specs.

Perhaps not with success typing. But I am a firm believer that Gradual typing will still have its day