Plumbing - useful functions to work with Elixir pipes

Your Libraries & OS Mentoring Libraries
,
1

There was a feature proposal recently on Elixir core mailing list where an idea born. I got very interested in the idea, but it is not going to be added to the language core. So @josevalim suggested to create a plumbing package for it. For now, it has these functions:

[66, 67, 68]
|> pipe(&List.insert_at(&1, 0, 65)) # `pipe/2` sends the pipe subject to an anonymous function and returns its result
|> tee(&IO.puts/1) # `tee/2` sends the pipe subject to an anonymous function but returns the unmodified subject
ABCD
=> 'ABCD'

The plan is to add some more useful functions, ideas and PR’s are very welcome!

14 Likes
2

I’ve used the laravel collection package quite a bit and they named the functionality you named tee/2 tap, which I found quite nice: https://laravel.com/docs/5.5/collections#method-tap

1 Like
3

Just explaining, the tee name comes from this: https://en.wikipedia.org/wiki/Tee_(command)

Since the pipe operator idea has the shell pipe as an inspiration, the tee name was adopted.

3 Likes
4

Oooh, I like this idea! Definitely nicer in my opinion than resorting to macros that modify or enhance pipe’s directly. I love that these are just simple but useful functions that are very easy to reason about and discover. Can’t think of any new useful plumbing functions at the moment.

Although I may submit a docs PR or a PR that adds testing via stream_data.

1 Like
5

I do have some ideas, but I would like to discuss its name, it’s pretty much like this:

"./test.txt"
|> Plumbing.valve(&File.read/1) # 1
|> Enum.map(&String.upcase/1)
|> Plumbing.pipe(&File.write("./upcase_test.txt", &1))
|> Plumbing.valve() # 2
|> Plumbing.tap() # 3
=> {:ok, ["FILE CONTENT"]}

Explaining:

  • 1 - Plumbing.valve/2 would execute the given function passing the subject as the first param, and the send the function result to Plumbing.valve/1.
  • 2 - Plumbing.valve/1 would check the given subject, it must be one of {:ok, result} or {:error, reason}. When the subject is {:ok, result}, the return will be result, and when the subject is {:error, reason}, it raises a %PlumbingException{reason: reason}.
  • 3 - Plumbing.tap/1 would be actually a macro that takes the given code, execute it, but rescue the PlumbingException raised by Plumbing.valve/1 and return {:error, reason} in that case. Otherwise, if no PlumbingException is raised, it returns {:ok, result}, where result is the return of the code execution.

I’ve posted here instead of directly doing it because I’m not sure if the names I’ve chosen for the functions are good for understanding. I mean, IMO they totally make sense, but maybe tap can be confusing for people coming from ruby for example, which has a tap function on Object which does pretty much what our tee function does. But I can’t think of another good name for it, so…

1 Like
6

Would be great if similar could be done with respect to Flows.

1 Like
7

What if Plumbing.valve/1 get subject that is neither {:ok, result} nor {:error, reason} ?

8

It will probably raise a FunctionClauseError

1 Like
9

@NobbZ answer indeed:

10

My $.02, keep it simple.

Remember that every time someone proposes a new extension to the pipe operator, it is rejected because it will make the pipeline hard to read.

If I see the pipeline you posted above in actual code, I would have a hard time understanding what it does. Code should be optimized for reading.

6 Likes
11

Well, I do agree with KISS. But in this case, don’t you think the lib is “too small to be a lib”? I mean, pipe and tee by theirself are useful of course, but it’s nothing other people couldn’t make by their own in 5 minutes of coding. What’s the point of being a lib if it’s that’s small?

1 Like
12

Tee with IO puts could be written as:

|> Enum.map(&(IO.puts(&1) && &1))

I would rather use that than introduce a dependency.

13

Still, new functions should have their own value. We shouldn’t add them just for padding. We do want to avoid having a bunch of very small packages but in some cases, that’s all you need.

I agree that people can just copy those functions to their app and that is totally fine. The point of a package is to be useful, that doesn’t necessarily mean it has to be used. If the idea is pushed forward, then mission accomplished.

3 Likes
14

I think tee is quite useful for quick puts-debugging inside a series of pipes.

What is the current value of data being transformed in the middle of certain pipes?

|> Enum.map(&(IO.puts(&1) && &1))

That’s lots of ampersand, but still readable.

1 Like
15

Enum.map(fn x -> IO.puts x; x end)

No ampersands :wink:

1 Like
16

Fair enough! You convinced me. :slight_smile:

17

What I’d like to see is a way to add named function parameters, that can then be (re) used in pipes.