GenericJam

GenericJam

Sending Functions Instead of Data

In several talks, Joe Armstrong talks about sending functions and doing the processing on the other side of the connection instead of sending the data back and forth. Presumably this saves bandwidth, etc.

Is anyone doing this in practice? In what domain and what problems does it solve? Should everyone be doing it this way?

Most Liked

al2o3cr

al2o3cr

If you’re using Agent, you’re doing it already - for instance, Agent.update/3 passes an anonymous function to the agent’s process, which then calls the function with the agent’s state.

The bandwidth being saved here is memory bandwidth; bringing the function to the data avoids the overhead of copying the data to a different process.

Qqwy

Qqwy

TypeCheck Core Team

It is a very clear power vs. clarity trade-off. See for instance the paper “Out of the Tar Pit” for more information on why too much power in a programming environment can be considered a bad thing. (I very much recommend this paper!)

As quick summary (the paper describes it much better), consider:

  • What if there was a mistake in the function? How do you ‘fix’ it later? That’s a lot harder to do with a compiled function than with ‘data representing code’.
  • How easy is it to reason about arbitrary compiled functions that are being sent from one computer-system to another, vs. computer-systems giving each-other explicitly defined requests/responses?
  • How would you test such a system?
  • What about security?

So: While it is a cool concept in theory, in practice you should try very hard to avoid it, because that will make your application more resilient. (With ‘resilience’ being an informal notion of ‘easier to understand’ + ‘easier to test’ + ‘easier to adapt to changing requirements’)

Adzz

Adzz

Galaxy brain: what if your functions are data.

Let’s take filtering a list of integers as an example. We could define Filter as a protocol like so:

defprotocol Filter do
  defstruct [:collection, :predicate]
  def apply(collection, predicate)
end

Now let’s encode our predicate as data:

defprotocol IsOdd do
  defstruct filter: &__MODULE__.filter/1
  def filter(a)
end

defimpl IsOdd, for: Integer do
  def filter(a) do
    require Integer
    Integer.is_odd(a)
  end
end

Now Lets implement the zip for a list

defimpl Filter, for: List do
  def apply(list, predicate) do
    Enum.filter(list, fn x -> predicate.filter.(x) end)
  end
end

All of that lets us do this:

Filter.apply([1,2,3], %IsOdd{})

Which gets us close. We now just need to capture all of that in its own struct. We’ll define a general function application protocol:

defprotocol Function do
  def apply(function)
end

Then implement if for Filter:

defimpl Function, for: Filter do
  def apply(filter) do
    Filter.apply(filter.collection, filter.predicate)
  end
end

Now we can create our Filter function as a data structure, and as long as where ever we are sending it has the right protocol implementations we can consume it:

%Filter{collection: [1,2,3], predicate: %IsOdd{}}
|> Function.apply( )

What even more interesting is because it’s all protocols each dimension of the filtering problem is extensible. Filtering a collection has 3 dimensions to the problem, the collection being filtered, the items in the collection and the predicate that determines whether something stays in the collection.

Lets now make it so that we can filter on Decimals inside lists:

defimpl IsOdd, for: Decimal do
  def filter(a) do
    Decimal.positive?(a)
  end
end

%Filter{collection: [Decimal.new("1"), 2, 3], predicate: %IsOdd{}}
|> Function.apply( )

Okay and now let us filter on maps as well as lists:

defimpl Filter, for: Map do
  def apply(map, predicate) do
    Enum.filter(map, fn {k, v} -> predicate.filter.(v) end)
  end
end

Function.apply(%Filter{collection: %{a: 1, b: 2, c: 3}, predicate: %IsOdd{}})

Disclaimer, I just find this interesting I have no idea whether it’s a good idea to actually use.

Last Post!

Adzz

Adzz

I had not heard that thanks I’ll have a read.

Yea I find the protocol thing intersting I’ve been playing with the concept on a branch of my zip library here: GitHub - Adzz/Zip at go-crazyy-ah-ah-go-stupid-oh-oh · GitHub

and wrote about it here: https://medium.com/@ItizAdz/zip-elixir-abusing-protocols-for-triple-dispatch-and-ultimate-flexibility-4c817a5940d6

It felt like I was heading towards creating a poor type system in some way. and I’m sure it links to defunctionalization somewhow Defunctionalization: Everybody Does It, Nobody Talks About It | SIGPLAN Blog

Where Next?

Popular in Discussions Top

jer
I’ve been using umbrellas for a while, and generally started off (on greenfield projects at least) by isolating subapps based on clearly ...
New
chuck
Let me start by stating an assumption: Phoenix is a great approach to building REST APIs. There are many reasons for this, but I will ass...
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
Nvim
Elixir appears to be a superior language to Python. I don’t see any advantage of Python over Elixir. Are there any?
New
nunobernardes99
Hi there Elixir friends :vulcan_salute: In a recent task I was on, I needed to check in two dates which of them is the maximum and which...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New
shishini
I think this twitter post and youtube video didn’t get as much attention as I hoped I am still new to Elixir, so can’t really judge ...
New

Other popular topics Top

Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3271 131117 1222
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New

We're in Beta

About us Mission Statement