GenericJam

GenericJam

Sending Functions Instead of Data

In several talks, Joe Armstrong talks about sending functions and doing the processing on the other side of the connection instead of sending the data back and forth. Presumably this saves bandwidth, etc.

Is anyone doing this in practice? In what domain and what problems does it solve? Should everyone be doing it this way?

Most Liked

al2o3cr

al2o3cr

If you’re using Agent, you’re doing it already - for instance, Agent.update/3 passes an anonymous function to the agent’s process, which then calls the function with the agent’s state.

The bandwidth being saved here is memory bandwidth; bringing the function to the data avoids the overhead of copying the data to a different process.

Qqwy

Qqwy

TypeCheck Core Team

It is a very clear power vs. clarity trade-off. See for instance the paper “Out of the Tar Pit” for more information on why too much power in a programming environment can be considered a bad thing. (I very much recommend this paper!)

As quick summary (the paper describes it much better), consider:

  • What if there was a mistake in the function? How do you ‘fix’ it later? That’s a lot harder to do with a compiled function than with ‘data representing code’.
  • How easy is it to reason about arbitrary compiled functions that are being sent from one computer-system to another, vs. computer-systems giving each-other explicitly defined requests/responses?
  • How would you test such a system?
  • What about security?

So: While it is a cool concept in theory, in practice you should try very hard to avoid it, because that will make your application more resilient. (With ‘resilience’ being an informal notion of ‘easier to understand’ + ‘easier to test’ + ‘easier to adapt to changing requirements’)

Adzz

Adzz

Galaxy brain: what if your functions are data.

Let’s take filtering a list of integers as an example. We could define Filter as a protocol like so:

defprotocol Filter do
  defstruct [:collection, :predicate]
  def apply(collection, predicate)
end

Now let’s encode our predicate as data:

defprotocol IsOdd do
  defstruct filter: &__MODULE__.filter/1
  def filter(a)
end

defimpl IsOdd, for: Integer do
  def filter(a) do
    require Integer
    Integer.is_odd(a)
  end
end

Now Lets implement the zip for a list

defimpl Filter, for: List do
  def apply(list, predicate) do
    Enum.filter(list, fn x -> predicate.filter.(x) end)
  end
end

All of that lets us do this:

Filter.apply([1,2,3], %IsOdd{})

Which gets us close. We now just need to capture all of that in its own struct. We’ll define a general function application protocol:

defprotocol Function do
  def apply(function)
end

Then implement if for Filter:

defimpl Function, for: Filter do
  def apply(filter) do
    Filter.apply(filter.collection, filter.predicate)
  end
end

Now we can create our Filter function as a data structure, and as long as where ever we are sending it has the right protocol implementations we can consume it:

%Filter{collection: [1,2,3], predicate: %IsOdd{}}
|> Function.apply( )

What even more interesting is because it’s all protocols each dimension of the filtering problem is extensible. Filtering a collection has 3 dimensions to the problem, the collection being filtered, the items in the collection and the predicate that determines whether something stays in the collection.

Lets now make it so that we can filter on Decimals inside lists:

defimpl IsOdd, for: Decimal do
  def filter(a) do
    Decimal.positive?(a)
  end
end

%Filter{collection: [Decimal.new("1"), 2, 3], predicate: %IsOdd{}}
|> Function.apply( )

Okay and now let us filter on maps as well as lists:

defimpl Filter, for: Map do
  def apply(map, predicate) do
    Enum.filter(map, fn {k, v} -> predicate.filter.(v) end)
  end
end

Function.apply(%Filter{collection: %{a: 1, b: 2, c: 3}, predicate: %IsOdd{}})

Disclaimer, I just find this interesting I have no idea whether it’s a good idea to actually use.

Last Post!

Adzz

Adzz

I had not heard that thanks I’ll have a read.

Yea I find the protocol thing intersting I’ve been playing with the concept on a branch of my zip library here: GitHub - Adzz/Zip at go-crazyy-ah-ah-go-stupid-oh-oh · GitHub

and wrote about it here: https://medium.com/@ItizAdz/zip-elixir-abusing-protocols-for-triple-dispatch-and-ultimate-flexibility-4c817a5940d6

It felt like I was heading towards creating a poor type system in some way. and I’m sure it links to defunctionalization somewhow Defunctionalization: Everybody Does It, Nobody Talks About It | SIGPLAN Blog

Where Next?

Popular in Discussions Top

New
CharlesO
Erlang :list.nth simple, but 1 - based nth(1, [H|_]) -> H; nth(N, [_|T]) when N > 1 -> nth(N - 1, T). Elixir Enum.at … coo...
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31771 143
New
sergio
There’s a new TIOBE index report that came out that shows Elixir is still not in the top 50 used languages. It also goes on to call Elix...
New
chulkilee
Here are the list of HTTP client libraries/wrappers, and some thoughts on HTTP client in general. I’d like to hear from others how they w...
New
cvkmohan
The upcoming Phoenix 1.6 release looks very interesting. Became a habit to watch the commits - and - what they are bringing in. phx.gen...
New
Rustixir
Hi everyone, im working on find best language/framework/system for high concurrency, high performance and stable performance after wor...
New

Other popular topics Top

New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 31586 112
New

We're in Beta

About us Mission Statement