GenericJam
Sending Functions Instead of Data
In several talks, Joe Armstrong talks about sending functions and doing the processing on the other side of the connection instead of sending the data back and forth. Presumably this saves bandwidth, etc.
Is anyone doing this in practice? In what domain and what problems does it solve? Should everyone be doing it this way?
Most Liked
al2o3cr
If you’re using Agent, you’re doing it already - for instance, Agent.update/3 passes an anonymous function to the agent’s process, which then calls the function with the agent’s state.
The bandwidth being saved here is memory bandwidth; bringing the function to the data avoids the overhead of copying the data to a different process.
Qqwy
It is a very clear power vs. clarity trade-off. See for instance the paper “Out of the Tar Pit” for more information on why too much power in a programming environment can be considered a bad thing. (I very much recommend this paper!)
As quick summary (the paper describes it much better), consider:
- What if there was a mistake in the function? How do you ‘fix’ it later? That’s a lot harder to do with a compiled function than with ‘data representing code’.
- How easy is it to reason about arbitrary compiled functions that are being sent from one computer-system to another, vs. computer-systems giving each-other explicitly defined requests/responses?
- How would you test such a system?
- What about security?
So: While it is a cool concept in theory, in practice you should try very hard to avoid it, because that will make your application more resilient. (With ‘resilience’ being an informal notion of ‘easier to understand’ + ‘easier to test’ + ‘easier to adapt to changing requirements’)
Adzz
Galaxy brain: what if your functions are data.
Let’s take filtering a list of integers as an example. We could define Filter as a protocol like so:
defprotocol Filter do
defstruct [:collection, :predicate]
def apply(collection, predicate)
end
Now let’s encode our predicate as data:
defprotocol IsOdd do
defstruct filter: &__MODULE__.filter/1
def filter(a)
end
defimpl IsOdd, for: Integer do
def filter(a) do
require Integer
Integer.is_odd(a)
end
end
Now Lets implement the zip for a list
defimpl Filter, for: List do
def apply(list, predicate) do
Enum.filter(list, fn x -> predicate.filter.(x) end)
end
end
All of that lets us do this:
Filter.apply([1,2,3], %IsOdd{})
Which gets us close. We now just need to capture all of that in its own struct. We’ll define a general function application protocol:
defprotocol Function do
def apply(function)
end
Then implement if for Filter:
defimpl Function, for: Filter do
def apply(filter) do
Filter.apply(filter.collection, filter.predicate)
end
end
Now we can create our Filter function as a data structure, and as long as where ever we are sending it has the right protocol implementations we can consume it:
%Filter{collection: [1,2,3], predicate: %IsOdd{}}
|> Function.apply( )
What even more interesting is because it’s all protocols each dimension of the filtering problem is extensible. Filtering a collection has 3 dimensions to the problem, the collection being filtered, the items in the collection and the predicate that determines whether something stays in the collection.
Lets now make it so that we can filter on Decimals inside lists:
defimpl IsOdd, for: Decimal do
def filter(a) do
Decimal.positive?(a)
end
end
%Filter{collection: [Decimal.new("1"), 2, 3], predicate: %IsOdd{}}
|> Function.apply( )
Okay and now let us filter on maps as well as lists:
defimpl Filter, for: Map do
def apply(map, predicate) do
Enum.filter(map, fn {k, v} -> predicate.filter.(v) end)
end
end
Function.apply(%Filter{collection: %{a: 1, b: 2, c: 3}, predicate: %IsOdd{}})
Disclaimer, I just find this interesting I have no idea whether it’s a good idea to actually use.
Last Post!
Adzz
I had not heard that thanks I’ll have a read.
Yea I find the protocol thing intersting I’ve been playing with the concept on a branch of my zip library here: GitHub - Adzz/Zip at go-crazyy-ah-ah-go-stupid-oh-oh · GitHub
and wrote about it here: https://medium.com/@ItizAdz/zip-elixir-abusing-protocols-for-triple-dispatch-and-ultimate-flexibility-4c817a5940d6
It felt like I was heading towards creating a poor type system in some way. and I’m sure it links to defunctionalization somewhow Defunctionalization: Everybody Does It, Nobody Talks About It | SIGPLAN Blog
Popular in Discussions
Other popular topics
Chat & Discussions>Discussions
Latest on Elixir Forum
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex










