drewble

drewble

Relying on :erpc for networked nodes

With several Phoenix apps clustered with libcluster, it’s easy to make calls to remote nodes via :erpc.

Say I have two different public-facing apps, and one needs to fetch data from the other to serve requests. Is it a reliable pattern to use :erpc for this relationship? Are there known limitations or performance hits for concurrent requests over :erpc?

Most Liked

tj0

tj0

I’ve only used :rpc (or software that used rpc, erpc looks like a newer version) and I’ve never really hit limits on this. The situation was:

  1. Everything was in same dc(no wan)
  2. Everything was trusted.
  3. Less than 15 nodes connected (other network topologies exist that can connect more, but it was unnecessary as boxes wese beefy).

I don’t think there’s any reason not to use it? The alteratives aren’t that great either. Spending cpu serializing/deserializing json over http? After seeing 25% of the cpu going to parsing, deciding to go to a binary format like messagepack? As long as everything is trusted, single datacenter, and no reason to talk to non-BEAM languages, why not use the standard protocol? For instance, I never had any issues with riak clusters and I can’t think of any application that would be more chatty.

Then on modern hardware, I don’t think it will be very easy to hit the limits. Anyway, for further information:

Personally, I would be more concerned about security and the introduction of other languages than the scale in most situations.

drewble

drewble

Looking into this more, erpc is significantly different than rpc. The issue that is commonly raised is that the original rpc implementation relied on a single GenServer process named rex to handle rpc calls. This causes performance degradation under high call volumes due to mailbox flooding. The charts in the DE-Bench PDF above indicate that using spawn instead of rpc is not subject to the same limits, as it spins up a new process per-call.

erpc basically builds the alternative approach into the Erlang kernel, using spawn_request to make that actual request to the remote node as mentioned in the Kernel 7.0 release notes and found in the Erlang erpc implementation. rex is entirely absent from the :erpc implementation, and rpc itself now relies on erpc for many operations, likely for this reason.

The theoretical limitations of erpc may be the port blocking described in the gen_rpc library README. For the purposes of using erpc to make intra-cluster API calls with reasonably sized response payloads, this is probably not an issue.

D4no0

D4no0

To be fair I’m not a big fan of doing rpc calls without a predefined contract, that can be enforced on both ends, it is just too easy to break the contract by accident.

IMO a monolith that has good separation of code will be times more easier to manage than those 2 separate services trying to communicate via rpc.

Last Post!

Sanjer

Sanjer

Thanks @drewble for this.

I am exploring to build a highly concurrent request-response system, using new erpc. Your findings helps me, get guided into right direction.

Any lessons learnt, if you have used erpc already in production.

If you could share, which may help us?

Where Next?

Popular in Questions Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

Other popular topics Top

electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New

We're in Beta

About us Mission Statement