chaitanya

chaitanya

Efficient way to establishing communication between Phoenix applictions

Hi,
i plan to build 2 phoenix applications, let’s say appA and appB.
AppA will be having multiple phoenix containers (to match scalability requirements) in a cluster. AppB (single node) will maintain state in ETS tables. ETS tables on AppB can have at max 5gb of data. I am looking out for a way to setup a consistent connection between 2 apps. AppA should be able to pull data from AppB.

i can try with HTTPoison. but as that is a http protocol, when i try with a million concurrent connections, it can be slow.

instead i am looking out for a better alternative like a channel. is it possible to set a persistent connection between both the applications with a channel and then pull data when required?

Initially i thought of placing my ETS tables inside one phoenix application. however due to the data consistency issues that araise during a network split, i wanted to separate state from application and AppA will be stateless and AppB will contain ETS tables.

Thanks for taking time in reading my point. Any suggestion in this direction can be helpful.

Most Liked

Qqwy

Qqwy

TypeCheck Core Team

As a more general answer, in decreasing order of efficiency:

  • Don’t split your app into multiple apps. (after all, the fastest kind of communication is no communication.)
  • Run your applications in the same cluster, and use distributed erlang to communicate. (You can now use Erlang’s battle-tested distributed message-passing systems, which allows for fine-grained and transparent communication.)
  • Run two applications at remote locations, and communicate over a socket using Erlang’s built-in binary_to_term/term_to_binary to encode/decode datatypes. (More overhead, but does not require apps to run in same distributed erlang cluster.)
  • Run two applications and perform HTTP requests to each-other. (Slow, but still usable if one of the apps is not a BEAM app, or if a persistent socket connection cannot be established.)

Note that these all take as premise that you really want two applications that communicate. The other answers, about e.g. using a shared database are very well worth considering, depending on the exact situation!

alvises

alvises

Tell me If I get it right: You want A to be replicated without any state (not even cache to avoid issues during a network split), and the only source of truth is B, a single node.

If B doesn’t make any particular transformations and processing, instead of making your own db in AppB, I would simply use a database like redis or postgres (which are battle-tested!). Both redis and postgres can be partitioned and replicated - I would use them instead of reinventing the wheel :slight_smile:

if instead you have only the app-A replicas storing data in a distributed fashion, without any database or B node, sure… you have to deal with consistency issues, network partitions and all that jazz… if you can accept eventual consistency, take a look at CRDTs GitHub - derekkraan/delta_crdt_ex: Use DeltaCrdt to build distributed applications in Elixir · GitHub

LostKobrakai

LostKobrakai

How much data do you need to be sent? Why would there be millions of concurrent connections?

Where Next?

Popular in Questions Top

New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
JulienCorb
I am trying to implement my new.html.eex file to create new posts on my website. new.html.eex: <h1>Create Post</h1> <%= ...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New

Other popular topics Top

danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29603 241
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
klo
Got a question about when to concat vs. prepending items to list then reversing to achieve appending. So i know lists boil down to [1 | ...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement