chaitanya
Efficient way to establishing communication between Phoenix applictions
Hi,
i plan to build 2 phoenix applications, let’s say appA and appB.
AppA will be having multiple phoenix containers (to match scalability requirements) in a cluster. AppB (single node) will maintain state in ETS tables. ETS tables on AppB can have at max 5gb of data. I am looking out for a way to setup a consistent connection between 2 apps. AppA should be able to pull data from AppB.
i can try with HTTPoison. but as that is a http protocol, when i try with a million concurrent connections, it can be slow.
instead i am looking out for a better alternative like a channel. is it possible to set a persistent connection between both the applications with a channel and then pull data when required?
Initially i thought of placing my ETS tables inside one phoenix application. however due to the data consistency issues that araise during a network split, i wanted to separate state from application and AppA will be stateless and AppB will contain ETS tables.
Thanks for taking time in reading my point. Any suggestion in this direction can be helpful.
Most Liked
Qqwy
As a more general answer, in decreasing order of efficiency:
- Don’t split your app into multiple apps. (after all, the fastest kind of communication is no communication.)
- Run your applications in the same cluster, and use distributed erlang to communicate. (You can now use Erlang’s battle-tested distributed message-passing systems, which allows for fine-grained and transparent communication.)
- Run two applications at remote locations, and communicate over a socket using Erlang’s built-in
binary_to_term/term_to_binaryto encode/decode datatypes. (More overhead, but does not require apps to run in same distributed erlang cluster.) - Run two applications and perform HTTP requests to each-other. (Slow, but still usable if one of the apps is not a BEAM app, or if a persistent socket connection cannot be established.)
Note that these all take as premise that you really want two applications that communicate. The other answers, about e.g. using a shared database are very well worth considering, depending on the exact situation!
alvises
Tell me If I get it right: You want A to be replicated without any state (not even cache to avoid issues during a network split), and the only source of truth is B, a single node.
If B doesn’t make any particular transformations and processing, instead of making your own db in AppB, I would simply use a database like redis or postgres (which are battle-tested!). Both redis and postgres can be partitioned and replicated - I would use them instead of reinventing the wheel ![]()
if instead you have only the app-A replicas storing data in a distributed fashion, without any database or B node, sure… you have to deal with consistency issues, network partitions and all that jazz… if you can accept eventual consistency, take a look at CRDTs GitHub - derekkraan/delta_crdt_ex: Use DeltaCrdt to build distributed applications in Elixir · GitHub
LostKobrakai
How much data do you need to be sent? Why would there be millions of concurrent connections?
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance










