smanza

smanza

Hello.

I am developing an application which include P2P services. Each service is using tcp server/client . And I am interested to use GraphQL specially Absinthe for the inter service or inter node communication . Many of P2P are using RPC but I want to leverage the concept of graphql around the query aggregation and field projection to reduce bandwidth and increase throughput.

My first idea was to use ETF with field projection but it seems a bit verbose, non standardised.
Exemple: {:my_method, [param1: val], [:field1, field2: [:field3]]).

So graphql seems good for that.
Unfortunately often services communicate over rest or rpc but graphql endpoints are often on the external client side but not really inside.

Do you have ant thoughts about using GraphQl as internal service communication ?

Showing Posts 1 to 10

dimitarvp

dimitarvp

GraphQL is, in my eyes, good for everything else except internal communication. You also don’t get any field protection; in your normal Absinthe configuration (and I suspect that of many other libraries) you can poll whichever field as long as it’s actually a part of your persistence schema (the DB).

GraphQL is also notoriously bad when it comes to caching endpoints. It’s doable but plain old REST endpoints are times easier to cache by proxies, relieving stress off of your servers (if you put them behind something like Cloudflare or dedicated Squid/Varnish proxy machines).

An argument can be made that GraphQL is more ergonomic and intuitive than many others and I’ll immediately agree if you raise that point. But in terms of technical complexity to integrate + maintain + the cost-benefit in your intended use-case… I’d say it’s a fairly risky choice.

smanza

smanza OP

Thanks for your reply @dimitarvp.

For me the keypoint of GraphQL is really to be able to choose which fields of the resource I want to retrieve.
In my case the inter-communication is through binary protocol using TCP. So GraphQL was a nice approach since it’s transport agnostic . Because as many binary inter-communication services are based on RPC mostly, the data retrieved is everytime the entire set or per use case.

For the caching purpose, in my use case this does not really matter, because the data requested may change and the node involving into a specific shard of the data will change also quite often.

Otherwise I can use a custom RPC mechanism with ETF to filter fields but it will really custom handmade solution and not really so nice to write. I know than many people think RPC calls are faster than REST /GraphQL approach must I wondering if it’s really the case if the data is overloaded the bandwidth will be impacted.

rodrigues

rodrigues

If your microservices are written in Elixir/Erlang, and you want them to talk with each other, you can use the baked-in distributed features :+1:

smanza

smanza OP

I was thinking so, but the communication will be in a untrusted network(public internet) as P2P decentralized system so some malicious could try to attemp to kill nodes , execute malicious function calls, etc.
So unless recreate a create custom erlang distribution driver, the best may remains the classical endpoint communication.

dimitarvp

dimitarvp

Maybe FlatBuffers on top of HTTPS, then? Or you said you don’t want to be dependent on HTTP?

hauleth

hauleth

This is not ETF, this is Elixir representation of decoded ETF. I have created subset of ETF (no support for atoms and improper lists) named Ernie with example implementation in Rust.

I would tell you to use either:

  • C Nodes (despite the name, it can be implemented in any language)
  • Any binary format (FlatBuffers, ProtoBufs, Cap’n’proto, custom one) with message passing in any form (HTTP, RabbitMQ, ZMQ, whatever you like)
smanza

smanza OP

Indeed I’m not using the BERT or Ernie specification but using ETF with :erlang.term_to_binary to encode freely any term in a binary format without involving any dependencies which come almost for free.
Even if protobuf or Cap’nProto produces a smaller encoded data, Erlang Term Format is definitevely the fastest data encoding on Erlang/Elixir and do not require to generate code.

Does the C nodes helps to avoid the erlang distribution via cookie and capable to be in an unsafe network support ?

smanza

smanza OP

Yes I do want to be dependant of the transport, because this layer may evolve for example: TCP,UDP, QUIC,SCPT, WebRTC,etc.. And the data and format exchanged should remains the same to ensure compatiblity.

dimitarvp

dimitarvp

I’d still recommend FlatBuffers, check it out. There’s a really good Elixir library, too.

The transport could be anything. I’d choose QUIC but there are still no Erlang/Elixir libraries. HTTP/2 or gRPC should be more than adequate for a while.

If you are looking for something P2P-like, I recommend checking out Phoenix.Presence.

ityonemo

ityonemo

Is it better to use distribution, or to send a erlang term over ssl using term_to_binary?

Where Next? Top

Trending in Discussions Top

wmnnd
Hey folks, I wanted to share with you a project I’ve been working on for a while: It’s an Open Source (AGPLv3) newsletter tool. Think Ma...
New
axelson
Hi there! :wave: @frigidcode and I (but mostly him) have been running an Elixir Book club, we’re almost done with Designing Elixir Syste...
New
alexslade
Fly’s CEO posted this recently - Turn And Face The Strange · The Fly Blog It says that Fly is going all-in on sprites, which is a worry ...
New
sntran
In my third year of college, around 2010, I found Erlang. A few years later, I found Elixir. I liked the ideas at once: many small proces...
New
Ganna
Hi everyone, I have 8 years of production experience with Erlang/OTP and I’m just starting to get to know Elixir. What would you learn f...
New
johnkirkham
A note on form. Joe Armstrong, co-creator of the Erlang programming language, died in April 2019. What follows is a work of narrative non...
New
gtcode
I’ve been tinkering with Lynx (by José) via hold_ledger. I used AI to write the code, a separate specification, the tests, and the Lean p...
New

Other Trending Topics Top

dominicletz
Hey there, I haven’t seen a list of useful Elixir specific skills for the popular AI tools yet so I’m starting one here with my first. ...
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
KristerV
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New
OndrejValenta
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly) AI dream has many faces, but general...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews