I want to wrap a library written in another language so it can be used in Elixir. Because the library contains significant state, I am reluctant to wrap it with NIF. I can make it a command line tool and use an Erlang port to communicate with it but then I lose parallelism. My current thinking is to make it a daemon process listening on a UNIX domain socket and communicate with it. What will you guys do in this situation? Is there a good example of a simple protocol to talk to a backend server? I feel HTTP is a little too heavy for my case.
1 Like
You can try using a pool of instances of that library, for example, with GitHub - dashbitco/nimble_pool: A tiny resource-pool implementation for Elixir, it might help (or not) with parallelism.
1 Like
Would it be possible to multiplex multiple calls through single port? HTTP/2 is pretty much doing the same.
1 Like
I’ve used ZeroMQ to communicate between two different processes written in different languages with pretty good results. They have libraries for many languages, and I think there’s a community driven one for Elixir (haven’t tested it).
From their docs:
ZeroMQ (also spelled ØMQ, 0MQ or ZMQ) is a high-performance asynchronous messaging library, aimed at use in distributed or concurrent applications. It provides a message queue, but unlike message-oriented middleware, a ZeroMQ system can run without a dedicated message broker.
ZeroMQ supports common messaging patterns (pub/sub, request/reply, client/server and others) over a variety of transports (TCP, in-process, inter-process, multicast, WebSocket and more), making inter-process messaging as simple as inter-thread messaging. This keeps your code clear, modular and extremely easy to scale.
3 Likes
Thanks for all the suggestions. I am planning to use plain GenServer + Port for now, it is the path with least resistance. Parallelism can be achieved by multiplexing the port and embedding a sequence number to match responses back to the requests.
