Partisan for Elixir - need help for things not covered in the documentation

Partisan does look to be used in production, and have a new maintainer: Handing off Maintenance of Partisan

Isn’t hex package giving everything you need? There are the links to the GitHub repository as well as documentation which includes even it’s own cheatsheet.

There just seem to be many concepts as per the video https://www.youtube.com/watch?v=KrwhOkiifQ8 . They are not covered in the documents e.g.

• how do you connect the nodes? Is there any auth like you have with disterl.

• do they need to be on the same subnet etc

• how do you enable the the different topologies (peer, client server, membership models)

• how do you do tracing and replay etc.

There’s a lot of stuff not covered by the docs. Just curious if there was anything else incl a partisan elixir book.

hb

Some more…

• Does Node.spawn work the same with partisan? Does it work over partisan? I read somewhere it’s exclusively for messages. I can think of building a custom version to make this work - but again. Would be nice if the docs answered this…specifically for elixir.

thanks
hb

Hi @macrocreation

Good to hear you are interested in Partisan. As mentioned above I am maintaining the library and working hard on getting v5 out including the docs.

BTW notice the latest docs are in Partisan — partisan v5.0.0-beta.21.

Being an Erlang lib you can obviously call it from Elixir but we are currently working on having an Elixir API (wrapper modules for Erlang’s).

We’ve been using Partisan in production since v3.

Let’s see if I can answer all your questions.

• how do you connect the nodes?
  Two options:

• manually by using partisan_peer_service:join/1 passing it the result of calling partisan:node_spec/0 on the target node (or building the spec by hand.
• using partisan_peer_discovery_agent which allows you to use a seed list of nodes or DNS lookup. Alternatively you can implement your own backed for the agent.

There is a third alternative I still need to validate works with the latest changes and is the Orchestration modules in Partisan which offer an implementation for K8s.

We are using the DNS agent within K8s with great results in production.

Is there any auth like you have with disterl.

Nope. You can use TLS though, and I will be working on enabling Cryptosign (equivalent to TLS but without certificates).

how do you enable the the different topologies (peer, client server, membership models)

You do that by setting the config parameter peer_service_manager to the module implementing the topology.

how do you do tracing and replay etc.

Some of them are enabled via config parameters (partisan_config module) and some other are defined in the Options of the API functions.

I am working on documenting those features as we speak.

In v5 we go beyond messaging

I set myself 4 goals for v5:

1. fix all known bugs in the full mesh topology (partisan_pluggable_peer_service_manager using partisan_full_memebership_strategy)
2. fix all known bugs in the HyParView topology (partisan_hyparview_peer_service_managerr)
3. Align Erlang’s and Partisan’s APIs
4. Implement as much of OTP as possible

I am one test case away from having both 1 and 2 done and I consider the implementation of 3 and 4 to be good to go (OTP implementation passes the Erlang’s Test Suites)

So some of the capabilities you are asking for are already exposed by the partisan module and the partisan_gen_* behaviours.

Mind you OTP only works for full mesh topology at the moment.

Thank you @aramallo for taking over the partisan project. It is a great project that shows how far we can take distributed Erlang and still be very performant. I look forward to the evolution of the project and hope I can find some room to contribute more than the tiny bit I have. I also hope I can actually use it in a project some day.

Thanks a lot @Ankhers! All the kudos to @cmeiklejohn for creating the library. I am still learning all its capabilities (now turning my focus on HyParView as it is where the real distinctive value of Partisan lies).

BTW I hope I improved on your remote monitor implementation . Still some more work to do there, specially if we want to try and cater for HyParView.

Looking forward to your contribs!

I’m sure you did lol. I don’t remember the details, but I think I implemented it fairly quickly based on a discussion somewhere because I wanted to get my hands dirty with partisan.

Is there some place outside of github that you and others involved in the project hangout? I know there used to be a gitter space or whatever it was called. Not sure if it is active now with new maintainers.

We only have GitHub but I can open up a channel at Bondy.io’s Slack (recently created)