Chord - A Flexible Library for Real-Time Context Management 🎵

Thanks for sharing. And looks like it would be quite straightforward to add a custom backend for CubDB. The most interesting stuff seems to be the delta management, which I guess is particularly useful for non-Ecto backends like the provided ETS and Redis.

Thanks for posting this. Quick feedback: a few more examples, and more real-world-like, would be very appreciated.

Agreed - not sure what/where it is supposed to be. What is a “context”?

Hi everyone,

Thank you so much for your feedback!

@dimitarvp I completely agree with you. I’ll be adding more real-world examples to the documentation soon. I’m also working on an interactive tool for Chord that will demonstrate its potential in a hands-on way, making it easier to understand its use cases.

@l3nz , great question! I chose the term “context” because it’s a flexible concept that can represent a different scenarios such as a group chat, VOIP call, video game session, and more. This flexibility ensures the library can be adapted to diverse scenarios without being tied to a specific domain.

Thanks again for sharing your thoughts, I really appreciate it!

Hi @stefan_z

This looks very interesting! I have a few questions

What do you mean by context? Context tends to be a word that has many meanings so would be helpful to clarify what that means. Is it just a container for state?

We currently have a bespoke solution over phoenix channels where we sync GenServer state from our server over to the client using json diffs. Would chord be a potential replacement to our solution?

I think I see what you mean, but in the end if I get it right it looks a bit like an Elixir Agent? I see there are mentions of deltas and collection of unused/expired ones, but what is the advantage of using those? I’m sure there is one, I just don’t see it in the docs.

Hi @venkatd,

You’re absolutely right, “context” is a term that can mean many things depending on the domain. In Chord, a context is indeed a container for state. For example:

• In a chat application, a context could represent a group chat, holding its metadata (e.g., participants, topic, etc.) and messages.
• In a game session, a context could represent the state of the game (e.g., player positions, scores, etc.).
• In a collaborative document editor, it could represent the document’s state, tracking edits and updates.

From what you describe, Chord sounds like a strong candidate for your use case. It provides:

• Delta tracking: Chord tracks changes to the state (deltas) and only sends what’s changed, reducing the amount of data sent over the wire.
• Flexible backends: You can choose in-memory storage (ETS) or distributed backends like Redis, depending on your scalability needs or even implement your own backend.
• Real-time synchronization: You can push state updates to clients through Phoenix Channels, similar to what you’re already doing.
• Stateless vs. Stateful Modes: Chord can work directly with a backend or plug into your existing GenServer-based solution, giving you flexibility depending on your architecture.

To make things easier to understand, I’ve also created a test tool called ChordLab, where you can see Chord in action. Currently, it demonstrates a chat simulation that includes public and private conversations, real-time syncing, and more. You can find the repository and instructions for running it on GitHub:
https://github.com/stefanzvkvc/chord_lab

Hi @l3nz,

I see what you mean about it resembling an Elixir Agent, but there are a few key differences and advantages that Chord brings, especially with deltas and their management:

• Flexibility in architecture: While Agents are a stateful abstraction, Chord works in both stateful (via GenServer) and stateless modes (direct calls to backends like Redis or ETS). This flexibility makes it easier to adapt Chord to a variety of use cases.
• Efficient state updates with deltas: Instead of syncing the entire state on every update, Chord tracks the changes (deltas). This minimizes bandwidth usage and makes real-time synchronization more efficient, especially for larger datasets or high-frequency updates.
• Delta retention and expiry: Chord allows you to configure how long deltas are retained and when expired deltas should be cleaned up. This gives you fine-grained control over memory and storage, which is something Agents don’t provide by default.
• Decoupled backend support: Chord allows you to plug in different backends, such as ETS for in-memory storage or Redis for distributed setups. This makes it highly scalable, which can be challenging to achieve with simple Agents.

Let me know if there are specific points you’d like to see covered more in-depth in the docs!

If I were you I’d use that as theproject description on GitHub

Hi @l3nz ,Thanks for the feedback! You’re absolutely right . I’ve updated the README with comprehensive examples to make it easier to understand and use. Feel free to check it out and let me know if there’s anything else you’d like to see!