EctoSync - sync changes from a cache

Subscribe to events emitted by EctoWatch, and cache the changed row. Use the sync/3 function to update an in-memory row or list of rows using the cached version of the changed row.

This library aims to solve the problem of tracking changes of database rows across processes without causing database stampedes. All possible update paths for each pair of schemas that are connected is determined to avoid brute-forcing through all preloaded associations.

Say you have a Post schema that has many Tags associated through a PostsTags schema and you want to automatically update the Tags for the loaded Posts in a LiveView:

appplication.ex


children: [...,
  {EctoSync, repo: TestRepo, watchers: EctoSync.watchers(Post, assocs: [:tags])},
  ...
  ]

PostLive.ex

alias MyApp.Posts.{Post, PostsTags}

def mount(params, session, socket) do
  posts = list_posts(preload: [:tags])
  if connected?(socket) do
    EctoSync.subscribe(Post, assocs: [:tags])
  end

  {:ok, assign(socket, posts: posts)}
end

def handle_info({{PostsTags, _event}, _} = sync_args, socket) do
  {:noreply, update(socket, :posts, &EctoSync.sync(&1, sync_args))}
end

If you want to add preloads to the changed row, this is possible by adding a Map like this:

EctoSync.sync(post, sync_args, preloads: %{Tag => [:posts]})

It is still in an early stage of development and has not been tested in production. I would love to get feedback on this package.

Package on hex.pm here:

Documentation:

Repo:

4 Likes

Just updated to 0.1.4!

Changes:

  • Has :where filters are now also supported.
  • :insert events are broadcast whenever an association has moved to another parent. So if %Post{person_id: 1} was updated to %Post{person_id: 2}, an insert event is published for all processes listening to {{Post, :inserted}, {person_id: 2}}

Working on tests for HasThrough, should be supported soon.

Version 0.2.0 has been released. It brings some changes and supports new features.

Features:

  • The algorithm updating the in-memory rows has been made more efficient by using a graph of the Schemas in the watchers and determining possible paths at application boot time.
  • :through is fully supported.
  • :where is supported on has_one and has_many.

Changes:

  • The messages sent by EctoSync are now a tuple of the following format: {EctoSync, {schema, event, identifiers, ref}}. This allows them to be distinguished from other events that are in the form of tuples.
  • The trigger names in Postgres are shortened automatically to make sure that the character limit is not surpassed.

Future work:

  • A library to provide utility functions to work with Phoenix LiveView
  • Batching system for events.
2 Likes