Designing Local-First features for Hologram - what's your dream DX?

Elixir Framework Forums Hologram Forum
, ,
21

Thanks for the kind words and the great breakdown!

On the HTTP choice - I touched on this a bit in my reply above, but to expand on your specific question: you’re right that HTTP is more predictable and durable as a transport. That was definitely part of the reasoning. The bigger driver though was that WebSockets forced me into distributed state for cookies and sessions - I had a CRDT-based cookie store syncing across nodes, and while it worked, it was a debugging nightmare. Cookies and sessions just work naturally with request-response. And then the practical stuff adds up - firewalls, no HTTP caching, mobile reconnection being flakier with persistent sockets, and each WebSocket holding a persistent process and connection state per client which eats more memory. The key insight was that Hologram doesn’t need WebSockets the way LiveView does - since the code runs in the browser, the server only gets hit for page fetching and commands, not every interaction. HTTP persistent connections are plenty for that, and it maps nicely onto offline queuing for local-first too.

On your wishlist - good news on the zero-latency interactivity front - Hologram already gives you that! Since the code runs directly in the browser, clicks and touches are handled on the client in step with the browser’s event loop - no server round-trip needed.

Near real-time updates on the client is definitely on the list too - that’s where the sync layer comes in.

As for offline resilience - the seconds/minutes case is definitely the most common scenario. Ideally I’d love for it to work offline indefinitely on the existing schema - that way we don’t have to make compromises. Not sure yet if that’s fully achievable, but it’s worth aiming high and seeing where the constraints actually bite.

On conflict strategies - First One Wins is clean and covers most cases, I agree. Throw Both Away feels more niche - curious if you have a specific use case in mind for that one? But the priority-based example is what really caught my eye - in-store terminal beating a shopping cart is a great real-world scenario. That’s essentially a “trust hierarchy” where some clients are more authoritative than others. I could see this being a declarative per-resource policy rather than a global setting.

Actually, quick follow-up on that: when the in-store terminal wins over the shopping cart, what should happen on the cart client? Silent override, a notification, something else? Curious what you’d want as a developer there.

On ordering - good call on this. Single-client ordering is something you get for free with WebSockets, but with HTTP separate requests can arrive out of order, so it needs to be handled explicitly. Cross-client ordering adds another layer on top of that. Both are on my radar - haven’t settled on the exact approach yet, there are a few directions I’m exploring.

Great timing with your upcoming project btw - would love to hear more about it as the design evolves!

22

Thanks Zach, appreciate the input! I’ll definitely be thinking about this as the design evolves :slight_smile:

23

Thank you for the detailed summary, much appreciated.

Throw Both Away feels more niche - curious if you have a specific use case in mind for that one?

Admittedly, no!

I can imagine cases where we might want this - say in a regulated environment - like if nurse at Station A and Nurse at Station B adjust the medication at the same time we wouldn’t want to just throw one message away and pick one arbitrarily. The concurrency represents an irreconcilable issue and I’d throw an error back (hey, Nurse A just tried to up the dosage and you tried to decrease it WTH!). Perhaps a bit too theoretical for the first pass.

Actually, quick follow-up on that: when the in-store terminal wins over the shopping cart, what should happen on the cart client? Silent override, a notification, something else? Curious what you’d want as a developer there.

Any time an update fails I would expect to get a callback message that allows me to do something on the client (and let the client decide) “Whoops, that seat was not actually available but here’s one right next to it!”

Thank you!

1 Like
24

@bartblast thanks for the thoughtful reply! I want to push back a bit and clarify what I was actually proposing: command-sync, not data-sync

I think this conversation (and the local-first space generally) has been framed around data synchronization: keeping client state and server state in sync. But I’d argue that for a server-authoritative architecture like Hologram, that framing leads us down a path of unnecessary complexity.

What if the only thing we need to sync upward is a log of commands the user tried to trigger?

Think about it as two separate flows:

  • Upward sync (client → server): Not data. Just commands. “Open ticket with subject X.” “Close ticket Y.” A log of intent, replayed on the server through the same business logic that would have run if the user were online.
  • Downward sync (server → client): Actual data, pushed into the local store based on authorization rules. This is the only direction the data model needs to care about.

Commands go up. Data comes down. Everything else is configured.

I propose this approach because:

  1. It preserves Hologram’s clean separation. Actions stay client-side and pure, commands stay where server effects happen. No blurred boundaries.
  2. One pattern handles everything. Simple CRUD and effects-heavy writes (Oban jobs, emails, external APIs) both go through commands. No second mechanism needed.
  3. Business logic IS the conflict resolution. The server receives instructions, not conflicting data. It processes them sequentially through the existing business logic. Two users closing the same ticket? The first succeeds, the second fails with “ticket is already closed”, exactly the same as when online.

Let me ground my thoughts once again in code snippets, and answers to specific questions. This time, I deviate from my Ash-centric examples:

How do commands go up?

Transparently, like @bartblast suggested. The developer calls put_command and the framework handles queuing, retry, and replay:

def action(:open_ticket, _params, component) do
  subject = component.state.new_subject
  # Write locally (optimistic), then dispatch command
  TicketStore.create(%{subject: subject, status: :open})

  component
  |> put_state(:new_subject, "")
  |> put_command(:create_ticket, %{subject: subject})
end

How does data come down?

It’s pushed from a server-side data model to a client-side local store. The server-side model declares what it syncs and to whom:

# Server-side: an Ecto schema implementing Hologram's sync behaviour
defmodule MyApp.Support.Ticket do
  use Ecto.Schema
  @behaviour Hologram.Sync.Source

  schema "tickets" do
    field :subject, :string
    field :status, Ecto.Enum, values: [:open, :closed]
    belongs_to :representative, MyApp.Support.Representative
    timestamps()
  end

  @impl Hologram.Sync.Source
  def sync_to, do: TicketStore

  @impl Hologram.Sync.Source
  def sync_scope(actor) do
    # Only sync tickets this user is authorized to see
    from t in __MODULE__, where: t.representative_id == ^actor.id
  end
end

# Or as an Ash resource -- the extension handles the behaviour for you
defmodule MyApp.Support.Ticket do
  use Ash.Resource,
    data_layer: AshPostgres.DataLayer,
    extensions: [Hologram.Sync.Source]

  sync do
    store TicketStore
    scope :authorized  # reuses Ash policies to determine what syncs
  end

  # ... attributes, actions, policies as before ...
end

The client-side store receives the data:

defmodule TicketStore do
  use Hologram.Store

  field :id, :string
  field :subject, :string
  field :status, :atom
  field :representative_id, :string
  field :inserted_at, :utc_datetime
  field :updated_at, :utc_datetime

  reconciliation :server_precedence  # default
end

Can one local store source data from multiple tables?

If you want it to:

# Two server-side models, both syncing to the same store
defmodule MyApp.Support.Ticket do
  @behaviour Hologram.Sync.Source

  schema "tickets" do
    field :subject, :string
    field :status, Ecto.Enum, values: [:open, :closed]
    field :source, :string, virtual: true, default: "ticket"
  end

  @impl Hologram.Sync.Source
  def sync_to, do: WorkItemStore
  def sync_scope(actor), do: from(t in __MODULE__, where: t.representative_id == ^actor.id)
end

defmodule MyApp.Support.Task do
  @behaviour Hologram.Sync.Source

  schema "tasks" do
    field :title, :string
    field :status, Ecto.Enum, values: [:pending, :done]
    field :source, :string, virtual: true, default: "task"
  end

  @impl Hologram.Sync.Source
  def sync_to, do: WorkItemStore
  def sync_scope(actor), do: from(t in __MODULE__, where: t.assigned_to_id == ^actor.id)
end

# One client-side store handling both
defmodule WorkItemStore do
  use Hologram.Store

  field :id, :string
  field :title, :string       # mapped from :subject or :title
  field :status, :atom
  field :source, :string      # "ticket" or "task"
end

How are conflicts resolved?

Configured on the local store. Server precedence is the default, but you can customize:

defmodule TicketStore do
  use Hologram.Store

  field :id, :string
  field :subject, :string
  field :status, :atom

  # Default: server data overwrites local optimistic writes
  reconciliation :server_precedence

  # Or per-field:
  # reconciliation :custom, fn field, local_value, server_value ->
  #   case field do
  #     :status -> server_value     # server wins on status
  #     :subject -> local_value     # keep local subject edits
  #     _ -> server_value           # default to server
  #   end
  # end
end
2 Likes
25

Isn’t it just plain HTTP? Upward commands are POSTs. and downward data are GETs.(can also use long poll to the updates async)

Don’t get me wrong, I like this idea and am doing something similar myself. However, if your application can be modeled this way, you probably don’t need Hologram.

26

It’s also important to know how to deal with potential conflicts before they occur, and specify that behavior ahead of time. Certain issues can probably get resolved, others require manual intervention. Checking for new emails vs registering for a new email address (or some other unique entity where the value matters to the end user). There should be different strategies.

27

I think conflict resolution will be the trickiest one to resolve. The right strategy probably depends on your data.

  • First write wins
  • Server wins (because ultimately it’s the source of truth)
  • Client wins (because we only care about the user)
  • Merge (a-la-git), fast track where possible, manual if necessary
  • …

I do like the approach of Lotus Notes (also found in CouchDB) where conflicts are not really resolved but left to the “user” (read: developer).

28

Hey @derek-zhou

Yeah, I think HTTP can work, but I’ll leave Bart to consider whether that’s a good fit for Hologram’s long-term goals.

I’ve actually modelled really complex applications with something like this. It’ll be interesting to learn if there are limitations you’ve experienced that I’m not considering.

29

I don’t know what you have considered, so I I can only tell you what I am doing. In my application, all business logic is on the server side, so it is obviously not local-first. Every user interaction are sent to the server side as POSTs. the reply of POST have no payload other than acknowledgements. The client side maintains an ongoing long poll loop separately to get updates, which are versioned; so it goes like this:

  • client: I have version X.Y, what is the latest?
  • server: if latest is still X.Y, stall using long poll. if latest is actually Z.W, return the diff of Z.W-X.Y. the long poll need to have a reasonable timeout, in which case the server returns no update and the client will poll again
30

I think you’re right that for a complex project, implementing this manually across the entire app could be fragile. I hope whatever architecture Hologram settles on removes these protocol and polling concerns from the developer.

BTW, Holologram as it is today is a nice fit for your project. IIRC, polling through looped calls to put_action with :delay is Hologram’s current recommendation for real-time features.

That’s somewhat similar to what you’re doing. You can try it out whenever you have some time.

31

Thanks @kingdomcoder! The command-sync framing is elegant - it’s essentially CQRS, and I have to admit it fits Hologram’s existing action/command architecture like a glove. But I think Hologram can push even further.

DX. The developer still writes to the local store and dispatches a command. I think that’s boilerplate that could be eliminated - write once, framework handles the rest. More code to manage means more code to reason about, account for, and keep consistent across every write path in the app.

Offline. This approach is offline-tolerant but not offline-capable. All business logic lives on the server, so commands are just intents that only get validated when they reach the server. A user submits a form offline, sees an optimistic result, then hours later the server says “invalid.” A multi-step checkout flow can’t proceed to step 2 because step 1 hasn’t been confirmed yet. Derived state (dashboard counts, state machine transitions) won’t update until the server processes the command. Queued commands are also tied to the version of business logic that created them - a schema or code migration while the user is offline means those commands replay against a server they weren’t written for. The client can record intent but can’t verify it.

Server-authoritative doesn’t mean client-passive. These aren’t mutually exclusive - the server can remain the final arbiter of truth while the client runs the same logic locally for instant validation and offline autonomy. And I’d flip the complexity argument: data-sync is harder to build as a framework, but simpler for the developer. Command-sync is simpler to build but pushes boilerplate and edge cases onto the developer - managing the two-step writes, handling deferred validation errors, reasoning about uncertain optimistic state, etc.

Hologram has a unique advantage we could leverage here. Hologram compiles Elixir to JavaScript - so for most cases, the same business logic that runs on the server can run on the client. Validation, state transitions, derived state - all happen locally, instantly, even offline. Some checks still need the server (e.g. email uniqueness), but that’s the exception, not the rule.

Beyond server-synced data. There are also use cases command-sync doesn’t naturally cover - data that lives on the client only (drafts, preferences, UI state) and ephemeral data synced across devices but never persisted on the server (think collaborative cursors like in Figma, or typing indicators in chat).

IMO the client shouldn’t need the server to think - and Hologram’s Elixir-to-JS compilation makes that possible. Thanks again for the detailed posts, really appreciate the depth of thought here! :slight_smile:

2 Likes
32

Fwiw I agree 100%.

Instant just posted more info on their approach. Granted they make a bunch of different tech choices (triple stores, clojure, etc) but there are some similarities and the spirit of their DX is nice.

33

Hey, @bartblast

The underlying architecture I’m proposing does not conflict with these concerns you’ve raised, IMO.

I hundred percent agree. As per the example in my original post, MyApp.Support.Ticket.open(subject) runs the same validation fully on the client when you call it from the action. That’s Hologram’s Elixir-to-JS compilation doing its thing.

Syntactic sugar can take care of this:

MyApp.Support.Ticket.open(subject, sync: true)  # Writes locally + dispatches a command under the hood
MyApp.Support.Ticket.open(subject, sync: false) # Writes locally only, no server sync

# This can default to whichever makes sense for Hologram
MyApp.Support.Ticket.open(subject)

The framework creates a simple command underneath and handles command dispatch for sync: true.

I consider the Hologram.Extension.IndexedDb in my original post and the Hologram sync protocol you earlier suggested as syntactic sugar for the simple case where Ticket (the server-side resource + validation) and TicketStore (the client-side store) overlap cleanly and are glued together into Ticket.

I don’t view this as “pushing” to the developer. Once again, almost every common case we anticipate can be layered with syntactic sugar on the underlying architecture, while allowing the developer complete control when they need to look under the hood and handle things in interesting ways. The point is: the underlying architecture is command-sync, and that difference matters when the developer needs to step outside the happy path.

One more thought: handling commands when the app is offline is already a problem Hologram has to solve. What happens right now if a user triggers an action that dispatches a command while disconnected? There’s presumably already a queue or retry mechanism in place (or planned). Command-sync simply formalises and extends that existing pattern into a full offline story.

This is a real concern, but it’s not specific to command-sync. Regardless of the approach, offline-capable apps will always need to deal with schema or code migrations that happen while a user is disconnected. Simple cases can come with pre-packaged Hologram solutions e.g. versioning, where the server knows how to handle or reject stale versions gracefully. More complex cases are genuinely hard, and perhaps the developer should actually think about them, but they’re hard for every offline architecture.