In my experience with React/Redux, the biggest downside to that design is that all your views become dependent on the store, and thus any changes to the store have the potential to break an indeterminate number of views in subtle and unpredictable ways.

For example, I have seen an app where view A (“edit person”) subscribes to a slice of the store via a selector like getPerson expecting it to have the props first_name, middle_name, and last_name, and view B (“show person”) uses the same selector expecting the prop full_name. I was brought in to work on the JSON REST API primarily, which was suffering from a lot of performance problems, one of them being that it did not adhere to the sparse fieldset spec. But trying to refactor it was a nightmare because every view in the whole app expected all the data that was possible to request was always available. To make matters worse, well-meaning FE devs had sprinkled safe navigation operators everywhere to handle the myriad cases of missing data, but that just meant we got bug reports about missing data that were much more time consuming to diagnose than actual page crashes. As this app had been using React/Redux for a few years this was dozens of views with hundreds of selectors. Tests, of course, were almost non-existent.

Now, you could argue this was a bad design/use of Redux, but that would make it at best a pretty fatal foot canon. Also, testing tools/culture might be much better with LV, but of course we can’t depend on tests to catch everything. Do you have an idea about how to address this issue in your project? Because my feeling was Redux created way more problems than it solved vs just simple prop drilling and we now only use Redux for “truly global data” (like current user data).

So, I see this as a problem with applying the reducer pattern to global stores, not global stores in general.

Global stores provide state that all components know about and can subscribe to. This is what allows data dependency injection.

The reducer pattern, state = reduce(state, action), is one way you can go about updating a global store, but it is by no means the only one. LiveQuery will not use it, at least externally.

Instead, LiveQuery will have a very, very similar API to ReactQuery. It’s store will have a fixed structure %{key -> data}, and it will expose some basic functionality to update that store in predefined ways. Basically just:

• getQuery/2 which will get the data for a query, loading it if needed.
• invalidateQuery/2 which will delete a query’s data, and trigger a re-render (during which the deleted query may be loaded if it’s still need).
• updateQuery/2 which will allow updating/setting query in place (useful for stuff like optimistic updates).

By structuring the store around your queries rather than your application state, we alleviate the issues you discussed because it allows selectors to no longer be tied to your application’s structure. Instead they’re tied to your queries, and it’s up to the consumer to coerce the query data it grabs into a shape it can use. This ends up having better performance characteristics since you think about how you’re getting your data.

It’s also worth pointing out that redux has itself realized this issue in the last few years, and Redux ToolKit (RTK) now strongly recommends against handling data loading in application code (e.g. load → coerce to application state structure → store). Instead, now it recommends using something like RTK Query which provides a very similar API to the one I’ve described, where data is loaded and stored in a flat key-value cache-like structure which exposes some generic operations.

The RTK Query docs really sum this up well:

Over the last couple years, the React community has come to realize that “data fetching and caching” is really a different set of concerns than “state management” . While you can use a state management library like Redux to cache data, the use cases are different enough that it’s worth using tools that are purpose-built for the data fetching use case.

I guess you could say I’m banking on the LiveView community realizing that “live data querying and caching” is really a different set of concerns than “state management”.

I am not expert in React development much less Redux but I am vaguely aware that they have been making changes, and that makes some sense. It would probably be more useful for someone who is more familiar with that side of things to chime in here. I would be interested in learning more about how what you’re proposing is different than the reducer pattern (I would probably need to see some examples).

But generally speaking from the point of view of someone who has to manage the DB/API side of things, I am wary of FE oriented state management abstractions that grow and “compete” with the abstractions on the BE (the performance issues I mentioned above were partially caused by FE needs being catered to without enough resistance by my predecessor, who in fairness simply had bigger fish to fry). But LV is really an entirely different ballgame in that regard…which is precisely why so many people are excited about it.

To sum up, right now LV’s approach to state management, if you can call it that, is simple, but cumbersome. The line in your write up that scared me a bit was " In effect, this is a bespoke cache" which is exactly what I feel the problem with Redux’s “reducer” pattern was: the FE built its own cache of the data and then pushed back all the well known problems of managing the cache to the API because fixing issues on the FE where they should be fixed (changing which data was fetched inside the component being changed) became too difficult.

So, this is on me. I misspoke. When I said this is a “bespoke cache”. I actually meant that this is a “generic cache”.

That is what makes it different from a reducer store. You can’t just do anything (like you normally can in a reducer function). The store has a strict structure (key => query) and a fixed set of operations (get_query/invalidate_query/update_query).

Sorry if that was confusing. I unintentionally said the exact opposite of what I meant!

So, I have a beta release out. I took the time to write up exactly what problem LiveQuery solves and how it solves that problem, here.

I will need to write a lot more documentation and tests before I consider this production ready, but I do have a playground app using it here which you can look at to see how this works from a consumer’s perspective. You’ll notice there is absolutely no “reducer” style code here. You effectively just write a simple GenServer that loads/reloads you data, and LiveQuery handles starting it when you need it, preventing it from crashing when you have buggy code, shutting it down when you’re done, caching reads in your live view to ensure stable data through out each render, etc. I does quite a bit, but has a surprisingly simple API.

@tfwright, if you have a moment, I’d love to hear your thoughts. I think this avoids the concerns you had, but maybe I’ve missed something. If so, I’d like to know.

Just a little update

I’ve come to the conclusion that :live_query’s core (on-demand live data) is unrelated to its first use-case (live views). So, I’m going to be splitting the package up into :live_view_query which will depend on a core package called :live_query. Once those package are relatively stable, I have plans to implement :phoenix_channel_query (consume live data on the client via a Phoenix Channel) and :gen_server_query (consume live data from a regular old GenServer).

This will take a little while to do, but I think separating the pieces up front will improve maintainability long term.

The day has come!

I’m excited to announce both LiveQuery and it’s companion live view client library have officially launched.

• LiveQuery — LiveQuery v0.3.1
• LiveQuery.Clients.LiveView — LiveQueryClientLiveView v0.1.3

I’ve tested them both extensively, and written heaps of documentation (and yet somehow there’s still more to write). Please consider these release candidates. They’re still in beta due to a lack of production use, but I feel confident the API surface will stay relatively stable from here until v1.

I plan to produce a lot more documentation and educational material over the next few months, but I believe there’s enough out for an interested observer to get a project using LiveQuery up and running today! If you try and run into issues please ping me, I’m happy to provide white glove support to anyone willing to take the time to use my code.

This is really cool! I like how you separated out :live_query core from :live_view_query and can see how LiveQuery could be a nice abstraction for larger apps that would benefit from a generic cache – especially since it seems like you’re looking into an ETS based approach for better concurrency based on your note here.

That said, I just wanted to point out another option somewhere in between prop drilling and what you’re proposing with LiveQuery to propogate pubsub events that could be beneficial for “medium sized” LiveView apps.

By pairing handle_info/2 and send_update/3 in a LiveView, you can propagate pubsub events without prop drilling by asynchronously updating a live component with new assigns via its id. This pattern could be especially “useful for updating a component that entirely manages its own state” as pointed out by the send_update/3 docs.

So while it’s true that only live views can subscribe to messages via handle_info/2, the conclusion drawn here isn’t necessarily true as a LiveView could relay the pubsub message to a nested LiveComponent via send_update/3 without storing it at the top and passing it down via props.

So, I’ve used the approach you’ve described for a long time, and it absolutely works. However, it has an Achilles heal:

Live components can’t clean up after themselves.

In other words, while a live component can autonomously subscribe a to a topic while mounting via the scheme you’ve described, it can’t cleanup it’s subscription when it unmounts. There’s no callback. This means you’ll end up with orphan subscriptions. For complex live views this could mean a lot of orphan subscriptions.

I tried getting the necessary callbacks added to LiveView, but the suggestion was shot down. Here’s the GitHub issues if you’re interested: Feature request: live component unmount callback · Issue #2454 · phoenixframework/phoenix_live_view · GitHub