Structuring a LiveView with many similar (dynamic?) components

I’ve just been doing something similar, building pluggable charts for borehole data.

What I ended up doing was:

• Pluggable data extractor modules (basically a “read_data” function in each module that receives a keyword list for context - e.g. dates - in my case a borehole identifier)
• A small piece of configuration (currently hard-coded, but in future will be user-configurable) that maps named datasets to the data extractor modules
• On mount or handle_params, I iterate the data extraction configuration to build a map of extracted datasets using the different extractor modules and put them on the socket as socket.assigns.dataset (in my case about 20Mb per liveview process - thankfully there’s only ever a handful of users!). There are also some derived datasets generated from the base ones (e.g. moving averages) - the data extraction process runs the base ones first, then generates the derived ones.

In the case where you are listening for live updates from other parts of the system I would just update the data held in socket.assigns.datasets in the appropriate handle_info

That sorts out the data extraction.

I also have pluggable renderers for each different type of visualisation I want to show (mostly based on Contex FWIW - they emit SVG so no JS hooks are required).

I then have a definition for each display element that defines the named dataset to use, the pluggable rendering module and any settings to control the rendering. This is added to the socket as display_blocks

Finally I have a component that is embedded in the main liveview along the lines of what you have:

<%= live_component(@socket, MyLayoutComponent, datasets: @datasets, display_blocks: @display_blocks, other_stuff: @other_stuff, id: "some-id") %>

MyLayoutComponent handles organising height & width of all the sub-components based on the settings in the passed display_blocks, passing in the correct dataset and settings and invoking the rendering. The whole thing re-renders when anything changes at the moment (data, settings or other things like the currently selected item - aka other_stuff), which is a bit inefficient but actually performs ok. I feel the code and approach is reasonably well organised and easy enough to extend with additional visualisations.

I hope this makes sense!

Thanks! This hints that my original idea would at least work. Shame that you lose the LV change tracking accuracy, though, so willing to hear any other ideas that would allow me to keep it.

There’s nothing structural there that makes me lose change-tracking. Just laziness on my part in not wrapping the rendering functions into LiveComponents. As it happens, I now have a glitch in the rendering sequence and some JS interop that is forcing me to do it properly for a couple of them!

Have you looked into Surface?

I have not. Is there anything specific there that would apply to this situation?

I have started building a first iteration of this and will post here how it works when there’s an MVP.

Disclaimer: I’ve contributed to Surface and built the SurfaceFormatter, so I’m biased.

Surface provides a somewhat “React-like” syntax for building reusable components. You can define the interface of what data should be passed into the component, with some compile time checks. It’s a more mature development experience for doing components than vanilla LiveView components in my opinion.

I hacked on it last week and came up with a first iteration, the meat of it is here: lib/code_stats_web/profile_live · f6ba35970a16f8fff89c0cbc38d80ecce9f53b16 · CodeStats / code-stats · GitLab

There are no docblocks yet and there’s a couple of unused functions laying around, but it’s a start. I’ll describe here how it works so that I can copy these to the docblocks later.

The building blocks here are Graphs, DataProviders, SharedData, and the live view itself. The relationships are something like:

ProfileLive <--1..n-- Graphs <--1..n-- DataProviders <--1..n-- SharedData

So the live view has many graphs, these are managed by Graphs.Catalogue. The graphs’ responsibility is to receive data and render it, and they are normal live components (not yet sure if they will need to be stateful or stateless). They also have a function render_wrapper that is used to render them into the live view, so that I don’t need to do just data: @data assigns, but I can instead get accurate assigns for each graph. Example from UserInfo:

<%=
  live_component(
    @socket,
    __MODULE__,
    user: assigns[DataProviders.UserInfo],
    total_xp: assigns[DataProviders.TotalXP].total_xp,
    recent_xp: assigns[DataProviders.TotalXP].recent_xp,
    last_day_coded: assigns[DataProviders.LastDayCoded].last_day_coded
  )
%>

I don’t actually know yet how this affects change tracking, because the live updates aren’t implemented, but I’m hoping for the best. And I think it’s nicer that the component only gets the data it needs and not all of it.

Now the graphs in turn have a set of DataProviders that they get data from. The purpose of the providers is to retrieve the initial dataset, and to update it (in the future) when any events come in. Each graph can get data from many providers and the providers can be used for many graphs.

Now, since some providers rely on the same data (like the last 12 hours of events) and I don’t want to repeat the queries, there is finally the SharedData module. Providers specify what shared data they need and SharedData is responsible for requesting it.

Finally we get to combining all of this in the live view:

# Get all the graphs available
graphs = Graphs.Catalogue.graphs()
# Get data providers required by the graphs (now that I think of this, this should take in the graphs as argument)
data_providers = Graphs.Catalogue.data_providers()
# Get the shared data requirements of the providers
required_shared_data = DataProviders.Utils.required_shared_data(data_providers)
# Retrieve the shared data
shared_data = DataProviders.SharedData.get_data(required_shared_data, user)
# Retrieve the initial data to show in the graphs based on the shared data
initial_data = Graphs.Catalogue.get_initial_data(shared_data, data_providers, user)
# Given the graphs, providers, and initial data, build the socket assigns using the module names as keys
socket = Graphs.Catalogue.assign_datas(socket, graphs, data_providers, initial_data)

Then in the template, I use

<%= Graphs.Catalogue.render_graphs(assigns) %>

which in turn is just a for statement that calls the render_wrapper of all the graphs.

Now, this all works great currently, but I don’t have the live updates implemented yet, so it’s not a full featured prototype. Additionally, I have a couple of open questions still:

• Currently, like shown above, I don’t use render but instead just call the functions directly. I wonder if this affects LV’s change tracking ability? I don’t see any other reason to use render because it would require a view and extra hassle.
• I have functions that look like this:

  def render_graphs(assigns) do
    ~L"""
    <%= for graph <- @graphs do %>
      <%= graph.render_wrapper(assigns) %>
    <% end %>
    """
  end
  

  and I wonder if the ~L wrapping is necessary or if I could just run the code directly.

I know that’s a lot to read and I don’t expect anyone to invest too much time into this, but if you have any comments or ideas or insults, I’d be glad to hear them.

Looks like I lose change tracking with this method. I guess it’s because I just call a function with <%= Graphs.Catalogue.render_graphs(assigns) %> and then the function looks like this:

def render_graphs(assigns) do
  ~L"""
  <%= for graph <- @graphs do %>
    <%= graph.render_wrapper(assigns) %>
  <% end %>
  """
end

Is that it? How could I keep change tracking with this structure?

EDIT: José mentioned this should be fine, so I guess there’s just something I don’t understand about LV. I will study more myself before making any more posts.

I have found Surface to be a very nice solution for liveview, it allows you to build “components” which are easily composable and the end result feels very much like writing HTML, only with magic tags which explode into whole liveview components. Very nice.

I think this git repo might be a great example of what you are trying to develop. I found it useful both for it’s content and for ideas on using LiveView. Shout out to the author!

https://github.com/code-shoily/covid19

Hey, thanks for the recommendation.

I quickly looked at the repository and it seems the components are still hardcoded in the live layout (for example DashboardLive). I.e. the component tags are written out in the template so their order is not easy to control. I wanted to be able to not specify the graph components in the template if possible, so that I can configure them per-user. I know I can use CSS Grid for positioning out of order on the screen, but I’d like to try if I can do it generically in LV.

I will come back to this thread when I have something working.