I’ve just been doing something similar, building pluggable charts for borehole data.

What I ended up doing was:

• Pluggable data extractor modules (basically a “read_data” function in each module that receives a keyword list for context - e.g. dates - in my case a borehole identifier)
• A small piece of configuration (currently hard-coded, but in future will be user-configurable) that maps named datasets to the data extractor modules
• On mount or handle_params, I iterate the data extraction configuration to build a map of extracted datasets using the different extractor modules and put them on the socket as socket.assigns.dataset (in my case about 20Mb per liveview process - thankfully there’s only ever a handful of users!). There are also some derived datasets generated from the base ones (e.g. moving averages) - the data extraction process runs the base ones first, then generates the derived ones.

In the case where you are listening for live updates from other parts of the system I would just update the data held in socket.assigns.datasets in the appropriate handle_info

That sorts out the data extraction.

I also have pluggable renderers for each different type of visualisation I want to show (mostly based on Contex FWIW - they emit SVG so no JS hooks are required).

I then have a definition for each display element that defines the named dataset to use, the pluggable rendering module and any settings to control the rendering. This is added to the socket as display_blocks

Finally I have a component that is embedded in the main liveview along the lines of what you have:

<%= live_component(@socket, MyLayoutComponent, datasets: @datasets, display_blocks: @display_blocks, other_stuff: @other_stuff, id: "some-id") %>

MyLayoutComponent handles organising height & width of all the sub-components based on the settings in the passed display_blocks, passing in the correct dataset and settings and invoking the rendering. The whole thing re-renders when anything changes at the moment (data, settings or other things like the currently selected item - aka other_stuff), which is a bit inefficient but actually performs ok. I feel the code and approach is reasonably well organised and easy enough to extend with additional visualisations.

I hope this makes sense!

Have you looked into Surface?

I hacked on it last week and came up with a first iteration, the meat of it is here: lib/code_stats_web/profile_live · f6ba35970a16f8fff89c0cbc38d80ecce9f53b16 · CodeStats / code-stats · GitLab

There are no docblocks yet and there’s a couple of unused functions laying around, but it’s a start. I’ll describe here how it works so that I can copy these to the docblocks later.

The building blocks here are Graphs, DataProviders, SharedData, and the live view itself. The relationships are something like:

ProfileLive <--1..n-- Graphs <--1..n-- DataProviders <--1..n-- SharedData

So the live view has many graphs, these are managed by Graphs.Catalogue. The graphs’ responsibility is to receive data and render it, and they are normal live components (not yet sure if they will need to be stateful or stateless). They also have a function render_wrapper that is used to render them into the live view, so that I don’t need to do just data: @data assigns, but I can instead get accurate assigns for each graph. Example from UserInfo:

<%=
  live_component(
    @socket,
    __MODULE__,
    user: assigns[DataProviders.UserInfo],
    total_xp: assigns[DataProviders.TotalXP].total_xp,
    recent_xp: assigns[DataProviders.TotalXP].recent_xp,
    last_day_coded: assigns[DataProviders.LastDayCoded].last_day_coded
  )
%>

I don’t actually know yet how this affects change tracking, because the live updates aren’t implemented, but I’m hoping for the best. And I think it’s nicer that the component only gets the data it needs and not all of it.

Now the graphs in turn have a set of DataProviders that they get data from. The purpose of the providers is to retrieve the initial dataset, and to update it (in the future) when any events come in. Each graph can get data from many providers and the providers can be used for many graphs.

Now, since some providers rely on the same data (like the last 12 hours of events) and I don’t want to repeat the queries, there is finally the SharedData module. Providers specify what shared data they need and SharedData is responsible for requesting it.

Finally we get to combining all of this in the live view:

# Get all the graphs available
graphs = Graphs.Catalogue.graphs()
# Get data providers required by the graphs (now that I think of this, this should take in the graphs as argument)
data_providers = Graphs.Catalogue.data_providers()
# Get the shared data requirements of the providers
required_shared_data = DataProviders.Utils.required_shared_data(data_providers)
# Retrieve the shared data
shared_data = DataProviders.SharedData.get_data(required_shared_data, user)
# Retrieve the initial data to show in the graphs based on the shared data
initial_data = Graphs.Catalogue.get_initial_data(shared_data, data_providers, user)
# Given the graphs, providers, and initial data, build the socket assigns using the module names as keys
socket = Graphs.Catalogue.assign_datas(socket, graphs, data_providers, initial_data)

Then in the template, I use

<%= Graphs.Catalogue.render_graphs(assigns) %>

which in turn is just a for statement that calls the render_wrapper of all the graphs.

Now, this all works great currently, but I don’t have the live updates implemented yet, so it’s not a full featured prototype. Additionally, I have a couple of open questions still:

• Currently, like shown above, I don’t use render but instead just call the functions directly. I wonder if this affects LV’s change tracking ability? I don’t see any other reason to use render because it would require a view and extra hassle.
• I have functions that look like this:

  def render_graphs(assigns) do
    ~L"""
    <%= for graph <- @graphs do %>
      <%= graph.render_wrapper(assigns) %>
    <% end %>
    """
  end
  

  and I wonder if the ~L wrapping is necessary or if I could just run the code directly.

I know that’s a lot to read and I don’t expect anyone to invest too much time into this, but if you have any comments or ideas or insults, I’d be glad to hear them.