Allow hook for LiveDashboard historical data at client connection

Generally I’d not use telemetry for retrieving historical data at all. There could be a config like:

config :live_dashboard, MyLiveDashboard,
  historical_data: %{
    [:my_app, :repo, :query] => {MyStorage, :historical_metric_data, []}
  }

Which would make the dashboard run MyStorage.historical_metric_data([:my_app, :repo, :query]) when it comes to rendering a chart for [:my_app, :repo, :query].

I like that API! And yeah, the idea wasn’t to use telemetry for retrieving it, but only as the trigger for a hook, but I think I like your idea better since it should involve no overhead when there’s no config present, whereas mine involved a receive call and a short timeout… I’ll start playing with that next, thanks @LostKobrakai and welcome any other thoughts of course also.

Presumably in above example if the argument list were non-empty, then historical_metric_data would have arity > 1 ? I suppose having an additional argument that could be a function passed in or something could make sense. Thanks again!

I’d basically prepend the event name to the args of the config, so the actual function would need an arity of +1 to the length of arguments in the config.

Oh so you think instead of MyStorage.historical_metric_data([:my_app, :repo, :query]) it would call MyStorage.historical_metric_data(:my_app, :repo, :query) and +1 to arity for each additional argument in the config list? Yeah not sure but that makes sense/is how I first read your code before I noticed it as a list… I remember seeing a convention like this in some other library recently but blanking what it was maybe something in ecto or telemetry_poller, I forget

Nope, I meant it like your first example: prepend the full event name as the first argument.

apologies, that’s what I’d tried to convey in my first reply which made me think you were correcting a misunderstanding there.

I also wonder though, your example above looks to use Application config, but it looks like the only config live_dashboard currently takes is in the arguments in router to live_dashboard, so guessing we’d pass in historical_data as an additional optional config there, e.g.:

  scope "/live-dashboard" do
    pipe_through [:browser, :admin]
    live_dashboard "/",
      metrics: MyApp.Telemetry, 
      historical_data: %{
        [:my_app, :repo, :query] => {MyStorage, :historical_metric_data, []}
      }
  end

That would probably work as well. Skipping the app env when possible is usually a good thing.

Hopefully also can treat the keylist as a prefix rather than an explicit match so for large numbers of metrics you can just put in the namespace they’re in once, e.g. [:my_app, :repo] and get all events that start with that… probably what you intended but just stating here as I’m toying with starting this in case anyone disagrees.

For anyone interested, a quick v1 of above is pushed up here GitHub - bglusman/phoenix_live_dashboard at historical_data · GitHub and I think this may work but had to break before I could start real testing, code reviews or testing suggestions welcome!

OK, pushed up a few more changes, this is working on the backend but there’s two different initialData functions in assets/js/metrics_live/index.js and I think all that’s needed is for one or both of them to be wired up to check for the hidden history div I added and parse in any spans there the same way it brings in data on liveView updates… if by any chance @LostKobrakai since it looks like you did most of the switch over from charts.js to uPlot you know the easiest/cleanest way to do that, I think that may finish this off, and I’d love your input on the implementation besides that, awesome, if you don’t have time, thanks for your help earlier anyway as it feels close now!