LiveView Streams with nested data updating weirdly

I am using the sweet new streams feature to load a timeline of closed pull requests via infinite scroll. It almost works. My streamed objects are not PRs, they are tuples of {<date>, <list of prs>}. Each item in this stream is a section (headered by the date) and within each section is the list of PRs closed on that date.

On first pass my data loads properly. When I trigger the first load of additional data, the new data appears correctly but the PRs from the first batch are gone. The dates and sections still appear, but the sections (ie, list of PRs for that date) are empty.

Am I doing something boneheaded with my state tracking or does LiveView not like these nested lists? Here’s the relevant code:

# changelog_live.ex

def mount(_params, _session, socket) do
  if connected?(socket) do
    year = Date.utc_today().year
    # nb: bucketize/1 function creates the list of {<date>, <list_of_prs>} tuples
    pull_requests = Mrgr.PullRequest.socks(current_user, year) |> bucketize()

    |> assign(:year, year)
    |> stream(:pull_requests, pull_requests, dom_id: &dom_id/1)
    |> ok()

defp dom_id({date, _prs}), do: "week-#{format_week(date)}"

def load_prs(%{assigns: assigns} = socket) do
  Mrgr.PullRequest.socks(assigns.current_user, assigns.year)
  |> bucketize()
  |> Enum.reduce(socket, fn pr, s ->
    # `pr` is the tuple
    stream_insert(s, :pull_requests, pr)

and the template:

# changelog_live.html.heex
<div id="changelog-list-body" phx-update="stream">
  <div :for={{dom_id, {date, prs}} <- @streams.pull_requests} id={dom_id}>
      <.h3><%= format_week(date) %></.h3>
      <span><%= Enum.count(prs) %></span>
    <.pr_list pull_requests={prs} /><!-- function component that renders list of PRs -->

the paging code has been omitted for brevity.

At first I see this, which is correct:

After I scroll down to trigger the stream append, the same area looks like this, which is incorrect:



good news! I’m a dope. (I think I answer all own ElixirForum questions this way :sweat_smile:).

My problem was that my bucketize/1 function had a bug that, when called subsequently during scrolling, would overwrite the previous keys with empty data. In other words, on first pass it would correctly return

  {2023, [%PullRequest{}]}

And on the second pass would return

  {2023, []},   # <--- this entry should not be here!
  {2022, [%PullRequest{}]}

The bottom line was my paging logic was incorrect and didn’t respect the yearly boundaries, so LiveView saw that the existing keys were updated with empty lists and happily blew away my data. Fixing this bug solved my problem and infinite scrolling works properly. Yay!


Great! This was in my list to check out so happy to see it was your bug and not mine :smiley: