btihen

btihen

Large infinite scrolls to last record, but not back to the first record

I have a long list of records in a table (~1000 records) - if I scroll down about 4 times, then I can no longer return to the top of the list - it seems to stop about 20 records or so before the first record. I was able to fix the scrolling up problem with a lot of javascript, it breaks with socket updates or document uploads, so I have reverted to the phoenix default as suggested at: Bindings — Phoenix LiveView v1.2.5 , except that we are using a table with a sticky header - just the table body scrolls and not the header. If I reload the page I can get back to the top record.

Does anyone see what we are doing wrong? Here is our code and what we versions we are running.

We are using:

"phoenix": "1.8.0",
"phoenix_ecto": "4.6.5",
"phoenix_html": "4.2.1",
"phoenix_live_reload": "1.6.0", 
"phoenix_live_view": "1.1.3",

and

erlang 26.2.5.9
elixir 1.18.3-otp-26

Here is our index.heex

<div class="flex">
  <div class="m-auto text-center text-2xl font-semibold leading-8 text-zinc-800">
    {gettext("Listing Invoices")}
  </div>
  <form class="m-auto px-16 text-center flex-auto" phx-change="search">
    <input
      class="w-full rounded-lg border-zinc-400 text-zinc-900 placeholder-zinc-400 text-sm focus:ring-0"
      type="text"
      name="search"
      phx-debounce="100"
      placeholder={gettext("type to search invoices...")}
    />
  </form>
  <.link patch={~p"/invoices/new"}>
    <.button>{gettext("Upload")}</.button>
  </.link>
  <%= if @has_sftp_host do %>
    <.button phx-click="sftp_import" class="ml-2 bg-sky-600 hover:bg-sky-700 text-white">
      <.icon name="hero-arrow-down-tray" class="mr-1 h-5 w-5 inline-block align-middle" />
      {gettext("SFTP Import")}
    </.button>
  <% end %>
</div>
<.table id="invoices" rows={@streams.invoices}>
  <:col :let={{_id, invoice}} label={gettext("Created")}>
    {invoice.inserted_at |> DateTime.shift(hour: 1) |> Calendar.strftime("%d.%m.%Y %H:%M")}
  </:col>
  <:col :let={{_id, invoice}} label={gettext("Original filename")}>
    {invoice.original_filename}
  </:col>
  <:col :let={{_id, invoice}} label={gettext("Storage file")}>
    <.link :if={invoice.storage_filepath} navigate={~p"/invoices/#{invoice.storage_filepath}"}>
      {invoice.storage_filepath}
    </.link>
  </:col>
  <:col :let={{_id, invoice}} label={gettext("State")} class="min-w-48">
    <.tooltip>
      <span class={state_color_for(invoice)}>{invoice.state}</span>
      <.tooltip_content :if={invoice.error_reason} class="bg-primary text-white">
        <p>{invoice.stage} stage:</p>
        <p>{invoice.error_reason}</p>
      </.tooltip_content>
    </.tooltip>
  </:col>
  <:action :let={{_id, invoice}}>
    <.link class="events" phx-click={show_events_modal(invoice.id)}>
      <.icon
        name="hero-information-circle text-sky-500"
        class="h-5 w-5 hidden group-hover:inline-block"
      />
    </.link>
  </:action>

  <:action :let={{id, invoice}}>
    <.link
      :if={invoice.tenant.allow_invoice_deletion}
      class="delete"
      phx-click={JS.push("delete", value: %{id: invoice.id}) |> hide("##{id}")}
      data-confirm={gettext("Are you sure?")}
    >
      <.icon name="hero-trash text-red-600" class="h-5 w-5 hidden group-hover:inline-block" />
    </.link>
  </:action>
</.table>

<.modal :if={@live_action in [:new]} id="invoice-modal" show on_cancel={JS.patch(~p"/invoices")}>
  <.live_component
    module={AvkServiceWeb.InvoiceLive.FormComponent}
    id="file-upload"
    title={@page_title}
    action={@live_action}
    auth_context={@auth_context}
    patch={~p"/invoices"}
  />
</.modal>

<.modal id="invoice-events-modal">
  <.header>
    {@invoice_filename}
    <:subtitle>{gettext("Processing Pipeline Log")}</:subtitle>
  </.header>

  <div class="flex flex-row text-sm font-semibold mt-8">
    <div class="basis-1/4 pr-2">Timestamp</div>
    <div class="basis-3/4 pr-2">Message</div>
  </div>

  <div class="max-h-80 overflow-scroll mt-2">
    <div :for={event <- @invoice_events} class="flex flex-row text-sm">
      <div class="basis-1/4 pr-2">{event.timestamp}</div>
      <div class="basis-3/4 pr-2">
        <span class={state_color_for(event)}>{event.message}</span>
      </div>
    </div>
  </div>
</.modal>

Here is our index.ex

defmodule AvkServiceWeb.InvoiceLive.Index do
  use AvkServiceWeb, :live_view

  require Logger

  alias AvkService.Admin
  alias AvkService.Invoicing
  alias AvkService.Repo
  alias AvkService.Util.PubSubTopic

  @sftp_consumer Application.compile_env(
                   :avk_service,
                   [:boundaries, :sftp_consumer],
                   AvkService.Boundary.SftpConsumer
                 )
  @storage Application.compile_env(
             :avk_service,
             [:boundaries, :storage],
             AvkService.Boundary.Storage
           )

  @impl true
  def mount(_params, _session, socket) do
    tenant_id = socket.assigns.auth_context.tenant_id
    AvkServiceWeb.Endpoint.subscribe(PubSubTopic.invoices(tenant_id))
    sftp_host = Admin.get_sftp_host_from_tenant_id(tenant_id)
    has_sftp_host = sftp_host != nil

    {:ok,
     socket
     |> assign(:has_sftp_host, has_sftp_host)
     |> assign(:render_events, false)
     |> assign(:invoice_events, [])
     |> assign(:invoice_filename, "")
     |> assign(page: 1, per_page: 20, search_terms: "")
     |> paginate_invoices(1)}
  end

  defp paginate_invoices(socket, new_page, reset \\ false) when new_page >= 1 do
    %{per_page: per_page, page: cur_page, search_terms: search_terms} = socket.assigns

    invoices =
      Invoicing.list_invoices(
        tenant_id: socket.assigns.auth_context.tenant_id,
        offset: (new_page - 1) * per_page,
        limit: per_page,
        search: search_terms
      )

    {invoices, at, limit} =
      if new_page >= cur_page do
        {invoices, -1, per_page * 3 * -1}
      else
        {Enum.reverse(invoices), 0, per_page * 3}
      end

    socket =
      case invoices do
        [] ->
          socket
          |> assign(end_of_timeline?: at == -1)

        [_ | _] ->
          socket
          |> assign(end_of_timeline?: false)
          |> assign(:page, new_page)
      end

    socket |> stream(:invoices, invoices, at: at, limit: limit, reset: reset)
  end

  @impl true
  def handle_params(params, _url, socket) do
    {:noreply, apply_action(socket, socket.assigns.live_action, params)}
  end

  defp apply_action(socket, :index, _params) do
    socket
    |> assign(:page_title, gettext("Listing Invoices"))
    |> assign(:invoice, nil)
  end

  defp apply_action(socket, :new, _params) do
    socket
    |> assign(:page_title, gettext("Upload Invoices"))
  end

  @impl true
  def handle_info(%{event: "created", payload: new_invoice}, socket) do
    {:noreply, stream_insert(socket, :invoices, new_invoice, at: 0)}
  end

  @impl true
  def handle_info(%{event: "failed", payload: new_invoice}, socket) do
    {:noreply, stream_insert(socket, :invoices, new_invoice, at: 0)}
  end

  @impl true
  def handle_info(%{event: "state_changed", payload: invoice}, socket) do
    {:noreply, stream_insert(socket, :invoices, invoice)}
  end

  def handle_event("next-page", _params, socket) do
    {:noreply, paginate_invoices(socket, socket.assigns.page + 1)}
  end

  def handle_event("prev-page", %{"_overran" => true} = _params, socket) do
    # When overran, reset to the first page and reset the stream/window
    {:noreply, paginate_invoices(socket, 1, true)}
  end

  def handle_event("prev-page", _params, socket) do
    if socket.assigns.page > 1 do
      {:noreply, paginate_invoices(socket, socket.assigns.page - 1)}
    else
      {:noreply, socket}
    end
  end

  def handle_event("search", %{"search" => search_terms}, socket) do
    {:noreply, paginate_invoices(socket |> assign(page: 1, search_terms: search_terms), 1, true)}
  end

  @impl true
  def handle_event("show_events", %{"id" => id}, socket) do
    invoice = Invoicing.get_invoice!(id) |> Repo.preload(:events)

    socket =
      socket
      |> assign(:render_events, true)
      |> assign(:invoice_filename, invoice.original_filename)
      |> assign(:invoice_events, invoice.events |> format_event_data())

    {:noreply, socket}
  end

  @impl true
  def handle_event("delete", %{"id" => id}, socket) do
    invoice = Invoicing.get_invoice!(id)

    with {:ok, :deleted} <- @storage.maybe_delete(invoice.storage_filepath),
         {:ok, _} <- Invoicing.delete_invoice(invoice) do
      {:noreply, stream_delete(socket, :invoices, invoice)}
    else
      _ -> {:noreply, socket |> put_flash(:error, gettext("Delete failed"))}
    end
  end

  @impl true
  def handle_event("sftp_import", _params, socket) do
    username = socket.assigns.auth_context.username
    tenant_id = socket.assigns.auth_context.tenant_id
    tenant = Admin.get_tenant!(tenant_id)
    Logger.info("SFTP Import: for tenant '#{tenant.name}' manually triggered by #{username}")
    fetch_results = @sftp_consumer.consume(tenant)

    case fetch_results do
      {:ok, files_results} when is_list(files_results) and length(files_results) > 0 ->
        success_message = summarized_results(files_results)
        {:noreply, put_flash(socket, :info, success_message)}

      {:ok, _} ->
        {:noreply, put_flash(socket, :info, gettext("No SFTP files to import."))}

      {:error, reason} ->
        {:noreply,
         put_flash(
           socket,
           :error,
           gettext("SFTP import failed: %{reason}", reason: inspect(reason))
         )}
    end
  end

  def state_color_for(invoice) do
    case invoice.state do
      "SUCCESS" -> "text-green-600 font-semibold"
      "WARNING" -> "text-yellow-600 font-semibold"
      "ERROR" -> "text-red-600 font-semibold"
      "REJECTED" -> "text-red-600 font-semibold"
      "OK" -> "text-green-600 font-semibold"
      _ -> "text-black"
    end
  end

  def show_events_modal(js \\ %JS{}, invoice_id) do
    dom_id = "invoice-events-modal"

    js
    |> JS.push("show_events", value: %{id: invoice_id})
    |> JS.show(to: "##{dom_id}")
    |> JS.show(
      to: "##{dom_id}-bg",
      transition: {"transition-all transform ease-out duration-300", "opacity-0", "opacity-100"}
    )
    |> show("##{dom_id}-container")
    |> JS.add_class("overflow-hidden", to: "body")
    |> JS.focus_first(to: "##{dom_id}-content")
  end

  defp format_event_data(events) do
    events
    |> Enum.map(fn event ->
      %{
        timestamp:
          event.inserted_at |> DateTime.shift(hour: 1) |> Calendar.strftime("%d.%m.%Y %H:%M:%S"),
        message: event.message |> String.replace(",", ", "),
        state: event.state
      }
    end)
  end

  defp summarized_results(file_results) do
    # Count the different types of results
    total_files = length(file_results)

    # Count submitted files
    submitted_count =
      Enum.count(file_results, fn
        {:ok, {_filename, _process_action, _archive_action}} -> true
        _ -> false
      end)

    # Count ignored files (invalid mime type)
    ignored_count =
      Enum.count(file_results, fn
        {:ignore, {_filename, _process_action, _archive_action}} -> true
        _ -> false
      end)

    # All other errors are failures
    failed_count =
      Enum.count(file_results, fn
        {:error, {_filename, _process_action, _archive_action}} -> true
        _ -> false
      end)

    gettext(
      "SFTP Import Results: %{total} files imported, (%{submitted} submitted, %{ignored} ignored, %{failed} failed)",
      total: total_files,
      submitted: submitted_count,
      ignored: ignored_count,
      failed: failed_count
    )
  end
end

Here is our table core_component (in case it is out of date):


  @doc ~S"""
  Renders a table with generic styling.

  ## Examples

      <.table id="users" rows={@users}>
        <:col :let={user} label="id"><%= user.id %></:col>
        <:col :let={user} label="username"><%= user.username %></:col>
      </.table>
  """
  attr :id, :string, required: true
  attr :rows, :list, required: true
  attr :row_id, :any, default: nil, doc: "the function for generating the row id"
  attr :row_click, :any, default: nil, doc: "the function for handling phx-click on each row"
  attr :page, :integer, default: 1, doc: "page to render"
  attr :end_of_timeline?, :boolean, default: false, doc: "are we at the end of the endless page?"

  attr :row_item, :any,
    default: &Function.identity/1,
    doc: "the function for mapping each row before calling the :col and :action slots"

  slot :col, required: true do
    attr :label, :string
    attr :class, :string
  end

  slot :action, doc: "the slot for showing user actions in the last table column"

  def table(assigns) do
    assigns =
      with %{rows: %Phoenix.LiveView.LiveStream{}} <- assigns do
        assign(assigns, row_id: assigns.row_id || fn {id, _item} -> id end)
      end

    ~H"""
    <%!-- We calc the height with a magic number since I couldn't make the flexbox auto sizing work --%>
    <div class="h-[calc(100vh_-_280px)] overflow-auto pr-4 mt-12">
      <table class="w-[40rem] sm:w-full">
        <thead class="text-left leading-6 text-zinc-600 bg-white">
          <tr>
            <th
              :for={col <- @col}
              class={"p-0 pb-4 pr-6 font-old #{col[:class]} sticky top-0 bg-white z-50"}
            >
              {col[:label]}
            </th>
            <th :if={@action != []} class="relative p-0 pb-4">
              <span class="sr-only">{gettext("Actions")}</span>
            </th>
          </tr>
        </thead>
        <tbody
          id={@id}
          phx-update="stream"
          phx-viewport-top={@page > 1 && "prev-page"}
          phx-viewport-bottom={@id == "invoices" && !@end_of_timeline? && "next-page"}
          phx-page-loading
          class={[
            if(@end_of_timeline?, do: "pb-10", else: "pb-[calc(200vh)]"),
            if(@page == 1, do: "pt-10", else: "pt-[calc(200vh)]")
          ]}
          class="relative divide-y divide-zinc-100 border-t border-zinc-200 leading-6 text-zinc-700"
        >
          <tr :for={row <- @rows} id={@row_id && @row_id.(row)} class="group hover:bg-zinc-50">
            <td
              :for={col <- @col}
              phx-click={@row_click && @row_click.(row)}
              class={["relative p-0", @row_click && "hover:cursor-pointer"]}
            >
              <div class="block py-4 pr-6">
                <span class="absolute -inset-y-px right-0 -left-4 group-hover:bg-zinc-50 sm:rounded-l-xl" />
                <span class={["relative"]}>
                  {render_slot(col, @row_item.(row))}
                </span>
              </div>
            </td>
            <td :if={@action != []} class="relative w-20 p-0">
              <div class="relative whitespace-nowrap py-4 text-right text-sm font-medium">
                <span class="absolute -inset-y-px -right-4 left-0 group-hover:bg-zinc-50 sm:rounded-r-xl" />
                <span
                  :for={action <- @action}
                  class="relative ml-4 font-semibold leading-6 text-zinc-900 hover:text-zinc-700"
                >
                  {render_slot(action, @row_item.(row))}
                </span>
              </div>
            </td>
          </tr>
        </tbody>
      </table>
    </div>
    """
  end

Marked As Solved

btihen

btihen

I have posted a basic phx project that has an example of the code that works (a basic list and one that mimics a table) also have the table that won’t return to the top record and also a very slow brut force table (using the unchanged default code phx generated) - the simplified problem and solution can be found at:

https://github.com/btihen-dev/phx_infinite_scroll_table_solution

In case someone has the time and ability to debug this strange behavior.

PS - with VERY large number of records (over 2K) about 1 in 10 or 20 times even the default solution is flakey and won’t return to the first record (I think it is when I scroll too fast)

Also Liked

btihen

btihen

PS - I made a simple table (without a sticky header) with 1000 records and I had the same problem. I couldn’t get back to the top of the list after a few pages. It appears with logging that the prev-page hook never gets called - even at the top of the stream list.

Where Next?

Popular in Questions Top

hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a &gt; b) do {:ok, "a"} end if (a &lt; b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: The documentation above suggests that while ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New

Other popular topics Top

Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

We're in Beta

About us Mission Statement