Add `:params` opt to JS.patch and JS.navigate, and opt to merge `phx-value-*`

Phoenix Forum Proposals/Suggestions
,
1

Goal: To make JS.patch and JS.navigate more interoperable with JS.push.

Scenario: Imagine making a reusable Phoenix component and you want to allow the user of your component to customize its interactive behavior. The JS module already gives us good tools for this, but if you want to allow for customized navigation events you have to use a callback function to generate the JS command (as demonstrated by the row_click attr in the default CoreComponents table.

This is different from push, which does support values, while patch and navigate don’t receive the values from phx-value or support setting their own values. This makes sense contextually as an event push is different from navigation.

With a slight modification we could allow users of .patch and .navigate to opt in to receiving the same phx-value-* as JS.push. Then in our component example above, the user of the component would have full control over the behavior without introducing additional complexity via callback functions.

Suggestion:

  1. Add :params as a way to pass query params to JS.patch and JS.navigate
  2. Add :values_as_params to control which phx-value-* should be included as query params. Defaults to false (or empty list?).

Examples:

cmd_1 = JS.patch("/things", values_as_params: true)

~H"""
<!--- a bunch of posts -->
<button phx-click={cmd_1} phx-value-page="2" phx-value-size="10">Next Page</button>
"""

Clicking the button would do a patch to /things?page=2&size=10.

cmd_2 = JS.patch("/things", values_as_params: [:page])

~H"""
<!--- a bunch of posts -->
<button phx-click={cmd_1} phx-value-page="2" phx-value-size="10">Next Page</button>
"""

Clicking the button would do a patch to /things?page=2.

cmd_3 = JS.patch("/things", params: %{size: 100}, values_as_params: [:page]

~H"""
<!--- a bunch of posts -->
<button phx-click={cmd_1} phx-value-page="2" phx-value-size="10">Next Page</button>
"""

Goes to /things?page=2&size=100
Using :params the user of the component can override or introduce their own params.

This would significantly simplify designing data driven components like tables. Users could easily reuse the same components for purely ephemeral uses using JS.push and more durable ones using JS.patch without having to change the component code.

I’d be happy to help work on an implementation if anyone else sees value in this.

Alternative approaches:

  1. Use callbacks functions to generate the JS commands, passing in whatever data is relevant (as mentioned above)
  2. Use JS.dispatch along with a custom event listener in app.js to implement the suggested functionality.
  3. We could choose to support custom JS commands instead, something like JS.custom where the user would register custom commands similar to how hooks are defined in the socket. (Basically a more streamlined version of alternative 2 above). This approach could also allow further customizations that are out of scope for the core LiveView library like supporting path params in the Javascript commands like /things/:id

Sketch of alternative 3 above:

const commands = {
  ParamPatch(view, el, href, opts) {
     const finalHref = determineHref(href, opts)
    # or a path param version might do
    # const finalHref = subsititutePathParams("/things/:id", {id: 5}
     view.liveSocket.pushHistoryPatch(finalHref, ... etc)
   }
}
let liveSocket = new LiveSocket("/live", Socket, { commands, ...}

  JS.custom_command("ParamPatch", opts?)
2 Likes
2

Hey @jakeprem,

I was planning to send a proposal to support modifying query parameters through JS.patch/navigate, for example like this:

JS.patch("?foo=bar") # -> sets current page's query parameters to foo=bar
JS.patch(query: %{"foo" => "bar"}, merge_query: true) # -> sets query parameter foo=bar and keeps existing
JS.patch(drop_query: ["foo"]) # -> removes the foo key from query paramters

I think modifying only parts of the URL is important for building complex components relying on URL state, therefore merging and dropping parameters, but keeping unrelated ones should be something we support in my opinion.

In a previous project I did something like this:

def patch_params(%JS{} = js, params, opts) do
  opts = Keyword.validate!(opts, [:type, :path, :remove_params])

  JS.dispatch(
    js,
    "patch-params",
    detail:
      Map.merge(
        %{params: Plug.Conn.Query.encode(params)},
        Map.new(opts)
      )
  )
end

window.addEventListener("link:patch-params", (e) => {
  const type = e.detail?.type || "patch";
  const path = e.detail?.path || document.location.pathname;
  const params = new URLSearchParams(e.detail?.params || "");
  const removeParams = e.detail?.remove_params || [];

  const searchParams = new URLSearchParams(document.location.search);
  Array.from(params.keys()).forEach((key) => searchParams.delete(key));
  Array.from(params.entries()).forEach(([key, value]) => {
    searchParams.append(key, value);
  });
  removeParams.forEach((key) => {
    searchParams.delete(key);
  });
  const search = searchParams.toString();
  const href = search ? (path + "?" + search) : path;
  if (type == "patch") {
    liveSocket.pushHistoryPatch(e, href, replace ? "replace" : "push", e.target);
  } else if (type == "navigate") {
    liveSocket.historyRedirect(e, href, replace ? "replace" : "push");
  } else {
    document.location.href = href;
  }
});

But this also relies on private liveSocket APIs (pushHistoryPatch/historyRedirect), so we should have an official way to do this.

To give some more light on the underlying issue: if you build a complex component that relies on URL state, for example a table with multiple filters (like selected countries, etc.), each filter needs a way to update the URL correctly. One approach that is often used is to store the parameters in the assigns and then construct the link on the server:

def handle_params(params, _uri, socket) do
  socket
  |> assign(:params, params)
  |> assign_country_filter(params)
  |> ...
  |> then(&{:ok, &1})
end

def render(assigns) do
  ~H"""
  ...
  <div id="country-filter">
    <.link :for={country <- @all_countries} patch={~p"/my-live-view?#{Map.merge(@params, %{"country" => country.iso})"}>{country.name}</.link>
  </div>
  ...
  """
end

This approach works, but has the big downside of creating huge diffs over the wire, re-sending each link whenever any parameter on the page changes.

So I’m not sure yet about overloading phx-value-* for JS.patch/navigate, but I’m definitely in favor for a way to manage query parameters without having to deal with the whole URL!

7 Likes
3

@steffend Was this implemented or is it still in the proposal phase? Would you be willing to accept a PR if the design has been approved? This would be a very helpful feature for a project I’m working on.

Specifically, I am working with array-valued keys in the query string for sorting a table with dynamic order, so the query string has something like sort[]=first_col-asc&sort[]=second_col-desc in it along with other params for things like pagination and dynamic filters. I think in addition to the :query and :drop_query options in your mockup, something like JS.patch(filter_query: fn {key, val} -> key == "sort[]" and !String.starts_with?(val, "first_col") end) would be useful for my use-case. Perhaps also a map_query option that takes a similar function over the {key, value} tuple. That would allow for specifying a filter and a map function to do things like sort order cycling on-click:

JS.patch(
  filter_query: fn {key, val} ->
    key == "sort[] and String.starts_with?(val, "first_col") and !String.ends_with?(val, "desc")
  end,
  map_query: fn {key, val} ->
    if key == "sort[]" and String.starts_with?(val, "first_col") and String.ends_with?(val, "asc") do
      {key, String.replace_suffix(val, "asc", "desc")
    else
      {key, val}
    end
  end,
)

I’m sure there’s a more efficient way to do this kind of thing, but the map/filter approach would probably be useful for other situations as well like removing all params with a common prefix that have been added dynamically by a component.

If that kind of api is not desired, I would still be happy to work on whatever the Phoenix team wants to implement for this feature if no one else has already picked it up. :slight_smile:

2 Likes
4

I did not invest the time to send an official proposal (or just do a PR) yet. It’s not that urgent any more as we now have an official way to change the URL from JS using liveSocket.js().patch() / liveSocket.js().navigate(), so the patch_params approach I sent earlier can be done without using private API like this:

window.addEventListener("link:patch-params", (e) => {
  const type = e.detail?.type || "patch";
  const path = e.detail?.path || document.location.pathname;
  const params = new URLSearchParams(e.detail?.params || "");
  const removeParams = e.detail?.remove_params || [];

  const searchParams = new URLSearchParams(document.location.search);
  Array.from(params.keys()).forEach((key) => searchParams.delete(key));
  Array.from(params.entries()).forEach(([key, value]) => {
    searchParams.append(key, value);
  });
  removeParams.forEach((key) => {
    searchParams.delete(key);
  });
  const search = searchParams.toString();
  const href = search ? (path + "?" + search) : path;
  if (type == "patch") {
    liveSocket.js().patch(href, { replace: e.detail?.replace });
  } else if (type == "navigate") {
    liveSocket.js().navigate(href, { replace: e.detail?.replace });
  } else {
    document.location.href = href;
  }
});

Now, since the idea of that script is to serialize the operations to do in the JS command, passing a function for map/filter is not something we can do. Instead, you could extend the snippet I shared with a new option, e.g. drop_matching: ["sort[]", "first_col"] and then add extra JS to handle that.

2 Likes
5

Oh, that’s great! Thanks for the extra detail. I wasn’t aware of the liveSocket.js().patch api. I am also using the internal pushHistoryPatch in an event handler to handle these complex query params, so I just wanted something that wouldn’t break between Phoenix releases. If there is an existing solution, I’ll go with that for now.

It would still be nice to avoid adding the custom JS for this kind of common case though, and I think there’s value in allowing document-relative patches with JS.patch in HEEx templates, so I can try to put together a prototype for that functionality in the next couple weeks if people are interested. No need to implement the complex query param filtering since the browser can handle that in these special cases. :slight_smile:

6

Something builtin would be nice but your patch_params/3 approach with the event listener looks like a good option in a meantime.

In fact, looking at your suggestion I don’t currently see any reason it’s not a good solution to my original question other than requiring a bit of custom code.

I was working on an interactive calendar component when I asked the initial question but I’ve been on more devops and backend tasks since then, it could be that there were specific issues I ran into with live components or more complex modular components but I’ll have get back into more UI work to see if I run into any of those again.

2 Likes
7

I mean - a prototype kind of exists with the code I shared - the main thing is coming up with a nice API that makes sense next to the existing JS.navigate and JS.patch functions. If you want to have a go at that, I’m happy to comment on whatever you propose!

3 Likes
8

Yeah, sorry, that’s what I meant by prototype. Something that could be reviewed for api compatibility and all that. I’ll give it a shot. Thanks!

3 Likes
9

Upvote this effort. Wanted just this today.

4 Likes
10

I’ll probably start on this next week. Will post a link to the draft PR once I have something worked out so people can check out the api. :slight_smile:

11

@steffend Here’s what I’m thinking for the API at this point:

JS.patch(query: 
  #localhost/index.html
  set: "?foo=bar",
  #localhost/index.html?foo=bar
  set: %{foo: "bar"},
  #localhost/index.html?foo=bar
  set: foo: "bar",
  #localhost/index.html?foo=bar
  merge: %{foo: "baz", lorem: "impsum"},
  #localhost/index.html?foo=baz&lorem=ipsum
  merge: lorem: "impsum", dolor: "sit",
  #localhost/index.html?foo=baz&lorem=ipsum&dolor=sit
  add: foo: "bar",
  #localhost/index.html?foo=baz&lorem=ipsum&dolor=sit&foo=bar
  add: %{foo: "baz"},
  #localhost/index.html?foo=baz&lorem=ipsum&dolor=sit&foo=bar&foo=baz
  add: lorem: ["amet", "quid", "novi"],
  #localhost/index.html?foo=baz&lorem=ipsum&dolor=sit&foo=bar&foo=baz&lorem=amet&lorem=quid&lorem=novi
  remove: foo: "baz",
  #localhost/index.html?lorem=ipsum&dolor=sit&foo=bar&lorem=amet&lorem=quid&lorem=novi
  remove: %{lorem: ["quid", "novi"]},
  #localhost/index.html?lorem=ipsum&dolor=sit&foo=bar&lorem=amet
  remove: :foo, :lorem
  #localhost/index.html?dolor=sit
)
  • Atoms indicate keys and strings indicate values in the resulting query string.
  • The operations are applied cumulatively in the order the keywords are provided to query:, so the behavior of any arbitrary combination of operations is well-defined.
  • set replaces the whole query relative to the current document.
  • add will append the key-value pairs into the query string without checking for existing keys. This can create duplicates, which is needed for simple array-valued params.
  • merge is add but it replaces existing keys instead of appending duplicates.
  • remove removes keys. If the argument is an atom, it removes all occurrences of the key, if the argument is a keyword or map, it only removes keys with matching values.
  • Using an array of strings or array of atoms anywhere that a scalar value is accepted has the semantics of operating on either all of the matching keys (if atoms) or all of the matching values (if strings).

Does this look good from a whiteboard design perspective?

I’m going to open an issue on the GH repo to track the work on this once I get started on it, but I wanted to make sure I wasn’t completely off track from the start before I start digging into the code next week. Thanks!

12

Thinking through the proposed API, I’m remembering Navigation Guards in Vue Router: Navigation Guards | Vue Router

That API generalizes better, as it gives us both the “before” and “after” URLs, and allows cancelling navigation or changing the target in any way we want.

The query: […] API looks very thorough for what it does, and made me feel it might be a lot of surface area to explain in docs. I wonder if we really need to solve param merging/updating in LiveView? It could be part of a library related to URL handling.

If we borrow from the Vue Router, the interface on LV’s side could be having an optional function as argument to patch and navigate that takes the URI before/after navigation and let’s you update it.

13

IIUC from previous comments, the %JS{} struct on the LV side can’t perform the actual updates in elixir code, it has to be completely declarative and then pass that information into the JS client-side code to do the actual query patching. I haven’t dug into the code yet, so I’m not sure what the exact limitations are, but that’s why I went with this instead of my original thought of just passing a function to manipulate the query params. :slight_smile:

14

Not sure if this will help, but my personal experience is that the only safe way to deal with “stateful” browser URLs is to always construct them anew and have their mutation and interpretation logic fully encapsulated within dedicated modules (or classes in JS). Adding params to or removing them from the current url is error prone for all but the simplest of cases.

For this purpose I have devised a “switch” pattern where for each different page I have a module (in Elixir) constructing its own structure (from_uri/2) based on the url (and the optional previous structure state), accompanied with intuitive state mutation functions encapsulating any and all state changes and then having a function to_uri/1 and to_string/1 for when I need to patch it.

A template example:

<div
  phx-click={js_patch( TodoListSwitch.new_item( @uri_switch))}
>
   ...
</div>

where js_patch/1 is a wrapper, and the String.Chars is implemented for TodoListSwitch:

  def js_patch( String.Chars.t()) do
    JS.patch( to_string( term))
  end

This way the logic of url construction and interpretation is fully encapsulated in a one module per web app page/context. Given that in our app we also have the back and return_to navigation, it’s a bit more complex than this, but my point here being that whatever you do, whether in Elixir or in JS, you need to manage a stateful url state responsibly, or otherwise it’s very likely going to be “leaking” the params over time (and therefore having bugs).

1 Like
15

Oh yeah, for a moment my mind ignored the Elixir/JS boundaries, sorry!

But then, the atom/string distinction would require some custom serialization of the declared JS command. I.e. in addition to the Elixir native API we need a JSON-serializable format to encode what the command is going to do, and then JS code implementing the additional functionality.

I had to go back a bit to remember more of the thread context. Considering what @steffend said, the missing is purely JavaScript code, right? The browser native `URLSearchParams` is probably the way to go as in the example. It’s flexible and probably anything in LiveView would just be a more rigid version of that.

Being able to “guard” and change in-flight navigations on the client would be a better goal IMHO (though I haven’t needed this in my personal LV journey). AFAIR today we can only listen to navigation events but can’t affect the navigation itself.

16

Right, there isn’t really a need for this feature per se. It’s more of a nice to have. It would be really nice to be able to patch query params relative to the document from function components in heex files instead of needing to add the custom JS. It’s not really a big deal for my current project since I already have that custom JS in place, but it seems like some other people want this feature, and I want to learn more about Phoenix, so I’m gonna give it a shot anyway. :slight_smile:

Thinking about it a bit more though, I do think this kind of thing could be helpful for library authors that want to provide reusable function components. I could see something like this opening up some possibilities for component libraries like what the different FE framework ecosystems have. Common things like sort and filter components that expose a documented set of query params that drive their logic could be published without needing the user to download/install any extra JS files. The backend part could even be included in a lib via LiveView lifecycle hooks for handle_params and on_mount, so that the user wouldn’t need to write any code at all to get the functionality. I might try to create a component like this as a sniff test for the usability while working on this feature. :slight_smile:

But yeah, the browser already provides all of the APIs for these things, and libraries are always an option for filling the gaps, but for me personally, being able to just add a param and trigger the handle_params callback with a phx-click attribute on a button in my heex template would be a major convenience boost since I prefer holding my page’s state in the URL. I’m not super strongly attached to this initial API design, but I do want this feature in some form or another.

1 Like
17

Oh yeah, go for it! I didn’t mean to stop the exploration. Life is all about learning :purple_heart:

:+1:

I have the same requirement for some LiveViews I implement, and I do the following today. Assuming you fully control the URL params on the server, here’s a stripped down version showing how to keep socket assigns and URL params in sync:

defmodule MyAppWeb.ExampleLive do
  use MyAppWeb, :live_view

  @impl true
  def render(assigns) do
    ~H"""
    <Layouts.app>
      <.form for={@form} id="settings-form" phx-change="update" phx-submit="save">
        <!-- ... -->
      </.form>
    </Layouts.app>
    """
  end

  @impl true
  def mount(_params, _session, socket) do
    socket =
      socket
      |> assign(
        settings: MyApp.user_settings(...)
      )

    {:ok, socket}
  end

  @impl true
  def handle_params(params, _uri, socket) when params == %{} do
    initial_params = settings_to_params(socket.assigns.settings)

    socket =
      socket
      |> push_patch(to: ~p"/example?#{initial_params}", replace: true)

    {:noreply, socket}
  end

  def handle_params(params, _uri, socket) do
    changeset = MyApp.change_settings(socket.assigns.settings, params)

    socket =
      case Ecto.Changeset.apply_action(changeset, :validate) do
        {:ok, settings} ->
          socket
          |> assign(
            settings: settings,
            form: to_form(changeset)
          )

        {:error, _changeset} ->
          # Navigation to an invalid state, revert back to default settings.
          settings = MyApp.default_settings()
          default_params = settings_to_params(settings)

          socket
          |> push_patch(to: ~p"/example?#{default_params}")
      end

    {:noreply, socket}
  end

  defp settings_to_params(%Settings{} = settings) do
    # filter/validate/sort settings into params keyword list
  end

  @impl true
  def handle_event("update", %{"settings" => settings}, socket) do
    # Update params without validation, deferred to `handle_params/3`.
    params =
      socket.assigns.settings
      |> MyApp.change_settings(settings)
      |> Ecto.Changeset.apply_changes()
      |> settings_to_params()

    socket =
      socket
      |> push_patch(to: ~p"/example?#{params}")

    {:noreply, socket}
  end

  def handle_event("save", %{"settings" => settings}, socket) do
    # ... persist settings ...
    {:noreply, socket}
  end
end

You can add a button:

<button type="button" phx-click="click" phx-value={@click_val} />

And make the magic happen:

  def handle_event("click", value, socket) do
    params = compute_new_params(value)

    socket =
      socket
      |> push_patch(to: ~p"/example?#{params}")

    {:noreply, socket}
  end
1 Like
18

I am gradually getting more familiar with the JS commands implementation. Given the need to serialize all commands within the HTML, most often in a tag attribute, I am forming the opinion that they shouldn’t carry too much logic/text in the serialized form.

For instance, if any part of the JS.patch(query: …) command would depend on an assign, the full long string, including all other JS commands piped together, would need to be sent to the client (and it can happen even if nothing changes, see Add JS.encode/1 function by rhcarvalho · Pull Request #4046 · phoenixframework/phoenix_live_view · GitHub). In fact, Steffen already hinted at this earlier in the thread.

I think a key decision point a proposal needs to address is where the new params are to be computed, client or server.

If on the client, URLSearchParams is our tool to modify params and liveSocket.js().patch/navigate to trigger navigation. Refer to Steffen’s example earlier.

If on the server, the most common solution I’ve seen and used is tracking (often validated/filtered) params in assigns, and LiveView.push_navigate/2 or LiveView.push_patch/2 to navigate. See my earlier example.

The case for computing params on the client is that it is good for integration with the broader JS ecosystem and preserving params the current LiveView may not care about, but other (JS) code does, such as analytics tokens, client-side library state (smart tables, charts, etc).

But there is a case for updating the URL/params directly from the server (push_navigate/2 and push_patch/2), and potentially a hybrid usage.

1 Like
19

I agree with Steffen in questioning the need for parity and reusing phx-value-* attributes. Both JS.patch and JS.navigate commands are related to navigation, while JS.push is about sending events from client to server.

Behind the scenes, JS.patch and JS.navigate also send internal events to the server. So a more generic/configurable library component should not have to decide about push/patch/navigate, but call JS.push always, and let the server-side code decide to trigger a patch or navigate using LiveView.push_navigate/2 or LiveView.push_patch/2, potentially informed by a key within the value payload.

The example I posted earlier shows how handle_event/3 can trigger a patch or navigate based on the event payload.

1 Like
20

Thanks for the insight! I haven’t started digging into it yet, but my thinking was that the query: options for JS.patch would just get serialized to a JSON object that the client-side JS code would use to update the query and push the patch. This ofc depends on how the JS.patch logic is actually implemented though. :slight_smile:

I think the client is the right place for query param patching tbh. Also, your point about serializing the atoms is something I was wondering about too. If the JS.patch logic is basically just serializing some JSON to send into a liveSocket.js().patch() call in the client code, then being able to translate the structure of the query: keyword’s value directly into a JSON object that the FE can use to manipulate the URLSearchParams would probably be ideal.