I ended up solving it via an async react-select, the react component from Mantine that I was using before just didn’t work well with async data. So that removes the issue of having to load the payload in the DOM initially.

But yes initially it would add the entire payload to every select.

Here is the react-select implementation that I ended up using in case anyone is interested:

<.remote_select form={f} field={:player1_id} url={player_options_url(@conn)} options={Map.fetch!(@player_options, :player1)} placeholder={select_player_placeholder()} />
... etc ...

  def remote_select(assigns) do
    ~H"""
    <.form_field {form_field_assigns(assigns)} ><.remote_select_tag {remote_select_assigns(assigns)} /></.form_field>
    """
  end

  def remote_select_tag(assigns) do
    assigns =
      assigns
      |> assign(:error, error?(assigns.form, assigns.field))

    ~H"""
    <.react controller="remote-select" props={extra_attributes(assigns)}>
      <%= Phoenix.HTML.Form.hidden_input(@form, @field, data: [remote_select_target: "input"]) %>
      <div data-remote-select-target="component"></div>
    </.react>
    """
  end

react/1 is a simple functional component that adds a div with display: contents, adds a Stimulus data-controller attribute and serializes props (converting underscore to camelcase, and so on.)

The remote-select Stimulus controller:

import { Controller } from '@hotwired/stimulus';

export default class extends Controller {
    static values = { props: Object };
    static targets = ['input', 'component'];

    connect() {
      ReactDOM.render(
        <RemoteSelect {...this.propsValue} defaultValue={this.defaultValue} onChange={this.handleChange} />,
        this.componentTarget
      );
    }

    disconnect() {
      ReactDOM.unmountComponentAtNode(this.componentTarget);
    }

    handleChange = (option) => {
      // prettier-ignore
      this.inputTarget.value = option === null ? '' : [].concat(option).map(o => o.value).join(',');
    };

    get defaultValue() {
      return this.propsValue.options.find((option) => option.value == this.inputTarget.value);
    }
  };

Notice how it concatenates values via ‘,’ in a single hidden input (in case multiple options are selected).

And finally the react component:

import React from 'react';
import AsyncSelect from 'react-select/async';
import { withTheme, withDefaults } from './react_select';

function RemoteSelect({ url, options, ...restProps }) {
  const loadOptions = (query) => {
    const remoteUrl = new URL(url);
    remoteUrl.searchParams.set('query', query);
    return fetch(remoteUrl.toString()).then((response) => response.json());
  };

  return (
    <AsyncSelect
      {...restProps}
      defaultOptions={options}
      noOptionsMessage={({ inputValue }) => (inputValue ? `Geen resultaten voor "${inputValue}"` : 'Type om te zoeken')}
      loadOptions={loadOptions}
    />
  );
}

export default withTheme(withDefaults(RemoteSelect));

Not much special going on here either except that it uses the url property to perform the async request