Unable to implement s3 uploads using SimpleS3Upload

Questions / Help
, , ,
1

I am trying to implement the official guide for s3 External Uploads.

Here is what my liveview looks like:

  @impl true
  def mount(_params, _session, socket) do
    {:ok,
      socket
      |> assign(:uploaded_files, [])
      |> allow_upload(:presentation, accept: ~w(.pptx), max_entries: 3, external: &presign_upload/2)}
  end

  @impl true
  def render(assigns) do
    ~H"""
      <form id="upload-form" phx-submit="save" phx-change="validate">
        <.live_file_input upload={@uploads.presentation} />
        <button type="submit">Upload</button>
      </form>

      <section phx-drop-target={@uploads.presentation.ref}>
        <%= for entry <- @uploads.presentation.entries do %>
          <article class="upload-entry">

            <figure>
              <.live_img_preview entry={entry} />
              <figcaption><%= entry.client_name %></figcaption>
            </figure>

            <%!-- entry.progress will update automatically for in-flight entries --%>
            <progress value={entry.progress} max="100"> <%= entry.progress %>% </progress>

            <%!-- a regular click event whose handler will invoke Phoenix.LiveView.cancel_upload/3 --%>
            <button type="button" phx-click="cancel-upload" phx-value-ref={entry.ref} aria-label="cancel">&times;</button>

            <%!-- Phoenix.Component.upload_errors/2 returns a list of error atoms --%>
            <%= for err <- upload_errors(@uploads.presentation, entry) do %>
              <p class="alert alert-danger"><%= error_to_string(err) %></p>
            <% end %>

          </article>
        <% end %>

        <%!-- Phoenix.Component.upload_errors/1 returns a list of error atoms --%>
        <%= for err <- upload_errors(@uploads.presentation) do %>
          <p class="alert alert-danger"><%= error_to_string(err) %></p>
        <% end %>

      </section>
    """
  end

  @impl true
  def handle_event("validate", _params, socket) do
    {:noreply, socket}
  end

  @impl true
  def handle_event("cancel-upload", %{"ref" => ref}, socket) do
    {:noreply, cancel_upload(socket, :presentation, ref)}
  end

  @impl true
  def handle_event("save", _params, socket) do
    socket
    |> consume_uploaded_entries(:presentation, fn %{key: key}, _ ->
      {:ok, %{"key" => key, "bucket" => "bucket"}}
    end)

    {:noreply, socket}
  end

  defp presign_upload(entry, socket) do
    uploads = socket.assigns.uploads
    bucket = "uploads"
    key = "public/#{entry.client_name}"

    config = %{
      access_key_id: System.fetch_env!("AWS_ACCESS_KEY_ID"),
      secret_access_key: System.fetch_env!("AWS_SECRET_ACCESS_KEY")
    }

    {:ok, fields} =
      SimpleS3Upload.sign_form_upload(config, bucket,
        key: key,
        content_type: entry.client_type,
        max_file_size: uploads[entry.upload_config].max_file_size,
        expires_in: :timer.hours(1)
      )

    meta = %{uploader: "S3", key: key, url: "http://#{bucket}.s3.us-east-005.backblazeb2.com", fields: fields}
    {:ok, meta, socket}
  end

  defp error_to_string(:too_large), do: "Too large"
  defp error_to_string(:too_many_files), do: "You have selected too many files"
  defp error_to_string(:not_accepted), do: "You have selected an unacceptable file type"
  defp error_to_string(:external_client_failure), do: "Cannot connect to external client"
end

and this is my uploader in app.js

let Uploaders = {}

Uploaders.S3 = function(entries, onViewError){
  entries.forEach(entry => {
    let formData = new FormData()
    let {url, fields} = entry.meta
    Object.entries(fields).forEach(([key, val]) => formData.append(key, val))
    formData.append("file", entry.file)
    let xhr = new XMLHttpRequest()
    onViewError(() => xhr.abort())
    xhr.onload = () => xhr.status === 204 ? entry.progress(100) : entry.error()
    xhr.onerror = () => entry.error()
    xhr.upload.addEventListener("progress", (event) => {
      if(event.lengthComputable){
        let percent = Math.round((event.loaded / event.total) * 100)
        if(percent < 100){ entry.progress(percent) }
      }
    })

    xhr.open("POST", url, true)
    xhr.send(formData)
  })
}

let csrfToken = document.querySelector("meta[name='csrf-token']").getAttribute("content")

let liveSocket = new LiveSocket("/live", Socket, {
  uploaders: Uploaders,
  params: {_csrf_token: csrfToken}
})

On clicking the upload button (phx-submit) I get this kinda error:

[error] GenServer #PID<0.735.0> terminating
** (KeyError) key "phx-F5V25PnPSWKnPADF" not found in: %{"phx-F5V3Ckri4nYq9hUB" => :presentation}
    :erlang.map_get("phx-F5V25PnPSWKnPADF", %{"phx-F5V3Ckri4nYq9hUB" => :presentation})
    (phoenix_live_view 0.18.18) lib/phoenix_live_view/upload.ex:192: Phoenix.LiveView.Upload.get_upload_by_ref!/2
    (phoenix_live_view 0.18.18) lib/phoenix_live_view/channel.ex:1213: anonymous fn/3 in Phoenix.LiveView.Channel.maybe_update_uploads/2
    (stdlib 4.3) maps.erl:411: :maps.fold_1/3
    (phoenix_live_view 0.18.18) lib/phoenix_live_view/channel.ex:218: Phoenix.LiveView.Channel.handle_info/2
    (stdlib 4.3) gen_server.erl:1123: :gen_server.try_dispatch/4
    (stdlib 4.3) gen_server.erl:1200: :gen_server.handle_msg/6
    (stdlib 4.3) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Last message: %Phoenix.Socket.Message{topic: "lv:phx-F5V25OcQsvehxgmB", event: "event", payload: %{"event" => "validate", "type" => "form", "uploads" => %{"phx-F5V25PnPSWKnPADF" => [%{"last_modified" => 1697213961787, "name" => "Presentation.pptx", "path" => "presentation", "ref" => "0", "relative_path" => "", "size" => 44902, "type" => "application/vnd.openxmlformats-officedocument.presentationml.presentation"}]}, "value" => "_target=presentation"}, ref: "168", join_ref: "167"}
State: %{components: {%{}, %{}, 1}, join_ref: "167", serializer: Phoenix.Socket.V2.JSONSerializer, socket: #Phoenix.LiveView.Socket<id: "phx-F5V25OcQsvehxgmB", endpoint: ActiveSlidesWeb.Endpoint, view: ActiveSlidesWeb.UserAccountLive, parent_pid: nil, root_pid: #PID<0.735.0>, router: ActiveSlidesWeb.Router, assigns: %{__changed__: %{}, current_user: #ActiveSlides.Accounts.User<__meta__: #Ecto.Schema.Metadata<:loaded, "users">, id: 3, email: "apple@gmail.com", name: "Apple Pie", confirmed_at: nil, inserted_at: ~N[2023-10-22 16:02:13], updated_at: ~N[2023-10-22 16:02:13], ...>, flash: %{}, live_action: :index, uploaded_files: [], uploads: %{__phoenix_refs_to_names__: %{"phx-F5V3Ckri4nYq9hUB" => :presentation}, presentation: #Phoenix.LiveView.UploadConfig<name: :presentation, max_entries: 3, max_file_size: 8000000, entries: [], accept: ".pptx", ref: "phx-F5V3Ckri4nYq9hUB", errors: [], auto_upload?: false, progress_event: nil, ...>}}, transport_pid: #PID<0.643.0>, ...>, topic: "lv:phx-F5V25OcQsvehxgmB", upload_names: %{}, upload_pids: %{}}

I have been stuck on this for so long! :face_with_spiral_eyes: any help would be appreciated

2

You’re using backblaze B2 and it works a little differently than amazon S3, this thread should get you up to speed: Backblaze and Phoenix LiveView Uploads, at the end someone offered a working example, if you still can’t do it we can help you here.

1 Like
3

Thank you for mentioning this helpful article, but after updating my code according to the example, I am still getting the same error, and the error triggers on the validate function

4

Can you check if there’s an error in the browser console or network panel when you try to upload the file?
Externals uploads are entirely driven client side so errors may show up in the console and I also tried using your code (with the exception of the presign upload) to upload a .pptx to Cloudflare R2 (which also doesn’t support POST to upload) and it worked flawlessly, which suggests that the error is happening in the url or in the js part of your code (my guess is in the JS)…

Other thing that I noticed now is that you’re using live_img_preview with a pptx, I don’t think that this will work and also validate isn’t doing anything so I don’t think that’s the problem.

5

Could you please explain what you mean by exception of presign upload?

I’ve checked the console and here are the logs, it suggests that the frontend is forwarding the preflight request from its end but the issue is with phx session being closed. (The js code is also posted above for reference if you notice any issue there)

image
image1112Ă—1202 117 KB

P.S. yes the live_img_preview isn’t needed, removed this bit.

6

Sure, it is just that I used my own presign_upload/2 instead of the one available at the docs, the one at the docs generate a bunch of fields required for Aws S3 that are appended to a FormData interface and POSTed to the server, the issue is that since B2 (and Cloudflare R2) can’t use POST and need to use PUT and in this case the FormData isn’t what we need, what we need is a presigned URL.

And as I suspected, the issue is happening in the js, if you check the network tab during the request you should see an 50X error.

This isn’t merged yet but may help you.