Uploading binary/multipart file with Tesla to AWS S3

tnaftali

Hi all,

I’m trying to migrate from HTTPotion to Tesla, as the former is deprecated now. I’m having some troubles migrating some code that uploads a file to AWS S3.

This is the old code, which currently works (I’m using AWSAuth to generate the url):

def upload_file_to_bucket(filename, binary_file) do
    signed_aws_resource_request_url = signed_aws_resource_request_url("PUT", filename)

    case HTTPotion.put(signed_aws_resource_request_url, body: binary_file, timeout: 120000) do
      %HTTPotion.ErrorResponse{message: message} ->
        {:error, "Request error: #{message}"}

      %HTTPotion.Response{body: body, headers: _headers, status_code: 200} ->
        {:ok, body}
    end
end

And I tried like this:

binary_file param is binary_file = File.read!(file.path)

  def upload_file_to_bucket(filename, binary_file) do
    signed_aws_resource_request_url = signed_aws_resource_request_url("PUT", filename)

    file = Tesla.Multipart.new()
      |> Tesla.Multipart.add_file_content(binary_file, filename)
      # I tried with the next line both commented and uncommented
      # |> Multipart.add_file(filename, filename: filename)

    response = Tesla.put(signed_aws_resource_request_url, file, headers: [{"content-type", "application/pdf"}])
    case response do
      {:ok, %Tesla.Env{body: body, headers: _headers, method: :put, opts: _opts, query: _params, status: 200, url: _url}} ->
        {:ok, body}

      {:ok, %Tesla.Env{body: body, headers: _headers, method: :put, opts: _opts, query: _params, status: _status, url: _url}} ->
        message = "Error uploading file #{filename}: #{Kernel.inspect(body, limit: :infinity)}"
        Logger.error(message)

        {:error, message}

      {:error, error} ->
        message = "Error uploading file #{filename}: #{error}"
        Logger.error(message)

        {:error, message}
    end
  end

But I keep getting this error:

<Error><Code>NotImplemented</Code><Message>A header you provided implies functionality that is not implemented</
Message><Header>Transfer-Encoding</Header>

Which from what I’ve investigated means that the request is sending an empty filename or empty body.

Has anyone done anything close using Tesla for uploading files? I keep looking at the Tesla documentation for multipart but it’s not working.

Thanks

1 comment

/phoenix #httpotion #tesla

2 1639 1

2024-05-09 21:44:13 UTC

Showing Posts 1 to 1

Show Best Posts
Show All (oldest first)
Show All (newest first)

idyll

For anyone who’s curious about this.

transfer-encoding: chunked is the is automatically added as a header when an adapter (such as Hackney) doesn’t know the size of the upload, aka content-length is not set.

In this case I think the multipart upload isn’t setting the length.

Instead of a multipart upload I’d suggest putting the filename in the signed url and then adding the content-type as a header.

If you need to upload a file larger than 2GB there’s a special way to do that – don’t chunk it.

For the above code the fix is to:

include the name in the signed upload URL, not a form post
put the entire file in the body
set the content-type as a header, probably using the MIME hex package to determine the mime type from the file extension.

Post #1

— All posts loaded —