Uploading binary/multipart file with Tesla to AWS S3

Phoenix Forum Questions / Help
, ,
1

Hi all,

I’m trying to migrate from HTTPotion to Tesla, as the former is deprecated now. I’m having some troubles migrating some code that uploads a file to AWS S3.

This is the old code, which currently works (I’m using AWSAuth to generate the url):

def upload_file_to_bucket(filename, binary_file) do
    signed_aws_resource_request_url = signed_aws_resource_request_url("PUT", filename)

    case HTTPotion.put(signed_aws_resource_request_url, body: binary_file, timeout: 120000) do
      %HTTPotion.ErrorResponse{message: message} ->
        {:error, "Request error: #{message}"}

      %HTTPotion.Response{body: body, headers: _headers, status_code: 200} ->
        {:ok, body}
    end
end

And I tried like this:

binary_file param is binary_file = File.read!(file.path)

  def upload_file_to_bucket(filename, binary_file) do
    signed_aws_resource_request_url = signed_aws_resource_request_url("PUT", filename)

    file = Tesla.Multipart.new()
      |> Tesla.Multipart.add_file_content(binary_file, filename)
      # I tried with the next line both commented and uncommented
      # |> Multipart.add_file(filename, filename: filename)

    response = Tesla.put(signed_aws_resource_request_url, file, headers: [{"content-type", "application/pdf"}])
    case response do
      {:ok, %Tesla.Env{body: body, headers: _headers, method: :put, opts: _opts, query: _params, status: 200, url: _url}} ->
        {:ok, body}

      {:ok, %Tesla.Env{body: body, headers: _headers, method: :put, opts: _opts, query: _params, status: _status, url: _url}} ->
        message = "Error uploading file #{filename}: #{Kernel.inspect(body, limit: :infinity)}"
        Logger.error(message)

        {:error, message}

      {:error, error} ->
        message = "Error uploading file #{filename}: #{error}"
        Logger.error(message)

        {:error, message}
    end
  end

But I keep getting this error:

<Error><Code>NotImplemented</Code><Message>A header you provided implies functionality that is not implemented</
Message><Header>Transfer-Encoding</Header>

Which from what I’ve investigated means that the request is sending an empty filename or empty body.

Has anyone done anything close using Tesla for uploading files? I keep looking at the Tesla documentation for multipart but it’s not working.

Thanks

2 Likes
2

For anyone who’s curious about this.

transfer-encoding: chunked is the is automatically added as a header when an adapter (such as Hackney) doesn’t know the size of the upload, aka content-length is not set.

In this case I think the multipart upload isn’t setting the length.

Instead of a multipart upload I’d suggest putting the filename in the signed url and then adding the content-type as a header.

If you need to upload a file larger than 2GB there’s a special way to do that – don’t chunk it.

For the above code the fix is to:

  • include the name in the signed upload URL, not a form post
  • put the entire file in the body
  • set the content-type as a header, probably using the MIME hex package to determine the mime type from the file extension.
1 Like