Uploading multiple large files throws timeout error

Phoenix Forum Questions / Help
1

I’m uploading multiple photos to AWS using a HTML form. Those files sizes are range from 5mb to 100mb. Each one of the photo is processed using the ID for image detection. Some of the photos are fine. However when the file is larger, it timeout during the image processing.

Let’s say I’m uploading 5 photos at a same time. For instance some photos are successfully processed but others throws the error.

Code:

  def process_image(id) do
    base_url = "https://<<image-processing-api>>/img_process"
    headers = %{Accepts: "application/json", Authorization: "Key #{api_key()}"}

    img_url = "https://#{domain()}/media/#{id}"
    body = Poison.encode!(%{inputs: [%{data: %{image: %{url: img_url}}}]})

    {:ok, response} = HTTPoison.post(base_url, body, headers, [recv_timeout: 120000, timeout: 120000])

    case response.status_code do
      200 ->
        {:ok, response.body}
      _ ->
        {:error, response.status_code}
    end
  end

Error:

Request: POST /api/media/upload
** (exit) an exception was raised:
    ** (MatchError) no match of right hand side value: {:error, %HTTPoison.Error{reason: :timeout, id: nil}}

Is it something to do with ssl for HTTPoison options?

2

So it is still timing out after 2 minutes? I suppose you can simply increase the timeout further?

3

Now I set it to recv_timeout: 150000 and I don’t get the error. However another error raised and I think the uploaded url isn’t available yet to do the image processing and which returns the following error.

Error

(exit) an exception was raised:
    response returns {:error, 400}

This only happens for larger files. I have no issues with file size less than 10mb.

endpoint.ex

plug(
    Plug.Parsers,
    parsers: [:urlencoded, {:multipart, 200_000_000}, :json],
    pass: ["*/*"],
    json_decoder: Jason,
    length: 800_000_000
  )
4

400 means “bad request” which could mean the file is too big, yeah. Better check their docs.

5

Increasing the timeout fixes the issue.

Also used try…rescue block for the response

# modified the case block from above code
     case response do
       {:ok, result} ->
     	  try do
	         result
          rescue
                _ ->
                   []
          end

       _ ->
         []
      end