Filtering Content-Length and status from HTTPoison AsyncHeaders

[teofilosalgado]

Hi all!
Recently I’ve been rewriting an old python downloader utility script in Elixir, but currently I am stuck in response header filtering.

I am starting my HTTPoison in a GenServer worker with:

    options = [
      stream_to: self(),
      timeout: :infinity,
      hackney: [:insecure, recv_timeout: :infinity]
    ]

    worker = HTTPoison.get(url, %{}, options)

then, I threat the response messages with a simple approach:

case message do
      %HTTPoison.AsyncHeaders{headers: headers} ->
        # Code I need goes here

      %HTTPoison.AsyncChunk{chunk: chunk} ->
        IO.binwrite(file, chunk)
        {:noreply, state}

      %HTTPoison.AsyncEnd{} ->
        File.close(file)
        {:stop, :normal, state}

      %HTTPoison.Error{reason: reason} ->
        {:stop, reason, state}

      _ ->
        {:noreply, state}
    end

My problem is that I need to strip from the headers the Content-Length and the status to handle errors like 404 or when my files exceed the specified maximum size. How could I accomplish that with HTTPoison? Am I doing something wrong?

Any help will be very appreciated!
Cheers.

[teofilosalgado]

Well, after some time trying different approaches I finally managed to make it work with:

      %HTTPoison.AsyncStatus{code: code} when code >= 400 ->
        # So, turns out HTTPoison have the very convenient type AsyncStatus!
        {:stop, code, state}

      %HTTPoison.AsyncChunk{chunk: chunk} ->
        IO.binwrite(file, chunk)
        {:noreply, state}

      %HTTPoison.AsyncHeaders{headers: headers} ->
        headers_map = Enum.into(headers, %{})
        # Now I am able to access all headers just like a regular map!
        IO.inspect(headers_map["Content-Length"], label: "Content length: ")
        {:noreply, state}

      %HTTPoison.AsyncEnd{} ->
        File.close(file)
        {:stop, :normal, state}

      %HTTPoison.Error{reason: reason} ->
        {:stop, reason, state}

      _ ->
        {:noreply, state}

[axelson]

Welcome to the forum! I’m glad you were able to get something working.

@teofilosalgado

      %HTTPoison.AsyncHeaders{headers: headers} ->
        headers_map = Enum.into(headers, %{})
        # Now I am able to access all headers just like a regular map!
        IO.inspect(headers_map["Content-Length"], label: "Content length: ")
        {:noreply, state}

What was the format of headers before the Enum.into call? If it was a Keyword List then you could use Keyword.get(headers, "Content-Length") to get the Content-Length header. A keyword list is a better format to work with for headers since headers can be repeated with different values and you can’t represent that with a map since a map cannot have duplicate keys.

[teofilosalgado]

Oh thanks!

It unfortunately isn’t an Keyword List . The keys aren’t atoms, they are regular tuple strings in a list like so: [{"Content-Length": "1549"}]. Is it possible to use Keyword List methods on such data structure?

[gregvaughn]

It’s a proplist. There’s an erlang module to manage them. For example:
:proplists.get_value("abc", [{"abc", 123}])

[teofilosalgado]

Your solution worked like a charm! Didn’t know about the existance of this structure. Will defintely read more about it in the erlang documentation. Thank you!