:httpc cheatsheet

:httpc HTTP client is part of Erlang standard library, and as such can be easily used in Elixir code too. One particular advantage of using :httpc is that you can do a lot with it even with a minimal number of external dependencies. I prefer to use :httpc in one-off scripts and in production.

The :httpc reference is available at the Erlang standard library documentation website, here:

httpc — OTP 29.0.2 (inets 9.7.1)

For now, I just wanted to publish an initial version of compilation of my own notes. Later I intent to update this post by adding notes to existing examples, adding references and even more examples. Feel free to modify the contents of the post too!

These examples are self-contained, can be copied to a script.exs file and executed via elixir script.exs command. Enjoy!

Index:

• sending a GET request returning a JSON payload
• sending a POST request with JSON request payload
• sending a POST request with no payload
• downloading a file
• uploading a file
• low-level tracing of HTTP interaction

General notes

• for :httpc to work, :inets app needs to be started beforehand; in case you intend to interact with HTTPS endpoints (which is what you want to do in most cases when making requests on public internet), :ssl app needs to be started as well; all examples bellow start both apps,

• all examples below pass a big-looking “parameter” called http_request_opts - this is how you make sure :httpc performs validation of the server’s TLS certificate; more on why this is necessary is described in “Erlang standard library: inets” page of “Secure Coding and Deployment Hardening Guidelines” by ERLef; without params under ssl key, you are likely to see the following warning on OTP < 26:

  [warning] Description: 'Authenticity is not established by certificate path validation'
       Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'
  

• :httpc appears to like char lists a lot; a common mistake I made in the beginning when using :httpc is feeding it with "https://example.com" a string, while it should be a a list of characters, e.g. ~c"https://example.com"

Sending a GET request returning a JSON payload

Example

:inets.start()
:ssl.start()

url = ~s"https://httpbin.org/get"
headers = [{~s"accept", ~s"application/json"}]

http_request_opts = [
  ssl: [
    cacerts: :public_key.cacerts_get()
  ]
]

:httpc.request(:get, {url, headers}, http_request_opts, [])

Sending a POST request with JSON request payload

Example

:inets.start()
:ssl.start()

url = ~c"https://httpbin.org/post"
headers = []
content_type = ~c"application/json"
body = :json.encode(%{hello: "world"})

http_request_opts = [
  ssl: [
    cacerts: :public_key.cacerts_get()
  ]
]

:httpc.request(:post, {url, headers, content_type, body}, http_request_opts, [])

Sending a POST request with no payload

Example

:inets.start()
:ssl.start()

url = ~s"https://httpbin.org/post"
headers = []
content_type = ~c""
body = ~c""

http_request_opts = [
  ssl: [
    cacerts: :public_key.cacerts_get()
  ]
]

:httpc.request(:post, {url, headers, content_type, body}, http_request_opts, [])

Downloading a file

Example

:inets.start()
:ssl.start()

url = ~c"https://www.rfc-editor.org/rfc/pdfrfc/rfc1149.txt.pdf"
headers = []

path_to_file =
  System.tmp_dir!()
  |> Path.join("rfc1149.txt.pdf")
  |> String.to_charlist()

http_request_opts = [
  ssl: [
    cacerts: :public_key.cacerts_get(),
    customize_hostname_check: [
      match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
    ]
  ]
]

{:ok, :saved_to_file} =
  :httpc.request(:get, {url, headers}, http_request_opts, [stream: path_to_file])

Uploading a file

Example

Mix.install([:multipart])

:inets.start()
:ssl.start()

part = Multipart.Part.file_field("/tmp/rfc1149.txt.pdf", "my_file")

multipart =
  Multipart.new()
  |> Multipart.add_part(part)

content_length =
  multipart
  |> Multipart.content_length()
  |> Integer.to_string()
  |> String.to_charlist()

content_type =
  multipart
  |> Multipart.content_type("multipart/form-data")
  |> String.to_charlist()

url = ~c"https://httpbin.org/anything"
headers = [{~c"Content-Length", content_length}]
payload = Multipart.body_binary(multipart)

http_request_opts = [
  ssl: [
    cacerts: :public_key.cacerts_get()
  ]
]

:httpc.request(:post, {url, headers, content_type, payload}, http_request_opts, [])

Notes

• some HTTP clients offer convenient API around sending file payloads; UX varies on the library API design, and sometime can serve as a source of confusion (see this discussion, for instance),

• because design of :httpc appears to make little-to-no assumptions regarding the semantics of request payload, one can fairly say that :httpc simply send bytes. As such, we should be able to “manually” construct any payload, including one to upload a file, and :httpc will just handle it,

• multipart package, originally mentioned by its author @engineeringdept here focuses on just putting together a bunch of bytes that look like a correct “HTTP payload containing a file upload”,

Low-level tracing of HTTP interaction

Example

:inets.start()
:ssl.start()

url = ~c"https://httpbin.org/get"
headers = []

http_request_opts = [
  ssl: [
    cacerts: :public_key.cacerts_get()
  ]
]

:httpc.set_options([verbose: :debug])
:httpc.request(:get, {url, headers}, http_request_opts, [])

Notes

• :debug option value will easily generate several megabytes of information, while :trace value can potentially generate several tens of megabytes, so use it with care.