gmile

gmile

:httpc cheatsheet

:httpc HTTP client is part of Erlang standard library, and as such can be easily used in Elixir code too. One particular advantage of using :httpc is that you can do a lot with it even with a minimal number of external dependencies. I prefer to use :httpc in one-off scripts and in production.

The :httpc reference is available at the Erlang standard library documentation website, here:

httpc — OTP 29.0.2 (inets 9.7.1)

For now, I just wanted to publish an initial version of compilation of my own notes. Later I intent to update this post by adding notes to existing examples, adding references and even more examples. Feel free to modify the contents of the post too!

These examples are self-contained, can be copied to a script.exs file and executed via elixir script.exs command. Enjoy! :slight_smile:

Index:

  • sending a GET request returning a JSON payload
  • sending a POST request with JSON request payload
  • sending a POST request with no payload
  • downloading a file
  • uploading a file
  • low-level tracing of HTTP interaction

General notes

  • for :httpc to work, :inets app needs to be started beforehand; in case you intend to interact with HTTPS endpoints (which is what you want to do in most cases when making requests on public internet), :ssl app needs to be started as well; all examples bellow start both apps,

  • all examples below pass a big-looking “parameter” called http_request_opts - this is how you make sure :httpc performs validation of the server’s TLS certificate; more on why this is necessary is described in “Erlang standard library: inets” page of “Secure Coding and Deployment Hardening Guidelines” by ERLef; without params under ssl key, you are likely to see the following warning on OTP < 26:

    [warning] Description: 'Authenticity is not established by certificate path validation'
         Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'
    
  • :httpc appears to like char lists a lot; a common mistake I made in the beginning when using :httpc is feeding it with "https://example.com" a string, while it should be a a list of characters, e.g. ~c"https://example.com"

Sending a GET request returning a JSON payload

Example

:inets.start()
:ssl.start()

url = ~s"https://httpbin.org/get"
headers = [{~s"accept", ~s"application/json"}]

http_request_opts = [
  ssl: [
    cacerts: :public_key.cacerts_get()
  ]
]

:httpc.request(:get, {url, headers}, http_request_opts, [])

Sending a POST request with JSON request payload

Example

:inets.start()
:ssl.start()

url = ~c"https://httpbin.org/post"
headers = []
content_type = ~c"application/json"
body = :json.encode(%{hello: "world"})

http_request_opts = [
  ssl: [
    cacerts: :public_key.cacerts_get()
  ]
]

:httpc.request(:post, {url, headers, content_type, body}, http_request_opts, [])

Sending a POST request with no payload

Example

:inets.start()
:ssl.start()

url = ~s"https://httpbin.org/post"
headers = []
content_type = ~c""
body = ~c""

http_request_opts = [
  ssl: [
    cacerts: :public_key.cacerts_get()
  ]
]

:httpc.request(:post, {url, headers, content_type, body}, http_request_opts, [])

Downloading a file

Example

:inets.start()
:ssl.start()

url = ~c"https://www.rfc-editor.org/rfc/pdfrfc/rfc1149.txt.pdf"
headers = []

path_to_file =
  System.tmp_dir!()
  |> Path.join("rfc1149.txt.pdf")
  |> String.to_charlist()

http_request_opts = [
  ssl: [
    cacerts: :public_key.cacerts_get(),
    customize_hostname_check: [
      match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
    ]
  ]
]

{:ok, :saved_to_file} =
  :httpc.request(:get, {url, headers}, http_request_opts, [stream: path_to_file])

Uploading a file

Example

Mix.install([:multipart])

:inets.start()
:ssl.start()

part = Multipart.Part.file_field("/tmp/rfc1149.txt.pdf", "my_file")

multipart =
  Multipart.new()
  |> Multipart.add_part(part)

content_length =
  multipart
  |> Multipart.content_length()
  |> Integer.to_string()
  |> String.to_charlist()

content_type =
  multipart
  |> Multipart.content_type("multipart/form-data")
  |> String.to_charlist()

url = ~c"https://httpbin.org/anything"
headers = [{~c"Content-Length", content_length}]
payload = Multipart.body_binary(multipart)

http_request_opts = [
  ssl: [
    cacerts: :public_key.cacerts_get()
  ]
]

:httpc.request(:post, {url, headers, content_type, payload}, http_request_opts, [])

Notes

  • some HTTP clients offer convenient API around sending file payloads; UX varies on the library API design, and sometime can serve as a source of confusion (see this discussion, for instance),

  • because design of :httpc appears to make little-to-no assumptions regarding the semantics of request payload, one can fairly say that :httpc simply send bytes. As such, we should be able to “manually” construct any payload, including one to upload a file, and :httpc will just handle it,

  • multipart package, originally mentioned by its author @engineeringdept here focuses on just putting together a bunch of bytes that look like a correct “HTTP payload containing a file upload”,

Low-level tracing of HTTP interaction

Example

:inets.start()
:ssl.start()

url = ~c"https://httpbin.org/get"
headers = []

http_request_opts = [
  ssl: [
    cacerts: :public_key.cacerts_get()
  ]
]

:httpc.set_options([verbose: :debug])
:httpc.request(:get, {url, headers}, http_request_opts, [])

Notes

  • :debug option value will easily generate several megabytes of information, while :trace value can potentially generate several tens of megabytes, so use it with care.

Most Liked

paulo-f-oliveira

paulo-f-oliveira

There is actually no working solution when you search online

There is a solution online, though it might not be immediately found if you’re searching for Elixir, and not Erlang: Erlang standard library: ssl | EEF Security WG. In the page body you’ll find

%% Erlang (OTP 25 or later)
ssl:connect("example.net", 443, [
    {verify, verify_peer},
    {cacerts, public_key:cacerts_get()},
    {depth, 3},
    {customize_hostname_check, [
        {match_fun, public_key:pkix_verify_hostname_match_fun(https)}
    ]}
]).

The Secure Coding and Deployment Hardening Guidelines, by the Erlang Ecosystem Foundation’s Security Working Group, is a great read, btw.

voltone

voltone

Nit:

There is no need to set the “Content-Type” header in a request that does not include a body. If you want to indicate to the server that you are ready to accept a JSON response body, us the “Accept” header instead:

headers = [{'accept', 'application/json'}]
darnahsan

darnahsan

Just appreciation response. I just started exploring escript yesterday and was trying out elixir clients and they kept failing on castore and then moved to httpc. Though it worked but it kept showing following warning

[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

There is actually no working solution when you search online but your post has the the solution I needed and now its works like a charm without any warning.

Thanks and gotta love the elixir/erlang/beam community

Where Next?

Popular in Wikis Top

OvermindDL1
It was stated at elixir-lang.org blocked in Russia · Issue #6172 · elixir-lang/elixir · GitHub that Mirrors of the primary Elixir website...
New
BartOtten
A wiki for Doom Emacs Doom is a configuration framework for GNU Emacs. It focuses on performance and customizability. It can be a founda...
New
yurko
Here are few pieces of (common) Linux knowledge that we use for reasonably small one server apps. We use Ubuntu but this should work for ...
New
gmile
:httpc HTTP client is part of Erlang standard library, and as such can be easily used in Elixir code too. One particular advantage of usi...
New
axelson
Preamble This Wiki is intended to be a community-maintained (see the Contribution Guidelines if interested) resource of common “gotchas” ...
New
georgeguimaraes
Hi people, since the new year is coming, I’d like to plan my travels for events in 2017. So, what events (Elixir or FP related) that you...
New
Eiji
At start some definitions: HTTPS (is a protocol for secure communication over a computer network which is widely used on the Internet) -...
New
nicbet
Introduction Now that the language is picking up support and maturing nicely, I’d like to start a collection of common and recurring Elix...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 47930 226
New
AstonJ
Wonder if we can compile a list of learning resources, blog posts, talks, threads etc to help those who are just learning about Contexts....
New

Other popular topics Top

marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a &gt; b) do {:ok, "a"} end if (a &lt; b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
dogweather
I wrote this comment on r/haskell, and it’s not popular there. :wink: But I think I’m on to something… Haskell reminds me of Java, and e...
New

We're in Beta

About us Mission Statement