:httpc
HTTP client is part of Erlang standard library, and as such can be easily used in Elixir code too. One particular advantage of using :httpc
is that you can do a lot with it even with a minimal number of external dependencies. I prefer to use :httpc
in one-off scripts and in production.
The :httpc
reference is available at the Erlang standard library documentation website, here:
For now, I just wanted to publish an initial version of compilation of my own notes. Later I intent to update this post by adding notes to existing examples, adding references and even more examples. Feel free to modify the contents of the post too!
These examples are self-contained, can be copied to a script.exs
file and executed via elixir script.exs
command. Enjoy!
Index:
- sending a GET request returning a JSON payload
- sending a POST request with JSON request payload
- sending a POST request with no payload
- downloading a file
- uploading a file
- low-level tracing of HTTP interaction
General notes
-
for
:httpc
to work,:inets
app needs to be started beforehand; in case you intend to interact with HTTPS endpoints (which is what you want to do in most cases when making requests on public internet),:ssl
app needs to be started as well; all examples bellow start both apps, -
all examples below pass a big-looking “parameter” called
http_request_opts
- this is how you make sure:httpc
performs validation of the server’s TLS certificate; more on why this is necessary is described in “Erlang standard library: inets” page of “Secure Coding and Deployment Hardening Guidelines” by ERLef; without params underssl
key, you are likely to see the following warning:[warning] Description: 'Authenticity is not established by certificate path validation' Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'
-
:httpc
appears to like char lists a lot; a common mistake I made in the beginning when using:httpc
is feeding it with"https://example.com"
(a string) URL, while it should be a'https://example.com'
URL
Sending a GET request returning a JSON payload
Example
:inets.start()
:ssl.start()
url = 'https://httpbin.org/get'
headers = [{'accept', 'application/json'}]
http_request_opts = [
ssl: [
verify: :verify_peer,
cacerts: :public_key.cacerts_get(),
customize_hostname_check: [
match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
]
]
]
:httpc.request(:get, {url, headers}, http_request_opts, [])
Sending a POST request with JSON request payload
Example
Mix.install([:jason])
:inets.start()
:ssl.start()
url = 'https://httpbin.org/post'
headers = []
content_type = 'application/json'
body = Jason.encode!(%{hello: "world"})
http_request_opts = [
ssl: [
verify: :verify_peer,
cacerts: :public_key.cacerts_get(),
customize_hostname_check: [
match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
]
]
]
:httpc.request(:post, {url, headers, content_type, body}, http_request_opts, [])
Sending a POST request with no payload
Example
:inets.start()
:ssl.start()
url = 'https://httpbin.org/post'
headers = []
content_type = ''
body = ''
http_request_opts = [
ssl: [
verify: :verify_peer,
cacerts: :public_key.cacerts_get(),
customize_hostname_check: [
match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
]
]
]
:httpc.request(:post, {url, headers, content_type, body}, http_request_opts, [])
Downloading a file
Example
:inets.start()
:ssl.start()
url = 'https://www.rfc-editor.org/rfc/pdfrfc/rfc1149.txt.pdf'
headers = []
path_to_file =
System.tmp_dir!()
|> Path.join("rfc1149.txt.pdf")
|> String.to_charlist()
http_request_opts = [
ssl: [
verify: :verify_peer,
cacerts: :public_key.cacerts_get(),
customize_hostname_check: [
match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
]
]
]
{:ok, :saved_to_file} =
:httpc.request(:get, {url, headers}, http_request_opts, [stream: path_to_file])
Uploading a file
Example
Mix.install([:multipart])
:inets.start()
:ssl.start()
part = Multipart.Part.file_field("/tmp/rfc1149.txt.pdf", "my_file")
multipart =
Multipart.new()
|> Multipart.add_part(part)
content_length =
multipart
|> Multipart.content_length()
|> Integer.to_string()
|> String.to_charlist()
content_type =
multipart
|> Multipart.content_type("multipart/form-data")
|> String.to_charlist()
url = 'https://httpbin.org/anything'
headers = [{'Content-Length', content_length}]
payload = Multipart.body_binary(multipart)
http_request_opts = [
ssl: [
verify: :verify_peer,
cacerts: :public_key.cacerts_get(),
customize_hostname_check: [
match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
]
]
]
:httpc.request(:post, {url, headers, content_type, payload}, http_request_opts, [])
Notes
-
some HTTP clients offer convenient API around sending file payloads; UX varies on the library API design, and sometime can serve as a source of confusion (see this discussion, for instance),
-
because design of
:httpc
appears to make little-to-no assumptions regarding the semantics of request payload, one can fairly say that:httpc
simply send bytes. As such, we should be able to “manually” construct any payload, including one to upload a file, and:httpc
will just handle it, -
multipart
package, originally mentioned by its author @engineeringdept here focuses on just putting together a bunch of bytes that look like a correct “HTTP payload containing a file upload”,
Low-level tracing of HTTP interaction
Example
:inets.start()
:ssl.start()
url = 'https://httpbin.org/get'
headers = []
http_request_opts = [
ssl: [
verify: :verify_peer,
cacerts: :public_key.cacerts_get(),
customize_hostname_check: [
match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
]
]
]
:httpc.set_options([verbose: :debug])
:httpc.request(:get, {url, headers}, http_request_opts, [])
Notes
-
:debug
option value will easily generate several megabytes of information, while:trace
value can potentially generate several tens of megabytes, so use it with care.