Oh yeah. My bad! Still feels like something not quite right about it though. Seems that a lib focused on explicitly handling web urls should maybe default to http or https rather than raising an error for a missing scheme. ddsdsdf.com should be recognized as an attempt at web url, shouldn’t it? We’re all used to just skipping the http:// part of any address we type into a browser.
stevensonmt
0
wojtekmach
Hex Core Team
I haven’t seen any other HTTP client (besides good old curl) doing that. Please let me know if you have any such examples.
0
wojtekmach
Hex Core Team
Hey everyone, some Req updates since v0.4.0 below.
New step: checksum
Sets expected response body checksum.
iex> resp = Req.get!("https://httpbin.org/json", checksum: "sha1:9274ffd9cf273d4a008750f44540c4c5d4c8227c")
iex> resp.status
200
iex> Req.get!("https://httpbin.org/json", checksum: "sha1:bad")
** (Req.ChecksumMismatchError) checksum mismatch
expected: sha1:bad
actual: sha1:9274ffd9cf273d4a008750f44540c4c5d4c8227c
New step put_aws_sigv4
Signs request with AWS Signature Version 4.
iex> req =
...> Req.new(
...> base_url: "https://s3.amazonaws.com",
...> aws_sigv4: [
...> access_key_id: System.get_env("AWS_ACCESS_KEY_ID"),
...> secret_access_key: System.get_env("AWS_SECRET_ACCESS_KEY"),
...> service: :s3
...> ]
...> )
iex>
iex> %{status: 200} = Req.put!(req, "/bucket1/key1", body: "Hello, World!")
iex> resp = Req.get!(req, "/bucket1/key1").body
"Hello, World!"
Request body streaming also works (though content-length header must be explicitly set):
iex> path = "a.txt"
iex> File.write!(path, String.duplicate("a", 100_000))
iex> size = File.stat!(path).size
iex> chunk_size = 10 * 1024
iex> stream = File.stream!(path, chunk_size)
iex> %{status: 200} = Req.put!(req, url: "/key1", headers: [content_length: size], body: stream)
iex> byte_size(Req.get!(req, "/bucket1/key1").body)
100_000
Req.Test
Functions for creating test stubs.
Stubs provide canned answers to calls made during the test, usually not responding at all to
anything outside what’s programmed in for the test.
Req has built-in support for stubs via :plug, :adapter, and (indirectly) :base_url
options. This module enhances these capabilities by providing:
-
Stub any value with
Req.Test.stub(name, value)and access it with
Req.Test.stub(name). These functions can be used in concurrent tests. -
Access plug stubs with
plug: {Req.Test, name}. -
Easily create JSON responses for Plug stubs with
Req.Test.json(conn, body).
Example
Imagine we’re building an app that displays weather for a given location using an HTTP weather
service:
defmodule MyApp.Weather do
def get_rating(location) do
case get_temperature(location) do
{:ok, %{status: 200, body: %{"celsius" => celsius}}} ->
cond do
celsius < 18.0 -> {:ok, :too_cold}
celsius < 30.0 -> {:ok, :nice}
true -> {:ok, :too_hot}
end
_ ->
:error
end
end
def get_temperature(location) do
[
base_url: "https://weather-service"
]
|> Keyword.merge(Application.get_env(:myapp, :weather_req_options, []))
|> Req.request(options)
end
end
We configure it for production:
# config/runtime.exs
config :myapp, weather_req_options: [
auth: {:bearer, System.fetch_env!("MYAPP_WEATHER_API_KEY")}
]
And tests:
# config/test.exs
config :myapp, weather_req_options: [
plug: {Req.Test, MyApp.Weather}
]
And now we can easily stub out values in concurrent tests:
use ExUnit.Case, async: true
test "nice weather" do
Req.Test.stub(MyApp.Weather, fn conn ->
Req.Test.json(conn, %{"celsius" => 25.0})
end)
assert MyApp.Weather.get_rating("Krakow, Poland") == {:ok, :nice}
end
Full changelog below:
v0.4.10 (2024-02-19)
-
run_finch: Default toconnect_options: [protocols: [:http1, :http2]]. -
run_finch: Change version requirement to~> 0.17, that is all versions up to1.0. -
put_aws_sigv4: Support streaming request body. -
auth: Always updateauthorizationheader. -
decode_body: Gracefully handle multiple content-type values. -
Req.Request.new/1: UseURI.parsefor now.
v0.4.9 (2024-02-14)
-
retry: Raise on invalid return from:retry_delayfunction -
run_finch: Update to Finch 0.17 -
run_finch: Deprecateconnect_options: [protocol: ...]in favour of
connect_options: [protocols: ...]]which defaults to[:http1, :http2], that is,
make request using HTTP/1 but if negotiated switch to HTTP/2 over the HTTP/1 connection. -
New step:
put_aws_sigv4- signs request with AWS Signature Version 4.
v0.4.8 (2023-12-11)
put_plug: Fix response streaming. Previously we were relying on unreleased
Plug features (which may never get released). Now, Plug adapter will emit the
entire response body as one chunk. Thus,
plug: plug, into: fn ... -> {:halt, acc} endis not yet supported as it
requires Plug changes that are still being discussed. On the flip side,
we should have much more stable Plug integration regardless of this small
limitation.
v0.4.7 (2023-12-11)
put_plug: Don’t crash if plug is not installed and :plug is not used
v0.4.6 (2023-12-11)
- New step:
checksum put_plug: Fix response streaming when plug usessend_resporsend_fileretry: Retry on:closed
v0.4.5 (2023-10-27)
-
decompress_body: Removecontent-lengthheader -
auth: Deprecateauth: {user, pass}in favour ofauth: {:basic, "user:pass"} -
Req.Request: Allow steps to be{mod, fun, args}
v0.4.4 (2023-10-05)
-
compressed: Check for optional depenedencies brotli and ezstd only at compile-time.
(backported from v0.3.12.) -
decode_body: Check for optional depenedency nimble_csv at compile-time.
(backported from v0.3.12.) -
run_finch: Add:finch_privateoption
v0.4.3 (2023-09-13)
-
Req.new/1: Fix setting:redact_auth -
Req.Request: AddReq.Request.get_option_lazy/3 -
Req.Request: AddReq.Request.drop_options/2
v0.4.2 (2023-09-04)
-
put_plug: Handle response streaming on Plug 1.15+. -
Don’t warn on mixed-case header names
v0.4.1 (2023-09-01)
- Fix Req.Request Inspect regression
26
sergio
margarza
Hello! I am fairly new to the elixir language, so please bear with me 
I have a question about the Req.Test module:
We are trying to test a method in case an API call fails with an status code of 404 and 500.
We have tried it with this code:
Req.Test.stub(:stub_req_plug, fn conn ->
conn
|> Plug.Conn.resp(404, "Not found")
|> Plug.Conn.send_resp()
end)
this returns {:ok, "Not Found"}, but we were wondering what we can do to get a return value of {:error, reason}
0
zachallaun
I believe you’re looking for the handle_http_errors step. It should look something like:
Req.get(test_endpoint, http_errors: :raise)
When you use the non-! variants (get as opposed to get!), this should result in the error tuple you’re looking for. When you use the ! variant, it’ll raise the exception.
2
wojtekmach
Hex Core Team
http_errors: :raise always raises the exception. There were previous discussions of adding a non-raising variant that folks can find in the issue tracker.
This is not possible using the :plug option at the moment. My advice would be to start a real HTTP server using bypass v2.1.0 — Documentation and simulate network failures using it. For example, you can set a low Req :receive_timeout and Process.sleep(1000) in your Bypass stub. Instead of setting plug: ... you’d be setting url: ... (usually base_url: ...). You can still use Req.Test module to stub out the url!
# config/test.exs
config :myapp, req_options: [
# you need latest Req for :base_url to accept a function
base_url: fn -> Req.Test.stub(:mystub) end
]
# some_test.exs
bypass = Bypass.open()
Req.Test.stub(:mystub, "http://localhost:#{bypass.port}")
Bypass.stub(bypass, fn conn ->
...
end)
The easiest network failure to test for is :econnrefused which you can get by hitting a port that is not bound so for example:
Req.Test.stub(:mystub, "http://localhost:9999")
assert MyApp.some_request() ==
{:error, %Mint.TransportError{reason: :econnrefused}
Just a friendly reminder, if you are able to pass Req options down through all the layers, that’s preferred than going through Req.Test. For example:
- Req.Test.stub(:mystub, "http://localhost:9999")
-
- assert MyApp.some_request() ==
+ assert MyApp.some_request(req_options: [base_url: "http://localhost:9999"]) ==
{:error, %Mint.TransportError{reason: :econnrefused}
In future version of Req we plan to support simulating transport errors in :plug, something like this:
plug = fn _conn ->
:econnrefused
end
assert Req.get(plug: plug) ==
%Req.TransportError{reason: :econnrefused}
exact API TBD. Stay tuned!
2
lud
Hey @wojtekmach
We are using the new test functionality and it works great. But we want to run a couple integration tests and we would like to disable the test plug for those tests.
Is there a simple way to ignore the specified plug? We do not control the Req instance in the test, so we do not want to pass that option. We want to avoid messing with Application.put_env in the test if possible. We would like to set a flag in the test process and have the plug “ignore itself” if possible.
I see that if a plug is defined, the adapter will be set to run_plug/1, so our plug will always be called. Is there a safe way to define a custom plug and fallback to Req.Test by default, but call run_finch if the flag is set ?
(everything happens in the same process as the test)
0
wojtekmach
Hex Core Team
Req.Test.stub/1,2 can be used to stub anything so you could stub Req options:
defmodule MyApp.MyClient do
def new(options) do
Req.new(...)
|> merge_test_options()
end
if Mix.env() == :test do
defp merge_test_options(req) do
Req.merge(req, plug: Req.Test.stub(__MODULE__))
end
else
defp merge_test_options(req) do
req
end
end
end
# test with stub:
Req.Test.stub(MyApp.MyClient, fn conn -> ... end)
# integration test:
Req.Test.stub(MyApp.MyClient, nil)
When you set plug: nil it is as if you never set the option in the first place.
It’s a little bit ugly to change production code to accomodate tests like but I believe it is the most straighforward option. There could be more indirect ways I did not consider yet.
0
lud
I’d like to avoid doing that if possible but if it is the only option then fine.
Thank you 
0