dnkbln

dnkbln

Req.get redirects and special characters

I use the Req library to download a resource. I got the resource from a podcast RSS feed.

enclosure_url = "https://example.com/file 1.mp3"
{:ok, resp} = Req.get(enclosure_url)

I got an error from the Req library. I added an encode Url step, but the resource has some redirects to the final location and in the redirects contains also special characters (eq. a blank).

enclosure_url = "https://example.com/file 1.mp3"
encode_url = URI.encode(enclosure_url)
{:ok, resp} = Req.get(encode_url)

https://example.com/file 1.mp3 → https://cdn.example.com/file 1.mp3

How could I solve this problem?

Remark: I hope the question is not stupid. I’m just starting out with Elixir programming.
I used Elixir with the Req library and without Phoenix.

Most Liked

wojtekmach

wojtekmach

Hex Core Team

This is a bug in Req, we should be calling your request steps when making the redirect. I’d have to track the history how we ended up there but FWIW here’s a minimal reproduction:

Mix.install([
  {:req, path: "~/src/req"},
  :bandit
])

{:ok, _} =
  Bandit.start_link(
    plug: fn
      conn, _ when conn.request_path == "/" ->
        conn
        |> Plug.Conn.put_resp_header("location", "/foo bar")
        |> Plug.Conn.send_resp(301, "redirecting")

      conn, _ when conn.request_path == "/foo%20bar" ->
        Plug.Conn.send_resp(conn, 200, "ok")
    end,
    port: 8000
  )

Req.new()
|> Req.Request.prepend_request_steps(
  encode_url: fn r ->
    dbg(:encode_url)
    update_in(r.url.path, &(&1 && URI.encode(&1)))
  end
)
|> Req.get!(url: "http://localhost:8000")

and here’s a fix:

diff --git a/lib/req/steps.ex b/lib/req/steps.ex
index 33e332e..8e8fc73 100644
--- a/lib/req/steps.ex
+++ b/lib/req/steps.ex
@@ -1813,6 +1813,8 @@ defmodule Req.Steps do
           |> build_redirect_request(response, location)
           |> Req.Request.put_private(:req_redirect_count, redirect_count + 1)

+        request = put_in(request.current_request_steps, Keyword.keys(request.request_steps))
+
         {request, response_or_exception} = Req.Request.run_request(request)
         Req.Request.halt(request, response_or_exception)
       else

Stay tuned!

Last Post!

dnkbln

dnkbln

I have tried to use the workaround. I encountered a problem if the URL contains query parameters and these contain special characters https://example.com/file 1.mp3 → https://cdn.example.com/file 1.mp3?file=file 1.mp3. The query part is not modified. An attempt is made to access https://cdn.example.com/file%201.mp3?file=file 1.mp3.

But thanks again for the quick help

Where Next?

Popular in Questions Top

baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New

Other popular topics Top

JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 31586 112
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New

We're in Beta

About us Mission Statement