Redirect in Router does not work as expected

I’m catching the following error in my router.ex:

  defp handle_errors(conn, %{kind: :error, reason: reason = %Bodyguard.NotAuthorizedError{}, stack: stacktrace}) do
    Plug.LoggerJSON.log_error(:error, reason, stacktrace)
    conn
    |> redirect(to: "/")
    |> put_flash(:error, gettext("This page does not exist!"))
    |> halt
  end

The matching works, but even though I want to redirect, I only get in my chrome a You are being redirected. message to my redirect path. What do I need to do to get this working?

I was able to get it working with the with statement in my controller function but I dislike this syntax so much I absolutely do not want to use it. Is there a way to get it working with the error?