Need Help Regarding Fetching Data from Multiple URLs

Questions / Help
1

I’m currently learning elixir and thus am facing problem in fetching data from multiple URLs with the help of HTTPoison, then bundling up all the response and sending them back. I’m using Phoenix as my server.

Basically I want to get the data from these URLs:
http://jsonplaceholder.typicode.com/posts
http://jsonplaceholder.typicode.com/comments
http://jsonplaceholder.typicode.com/photos
http://jsonplaceholder.typicode.com/users

And then bundle up the response like:

json conn, %{"success": true, "users": users, "posts": posts, "comments": comments, "photos": photos}

And send it back to the client

I tried looking into Issue #108 but was unable to figure out how to handle async requests. I was successfully able to get for sync request using:

def index(conn, %{"url" => url}) do
    
    case HTTPoison.get(url) do
      {:ok, %HTTPoison.Response{status_code: 200, body: body}} ->
        json conn, %{"success": true, "body": body}
      {:ok, %HTTPoison.Response{status_code: 404}} ->
        json conn, %{"success": false, "body": "Not Found"}
      {:error, %HTTPoison.Error{reason: reason}} ->
        IO.inspect reason
        json conn, %{"success": false}
      _ ->
        json conn, %{"success": true, "reason": "Other Status Code"}
    end
  end

Any help with a small example will be appreciated

1 Like
2

You might want to look at Tasks.

And do something like …

def index(conn, _) do
  urls = [
    "http://jsonplaceholder.typicode.com/posts3",
    "http://jsonplaceholder.typicode.com/comments",
    "http://jsonplaceholder.typicode.com/photos",
    "http://jsonplaceholder.typicode.com/users"
  ]
  
  result = urls
  |> Enum.map(&Task.async(__MODULE__, :fetch_url, [&1]))
  |> Enum.map(&Task.await(&1, 5_000))
  
  # Do what You want with result, it will be a list of {:ok, body} | {:error, reason}
end

def fetch_url(url)
  case HTTPoison.get(url) do
    {:ok, %HTTPoison.Response{status_code: 200, body: body}} ->
      {:ok, body}
    {:ok, %HTTPoison.Response{status_code: 404}} ->
      {:error, "Not Found"}
    {:error, %HTTPoison.Error{reason: reason}} ->
      {:error, reason}
    _ ->
      {:error, "Other Status Code"}
  end
end

But I am not sure of what You want.

It looks liks You want to get info for user, including his photos, posts and all comments for them.

In that case, You would need to provide a user_id as params, and build urls accordingly.

Note also that fetching comments depends on fetching posts first.

2 Likes
3

Thanks a billion. I think that should solve my problem.