Supervisor for HTTP request to external API?

Questions / Help
1

I’m building an Elixir Umbrella project to be a web service for Alexa Skills (here is the apps folder on GitHub.) When I speak a request to Alexa, Alexa sends a JSON request to my Phoenix app. My Phoenix app calls a function in my Alexa Skills app to process the request. I have an app to transform the JSON to Elixir structs and generate the response struct. Then I built another app in my umbrella that sends a HTTP request to get the latest train status for the nearest metro stop. The goal is to send that info back to Alexa formatted in Alexa response JSON format. I have it working, but I’m wondering if there is a more idiomatic way to structure it from an OTP perspective.

I built the metro app “wmata” using the Metex app from The Little Elixir and OTP Guidebook as a template. The Metex app makes an http request to a weather API and stores the results. To store the results it uses a GenServer. I just updated it to send the request to metro API.

For my app, I’m not interested in storing the results. I just want to format the JSON and send it back to my Alexa. Since I don’t need to store the results, I don’t think I need a GenServer. Furthermore, as I’ve read up on GenServers, it could potentially create a bottleneck in my overall application (not that I expect it to get that big, but I want to create my app using a proper design.) Also, I don’t know that a GenServer provides me a benefit in my case.

Here are a couple of posts I found on the topic.
Question Regarding GenServer Use
Task, GenServer, or GenStage

Should I simply remove the GenServer and call the function from the process the Phoenix app creates? Or do people set up something like a supervised Task to handle external API http requests? I’m using HTTPoison, so I’m guessing any supervision/error handling is done within HTTPoison or Hackney. Is a supervisor necessary in my case?

4 Likes
2

I was looking through the Programming Phoenix book and found the implementation for the Wolfram Info System in the OTP chapter is very similar to what I’m doing. It uses a Supervisor with a strategy of :simple_one_for_one, starts a Task for each request, then awaits the results.

I was able to remove the GenServer and use Task.Supervisor/4 and Task.await/0 to implement what I think might be a similar approach in my WMATA app.

3 Likes
3

Following up for anyone else playing around with this. I initially was using Task.Supervisor.async/4. However, I found when the http request process crashed, it was linked to the main process and would crash that as well.

I realized I needed to understand the fundamentals better, so I went back through the Programming Phoenix OTP chapter. I added an app to my project to get the status of the docks for the local bike share service using the OTP chapter as a guide.

At the end of the chapter, there is a callout “Jose says:” where he discusses the issue with Task.async/await (if the task crashes, it also crashes the caller.) The book has you build the application using Task and Process functions, but he says you could use Task.Supervisor.async_nolink function to replicate the functionality discussed in the chapter.

3 Likes
4

I refactored my app to use Task.Supervisor.async_nolink/4, Task.yield/2, and Task.shutdown/2 using the tests to make sure it continued to handle success, errors, and timeouts as expected.

It was a great exercise to understand how tasks work better. I started off using Task.Supervisor.async/4 and Task.await/2, but those weren’t giving me the desired results, so I had to take a look at the source code to understand what each of the functions were doing.

lib/wmata.ex:

defmodule WMATA do
  @backend WMATA.API

  def station_info(backend, query, owner) do
    backend.station_info(query, owner)
  end

  def get_station_info(station_code, platform, opts \\ []) do
    backend = opts[:backend] || @backend
    query = [station_code: station_code, platform: platform]
    owner = self()
    timeout = opts[:timeout] || 5000

    backend
    |> spawn_query(query, owner)
    |> handle_result(timeout)

  end

  def spawn_query(backend, query, owner) do
    Task.Supervisor.async_nolink(
      WMATA.TaskSupervisor, __MODULE__, :station_info, [backend, query, owner]
    )
  end

  def handle_result(task, timeout) do
    case Task.yield(task, timeout) || Task.shutdown(task) do
      {:ok, result} ->
        result
      {:exit, _reason} ->
        "There was an error with the request."
      nil ->
        "The request timed out."
    end
  end
end

test/wmata_test.exs

defmodule WMATATest do
  use ExUnit.Case

  defmodule TestBackend do
    def station_info([station_code: "result", platform: _], _owner) do
      "Success"
    end

    def station_info([station_code: "timeout", platform: _], owner) do
      send(owner, {:backend, self()})
      :timer.sleep(:infinity)
    end

    def station_info([station_code: "boom", platform: _], _owner) do
      raise "boom!"
    end
  end

  describe "get_station_info/3" do
    test "with backend results" do
      result = WMATA.get_station_info("result", "2", backend: TestBackend)

      assert result == "Success"
    end

    test "timeout returns no results and kills workers" do
      opts = [backend: TestBackend, timeout: 10]

      result = WMATA.get_station_info("timeout", "2", opts)

      assert result == "The request timed out."
      assert_receive {:backend, backend_pid}
      ref = Process.monitor(backend_pid)
      assert_receive {:DOWN, ^ref, :process, _proc, _reason}
      refute_received {:DOWN, _, _, _}
      refute_received :timedout
    end

    @tag :capture_log
    test "discards backend errors" do
      result = WMATA.get_station_info("boom", "2", backend: TestBackend)

      assert result == "There was an error with the request."
      refute_received {:DOWN, _, _, _}
      refute_received :timedout
    end
  end
end
7 Likes
5

Thanks axelclark at a different point in my discovery journey but this is just what I was looking for.

2 Likes