Using Phoenix as a "kitchen sink" web framework vs a lightweight toolkit?

Hi guys, I’m new to Elixir and my background has mainly been focused on JS & Node. I am not entirely happy with the Node ecosystem :roll_eyes: and for my next side project I’m looking for a stack that is productive, but also performant. Hence my interest in Elixir and Phoenix.

I’m currently going through Dave Thomas’s course Elixir for Programmers and he recommends against using Phoenix as a full-blown kitchen sink web framework ( à la Ruby on Rails). He states “A pure web interface has no need of a database—all persistence work should be performed in services. I strongly suggest you fight the temptation to stick everything in the Phoenix app.”

And given his reasoning, I’m inclined to agree. I can see how building code into separate, well-defined applications would probably lead to better maintainability and scalability. But, how would this impact productivity? Is this approach likely to slow you down do you think?

I just wondered what other people’s opinions are on this matter. How are you using Phoenix? Thanks!

In my side project Phoenix is just simple and small part of the umbrella that is only responsible for HTTP requests. Even more - I have 2 separate Phoenix applications that are then joined together into single “endpoint” where one application is responsible for public HTTP API and second one is responsible for application UI (via LiveView). In that way the logic is in completely separate application and doesn’t “know” about the HTTP or any other transport protocol at all.

2 Likes

Oh cool that’s a really good idea. One thing I like about using SPAs (at work) is the separation of concerns between the client and server. This seems like you get that separation, without a lot of the added complexity that comes with SPAs.

What made you choose that approach? And have you experienced any disadvantages? Sounds like it would take a little bit more time to put together?

Not really, it just required some knowledge how to write “custom” Plug handler to handle sockets. However in the end that was not that hard and in total is less than 300 lines while most of them are copied from the Phoenix Cowboy handler:

defmodule MyAppWeb.Endpoint.Handler do
  @moduledoc false

  if Code.ensure_loaded?(:cowboy_websocket) and
       function_exported?(:cowboy_websocket, :behaviour_info, 1) do
    @behaviour :cowboy_websocket
  end

  @connection Plug.Cowboy.Conn
  @already_sent {:plug_conn, :sent}

  # Note we keep the websocket state as [handler | state]
  # to avoid conflicts with {endpoint, opts}.
  def init(req, {endpoint, prefix, opts}) do
    conn = @connection.conn(req)

    try do
      path_info = strip_prefix(prefix, conn.path_info)
      conn = %{conn | path_info: path_info}

      case endpoint.__handler__(conn, opts) do
        {:websocket, conn, handler, opts} ->
          case Phoenix.Transports.WebSocket.connect(
                 conn,
                 endpoint,
                 handler,
                 opts
               ) do
            {:ok, %{adapter: {@connection, req}}, state} ->
              cowboy_opts =
                opts
                |> Enum.flat_map(fn
                  {:timeout, timeout} -> [idle_timeout: timeout]
                  {:compress, _} = opt -> [opt]
                  {:max_frame_size, _} = opt -> [opt]
                  _other -> []
                end)
                |> Map.new()

              {:cowboy_websocket, req, [handler | state], cowboy_opts}

            {:error, %{adapter: {@connection, req}}} ->
              {:ok, req, {handler, opts}}
          end

        {:plug, conn, handler, opts} ->
          %{adapter: {@connection, req}} =
            case MyAppWeb.Endpoint.call(conn, opts) do
              %Plug.Conn{halted: true} = conn ->
                conn

              conn ->
                conn
                |> handler.call(opts)
                |> maybe_send(handler)
            end

          {:ok, req, {handler, prefix, opts}}
      end
    catch
      :error, value ->
        stack = System.stacktrace()
        exception = Exception.normalize(:error, value, stack)
        exit({{exception, stack}, {endpoint, :call, [conn, opts]}})

      :throw, value ->
        stack = System.stacktrace()
        exit({{{:nocatch, value}, stack}, {endpoint, :call, [conn, opts]}})

      :exit, value ->
        exit({value, {endpoint, :call, [conn, opts]}})
    after
      receive do
        @already_sent -> :ok
      after
        0 -> :ok
      end
    end
  end

  defp strip_prefix([x | xs], [x | ys]), do: strip_prefix(xs, ys)
  defp strip_prefix([], ys), do: ys

  defp maybe_send(%Plug.Conn{state: :unset}, _plug),
    do: raise(Plug.Conn.NotSentError)

  defp maybe_send(%Plug.Conn{state: :set} = conn, _plug),
    do: Plug.Conn.send_resp(conn)

  defp maybe_send(%Plug.Conn{} = conn, _plug), do: conn

  defp maybe_send(other, plug) do
    raise "Cowboy2 adapter expected #{inspect(plug)} to return Plug.Conn but got: " <>
            inspect(other)
  end

  ## Websocket callbacks

  def websocket_init([handler | state]) do
    {:ok, state} = handler.init(state)
    {:ok, [handler | state]}
  end

  def websocket_handle({opcode, payload}, [handler | state])
      when opcode in [:text, :binary] do
    handle_reply(handler, handler.handle_in({payload, opcode: opcode}, state))
  end

  def websocket_handle(_other, handler_state) do
    {:ok, handler_state}
  end

  def websocket_info(message, [handler | state]) do
    handle_reply(handler, handler.handle_info(message, state))
  end

  def terminate(_reason, _req, {_handler, _prefix, _state}) do
    :ok
  end

  def terminate({:error, :closed}, _req, [handler | state]) do
    handler.terminate(:closed, state)
  end

  def terminate({:remote, :closed}, _req, [handler | state]) do
    handler.terminate(:closed, state)
  end

  def terminate({:remote, code, _}, _req, [handler | state])
      when code in 1000..1003 or code in 1005..1011 or code == 1015 do
    handler.terminate(:closed, state)
  end

  def terminate(:remote, _req, [handler | state]) do
    handler.terminate(:closed, state)
  end

  def terminate(reason, _req, [handler | state]) do
    handler.terminate(reason, state)
  end

  defp handle_reply(handler, {:ok, state}), do: {:ok, [handler | state]}

  defp handle_reply(handler, {:push, data, state}),
    do: {:reply, data, [handler | state]}

  defp handle_reply(handler, {:reply, _status, data, state}),
    do: {:reply, data, [handler | state]}

  defp handle_reply(handler, {:stop, _reason, state}),
    do: {:stop, [handler | state]}
end

And then “root” endpoint:

defmodule MyAppWeb.Endpoint do
  use Plug.Builder

  require Logger

  @endpoints [
    MyAppRest.Endpoint,
    MyAppUi.Endpoint
  ]

  def child_spec({scheme, options}) do
    dispatches =
      @endpoints
      |> gen_dispatches()
      |> check_dispatches()

    options =
      options
      |> Keyword.put(:cipher_suite, :strong)
      |> Keyword.put_new(:dispatch, _: dispatches)
      |> Keyword.put_new(:keyfile, System.get_env("SSL_KEY"))
      |> Keyword.put_new(:certfile, System.get_env("SSL_CERT"))
      |> Keyword.put_new_lazy(:port, fn -> port(scheme) end)

    spec =
      Plug.Cowboy.child_spec(
        scheme: scheme,
        plug: __MODULE__,
        options: options
      )

    update_in(spec, [:start], &{__MODULE__, :start_link, [scheme, &1]})
  end

  def start_link(scheme, {m, f, [ref | _] = a}) do
    case apply(m, f, a) do
      {:ok, pid} ->
        :logger.info(&info/1, {scheme, __MODULE__, ref})

        {:ok, pid}

      {:error, {:shutdown, {_, _, {{_, {:error, :eaddrinuse}}, _}}}} = error ->
        Logger.error(
          info({scheme, __MODULE__, ref}) <> " failed, port already in use"
        )

        error

      {:error, _} = error ->
        error
    end
  end

  defp gen_dispatches(endpoints) do
    for endpoint <- endpoints do
      url = endpoint.config(:url, [path: "/"]) |> Keyword.fetch!(:path)
      prefix =
        case Path.split(url) do
          ["/" | rest] -> rest
          rest -> rest
        end

      path =
        case url do
          "/" -> :_
          other -> Path.join(["/", other, "[...]"])
        end

      {path, MyAppWeb.Endpoint.Handler, {endpoint, prefix, endpoint.init([])}}
    end
  end

  defp check_dispatches(dispatches) do
    entries =
      dispatches
      |> Enum.map(&elem(&1, 0))
      |> Enum.sort()

    :ok = find_duplicate(entries)

    dispatches
  end

  defp find_duplicate([a, a | _]), do: raise "Duplicated prefix #{inspect(a)}"
  defp find_duplicate([_ | rest]), do: find_duplicate(rest)
  defp find_duplicate([]), do: :ok

  defp info({scheme, endpoint, ref}) do
    server = "cowboy #{Application.spec(:cowboy, :vsn)}"

    "Running #{inspect(endpoint)} with #{server} at #{uri(scheme, ref)}"
  end

  defp uri(scheme, ref) do
    {host, port} = :ranch.get_addr(ref)

    %URI{
      scheme: to_string(scheme),
      host: List.to_string(:inet.ntoa(host)),
      port: port
    }
  end

  defp port(scheme), do: String.to_integer(System.get_env("PORT_#{scheme}"))

  plug(
    MyAppWeb.Plugs.Health,
    applications: [:my_app, :my_app_web, :my_app_ui, :my_app_rest]
  )

  plug(Plug.Telemetry.ServerTiming)
  plug(MyAppWeb.Plugs.Trace)
  plug(Plug.Telemetry, event_prefix: [:phoenix, :endpoint])

  # In memory of:
  # - Terry Pratchett (default)
  # - Joe Armstrong
  plug(MyAppWeb.Plugs.Clacks, names: ["Joe Armstrong"])

  plug(MyAppWeb.Plugs.Measure)
end

Which also uses some custom plugs before it even reaches “main” handlers:

  • heath checks that responds with JSON containing application versions
  • custom tracing for OpenCensus (soon to be done via Telemetry and OpenTelemetry)
  • Server-Timing
  • Clacks subsystem
  • OpenCensus metrics (which I need to replace with Telemetry handlers)
4 Likes

And I am very inclined to disagree. Depending on if you are Google scale or not (and I strongly doubt you are) having a monolith where you have tight code control is the optimal solution. You can either use the Elixir umbrella project structure as @hauleth suggested or, as I do recently, just isolate the project’s different responsibilities in different coding directories – lib/yourapp_web for your Phoenix stuff and lib/yourapp for basically everything else. You can do it even more fine-grained and have something like lib/yourapp/domain for business logic, lib/yourapp/db for DB-specific code etc.

The sky is the limit.

Dave Thomas is an amazing teacher and I have deep respect for him. Amazing teachers shouldn’t be replacements for thinking for yourself though. :wink:

Coding monoliths are quite fine 90% of the time.

8 Likes

Not only fine, but are the best solution. All big companies started with monolith that was later split into micro services. Starting with micro services is IMHO the worst thing possible, as you will never know the boundaries beforehand.

3 Likes

Welcome :023:

There are some great discussions surrounding his thoughts, you can view them at the course’s tag: elixir-for-programmers-course :smiley:

There are three common ways to use Phoenix imo:

  • A monolith (similar to Rails)
  • Contexts (the middle ground and the sane default)
  • As a layer or multiple layers (as per Dave’s course or Umbrellas)

How you use them depends on your specific needs for the app, whether you’re focusing on speed of development, how ambitious the project is, etc.

It’s awesome that Phoenix is the flexible :003:

(I wonder if we should do a poll on this :lol:)

2 Likes

Yeah, even google has a monolith. I don’t think Dave is advocating for a micro services approach per se, at least my interpretation was that he would still suggesting a monolith (all in one repo). But one separated into small elixir applications and using Phoenix to handle http traffic and displaying views etc…

I’m not sure what’s different about your approach compared to Dave’s? I think I’ve quoted Dave out of context and it may have caused some confusion.

I agree!

Thank you and I’ll be sure to check them out.

Yeah that is really cool.

Sounds good, how do we do that?

Another option to keep an eye on is GitHub - sasa1977/boundary: Manage and restrain cross-module dependencies in Elixir projects it helps you enforce which modules in your application can call which other applications. It’s good because often separating your code into separate applications is too crude of a tool and you end up with inflexible boundaries.

4 Likes

I haven’t read the book so I don’t know what he’s talking about exactly or how up to date the book is.

But maybe he’s talking about keeping your business logic outside of the web related bits of a Phoenix app, which would be considered a good idea. But you can still have a single app project that happens to use Phoenix and your DB related code could live in contexts (which is like your public API for your business logic that your controllers can call).

1 Like

I do not quite understand that sentence…

It basically says, a database is not needed, use a database instead, as a database is just a service. A service that is built to store data.

What I might agree with though is, the presentation layer does not necessarily need to know where and what DB is actually used, but instead should retrieve and store data only through an abstraction layer. In phoenix this layer is called “contexts”.

1 Like

I believe that what Dave is advocating for is similar to what you’re saying in your second paragraph. However his approach is to split the business logic/persistence and the web interface into two separate applications, rather than using Phoenix Context’s to keep them in the same application but in different “namespaces”.

2 Likes