Reusable/mountable applications (ala Rails Engines)

Hello all,

is there some open source examples of a pluggable Hex packages that gives the main Phoenix application some other functionality? Things that are possibly valuable to more than one application like job console, accounting, subscription management.

Idea is to write it once and use it in all other projects. There should be some knowledge sharing (think current user). Something like Rails Engines [0]. Does anybody know of something? Do you have some pointers?

[0] https://guides.rubyonrails.org/engines.html

Why should those extra functionality be coupled to phoenix? And if so: What would you expect out of this coupling?

Maybe it shouldn’t! I am interested how people are solving this problem and if it’s possible or if there is some limitation to it.

I quite like the concept - things like wikis, media asset tracking and user management would be handy just to “pull in” from somewhere.

I think all the building blocks are there, e.g. umbrella apps (I’ve not had the pleasure of setting them up yet - see https://blog.appsignal.com/2019/04/16/elixir-alchemy-routing-phoenix-umbrella-apps.html for a discussion).

Pow (https://hexdocs.pm/pow/README.html#installation) packages views/templates/controllers into the library, and you activate them by calling pow_routes() in your router - that’s pretty low overhead for bringing new screens into your app.

The Oban web UI - see https://oban.dev/ - also looks pretty close to what you have in mind. You might ping @sorentwo to find out how it’s done.

I have already published it here (at least once), but I have created something “in the vain of” that:

defmodule MyWebAppWeb.Endpoint do
  use Plug.Builder

  require Logger

  @endpoints [
    MyWebAppRest.Endpoint,
    MyWebAppUi.Endpoint
  ]

  def child_spec({scheme, options}) do
    dispatches =
      @endpoints
      |> gen_dispatches()
      |> check_dispatches()

    options =
      options
      |> Keyword.put(:cipher_suite, :strong)
      |> Keyword.put_new(:dispatch, _: dispatches)
      |> Keyword.put_new(:keyfile, System.get_env("SSL_KEY"))
      |> Keyword.put_new(:certfile, System.get_env("SSL_CERT"))
      |> Keyword.put_new_lazy(:port, fn -> port(scheme) end)

    spec =
      Plug.Cowboy.child_spec(
        scheme: scheme,
        plug: __MODULE__,
        options: options
      )

    update_in(spec, [:start], &{__MODULE__, :start_link, [scheme, &1]})
  end

  def start_link(scheme, {m, f, [ref | _] = a}) do
    case apply(m, f, a) do
      {:ok, pid} ->
        :logger.info(&info/1, {scheme, __MODULE__, ref})

        {:ok, pid}

      {:error, {:shutdown, {_, _, {{_, {:error, :eaddrinuse}}, _}}}} = error ->
        Logger.error(
          info({scheme, __MODULE__, ref}) <> " failed, port already in use"
        )

        error

      {:error, _} = error ->
        error
    end
  end

  defp gen_dispatches(endpoints) do
    for endpoint <- endpoints do
      url = endpoint.config(:url, [path: "/"]) |> Keyword.fetch!(:path)
      prefix =
        case Path.split(url) do
          ["/" | rest] -> rest
          rest -> rest
        end

      path =
        case url do
          "/" -> :_
          other -> Path.join(["/", other, "[...]"])
        end

      {path, MyWebAppWeb.Endpoint.Handler, {endpoint, prefix, endpoint.init([])}}
    end
  end

  defp check_dispatches(dispatches) do
    entries =
      dispatches
      |> Enum.map(&elem(&1, 0))
      |> Enum.sort()

    :ok = find_duplicate(entries)

    dispatches
  catch
    {:duplicate, prefix} ->
      raise "Duplicated prefix #{inspect(prefix)} in\n#{inspect(dispatches)}"
  end

  defp find_duplicate([a, a | _]) do
    throw({:duplicate, a})
  end
  defp find_duplicate([_ | rest]), do: find_duplicate(rest)
  defp find_duplicate([]), do: :ok

  defp info({scheme, endpoint, ref}) do
    server = "cowboy #{Application.spec(:cowboy, :vsn)}"

    "Running #{inspect(endpoint)} with #{server} at #{uri(scheme, ref)}"
  end

  defp uri(scheme, ref) do
    {host, port} = :ranch.get_addr(ref)

    %URI{
      scheme: to_string(scheme),
      host: List.to_string(:inet.ntoa(host)),
      port: port
    }
  end

  defp port(scheme), do: String.to_integer(System.get_env("PORT_#{scheme}"))

  plug(
    MyWebAppWeb.Plugs.Health,
    applications: [:kokon, :kokon_web, :kokon_ui, :kokon_rest]
  )

  plug(Plug.Telemetry.ServerTiming)
  plug(Plug.Telemetry, event_prefix: [:phoenix, :endpoint])

  plug(PlugClacks, names: ["Joe Armstrong"])

  plug(MyWebAppWeb.Plugs.Trace)
  plug(MyWebAppWeb.Plugs.Measure)
end

This allows to have multiple Phoenix applications that “feel” independent, but are hosted by single Cowboy instance.

3 Likes

Phoenix Live Dashboard implemented similar way to rails engines. How to do it from scratch is still the open question.

As the person who co-developed Rails engines, one of the reasons why we needed Rails engines was because in Rails a lot of the values are global, so we needed a way to break them apart and isolate them into distinct instances (engines).

In Phoenix, it is the opposite. You don’t have a singleton Phoenix.application() like Rails.application(). You have distinct endpoints. You have multiple routers and you can choose which one to forward to or plug into your application, etc. Most configuration is fully compartmentalized except for 3 or 4 configuration values that you configure directly in the :phoenix app itself. That’s what allows a single Elixir application to have two distinct Phoenix endpoints running side by side on separate ports without any additional work.

The same applies to Ecto, you have multiple repositories which you access directly instead of a global default pool, and so on.

So my suggestion is to look at the examples listed here, the dashboard, oban pro, pow, to see how they are tackling those problems. Other than that, it is hard to answer what “from scratch” means, because even Rails engines encapsulates many aspects. For example, Devise was a rails engine, but not because it had to be its own application, but because it wanted its own config instance. And that is different from an engine that provided its own router. :slight_smile: More specific questions to the problem you are trying to solve, such as how to share routers, how tackle configuration, etc may provide more addressable points for discussion.

4 Likes

Thanks, that makes a lot of sense!