Architecting an app suite with a Phoenix 1.3 umbrella project

Phoenix Forum Questions / Help
,
1

This is a cross post from Stack Exchange. I should have posted it here in the first place.

I’m playing around with an architecture for a new suite of products using phx 1.3 and umbrella apps.

I have an existing Phoenix based enterprise class WebRTC soft phone (many keys, a display, multiple input and output audio device selection, and more). I have developed a Slack clone messaging app prototype with Phoenix. Both applications are fairly large I need to integrate the phone with the chat app into one front end that may be either just the phone, just the chat client, and both. I will need to add a lot of new features to the chat client moving forward I also want the architecture to support using the same client to provision additional setting on the call server (user based) and potentially a large number of admin level settings. I may also be adding other applications in the future like an operator panel, log viewer, and the list goes on… The client side JS is pretty simple, no front end framework. I render templates server side and push the html out over channels.

I would like to build this pluggable. Same endpoint and database. One common UX.

I think there will be two common apps in the umbrella, one for the Phoenix endpoint and a couple controllers and another for the main Repo and a couple schemas. I’m trying to figure out how difficult it will be to use two or more additional apps for each application. One for context and schema, another for controllers, views, templates, and brunch resource. Possibly another for 3rd party APIs.

To make this work, I’ll need a dynamic dispatch for routers in each of the apps. A method for handling migrations contained in each of the apps, and probably more that I have not thought of yet.

As anyone tried this? Are there any open source projects with a similar structure?

I have part of the solution for routing

defmodule PhxUmbrella.Web.Router do
  use PhxUmbrella.Web, :router
  # ...
  Application.get_env(:phx_umbrella_web, :routers)
  |> Enum.each(fn mod ->
    forward "/", mod
  end)
end

This works in my spike project. However, I ran into a circular dependency issue on my main project. For the above to work, the main web app needs to have the other apps as dependencies. However, If I have APIs in the main web app that need to be exposed to the other apps, I would need to setup a dependency that way. Adding bidirectional deps in mix does not appear to work as I had suspected (mix deps.get) just hangs.

I’ll keep posting on this topic if I see people are interested. Your feedback and ideas are most welcome.

2 Likes
2

Well, why not extracting this APIs to another app? Then the other child apps can depend of this new app, while your main app still depending on these child apps.

2 Likes
3

Yes, I was thinking the same. I’m trying to figure out the APIs I need to expose to the other apps.

1 Like
4

Figuring this out right here is a great design consideration no matter how fine grained you get, always think of how if you were to break things apart then what would be the best API for it. :slight_smile:

4 Likes
5

It turns out that this is much more complicated than I originally thought it would be. I’m at the point where I don’t think an umbrella solution is going to work.

The main issue I struggle with is umbrella dependencies. For example, I created a Users app and added stuff pertaining to users in that app including authentication, authorization, etc. Now, in another app (Chat for example), I want to add a relationship to the users schema like ownership of a message or room channel. I can include the Users app as an umbrella dependency. However, I want to add a has_many :messages, Chat.Message to the User schema. I don’t want to, and can’t, just add it to the users schema in the Users app since we can’t have circular dependencies.

As I just typed that last sentence a thought came to me :bulb:. I could create a User schema module in Chat, that adds the relationship. We can have multiple schema files for the same DB table. I’ll need to investigate that… Back to my post…

The other issue here is that I believe the whole idea of umbrellas is to create apps that can stand on their own. I need apps that are interdependent.

Last night I started playing around with the idea of a plugin approach. I’m working on a spike with some positive results. The idea is that I add a plugins folder to the top level of a non-umbrella project.

Each plugin is contained in its own folder and contains a config.exs file (right now at the top level but I’m going to move it to config/config.exs. Something like this:

── plugins
│   ├── comments
│   │   ├── config.exs
│   │   └── lib
│   │       ├── comments
│   │       │   ├── comment.ex
│   │       │   ├── comments.ex
│   │       │   └── post.ex
│   │       └── web
│   ├── plugin2
│   └── plugin3
│       └── config.exs

The config file for comments looks like this:

# plugins/comments/config.exs
use Mix.Config

config :ucc, :plugin, comments: [
  schemas: [Comments.Post]
]

And in my main config.exs

# config/config.exs
use Mix.Config
# ...
import_config("../plugins/*/config.exs")
import_config "#{Mix.env}.exs"

That takes care of plugin config.

Now, how to hook into the main project? I’m started with schema. In the comments plugin, I want to add a has_many :comments, Comments.Comments to the Post schema in the main project. (Ignore the name spacing for now. I’ll decide that later).

Here is what I have so far to handle this:

# plugins/comments/lib/comments/post.ex
defmodule Comments.Post do
  use Ucc.Plugin.Schema

  extend_schema Ucc.Blog.Post do
    has_many :comments, Comments.Comment
  end
end

and here is the main Post schema file

# lib/blog/post.ex    
defmodule Ucc.Blog.Post do
  use Ucc.Schema

  schema "blog_posts" do
    field :body, :string
    field :title, :string

    timestamps()
  end
end

I almost have the Ucc.Schema.schema/2 macro working. It looks like this:

# lib/ucc/schema.ex
defmodule Ucc.Schema do

  defmacro __using__(_) do
    quote do
      import unquote(__MODULE__)
      use Ecto.Schema
      import Ecto.Schema, except: [schema: 1, schema: 2]
    end
  end

  defmacro schema(table, do: block) do
    calling_mod = __CALLER__.module

    modules =
      get_schemas()
      |> Enum.map(fn mod ->
        Code.ensure_compiled(mod)
        mod
      end)
      |> Enum.reduce([], fn mod, acc ->
        if mod.module() == calling_mod, do: [mod | acc], else:  acc
      end)

    quote do
      Ecto.Schema.schema unquote(table) do
        unquote(block)
        Enum.map(unquote(modules), fn mod ->
          mod.schema() # not working yet
        end)
      end
    end
  end

  def get_schemas do
    :ucc
    |> Application.get_env(:plugin)
    |> Enum.reduce([], fn {plugin, list}, acc ->
      if mods = list[:schemas], do: acc ++ mods, else: acc
    end)
  end
end

Once I get the schema working, I’ll do the routes, controllers, views, and templates next.

Any thoughts on this approach?

4 Likes