Issues with Project Structure / Ecto Config

project-structure
#1

Hello! I’m writing a game app and am running into a couple issues that make me feel like I’m not organizing my project correctly. I’d greatly appreciate any pointers here. Thanks in advance.

I’m building an app where users connect over Phoenix channels, spinning up player state processes (GenServers) that connect to a game state process (also GenServer). The organization is as follows:

/state_app   #genserver modules that manage player/game state
/supervisor   #supervisor module to spin up game/player/registry supervisors
/websocket_server   #phoenix, only used for websockets/channels
/ecto_repo  #connects to postgres for persistance

Ideally I just run the state_app and everything starts up/is supervised by calls to the supervisor module.

I’m running into a couple issues:

  1. I’m having trouble getting the supervisor module to manage the phoenix channel endpoint. I can point it to my endpoint module and start things up, but nothing gets served on :4000. :observer shows the processes running. I put config :phoenix, :serve_endpoints, true in my dev.exs but no luck. Currently I have to start both the game state with iex -S mix and then the phoenix server with mix phx.server in a separate window.

  2. I have to put all my Ecto repo config into both the state_app config.exs and websocket_server config.exs, since I’m starting them separately. I think it’s because the configs for dependencies aren’t used? Does this mean I want to use an umbrella app?

I was hoping that the ecto repo / socket server would be stand-alone and I could minimize coupling in my code. Is my application just better suited to an umbrella app, am I organizing things incorrectly, or is it just configuration issues? Thanks much.

#2

Just running mix phx.server should start everything, the phoenix server and your game state (assuming everything is wired via your application’s supervisor properly). Or if you want the console then iex -S mix phx.server.

I personally don’t like umbrella apps, I prefer dependencies. However, why not just pass in the main Ecto Repo through everything, and if you start Ecto before starting the websocket server then there is no serialization issue.

As for umbrella’s, perhaps others could talk about those, I find them horribly messy, always have since my old erlang days so I’m probably biased. ^.^;

#3

Nothing wrong with having apps called data (structures and integrations with DBs and various storage backends), domain (or something similar that expresses business logic), then probably web and api that are the server-side rendering app, JSON app etc. etc.

Used that several times, it provides a very neat separation of concerns as long as you do explicit dependency management ({my_app, in_umbrella: true}).

Using path dependencies in this case is a very leaky and lame way of doing it. What if I have 5 of my apps in totally different directories both in dev and prod?

#4

Is this the proper relationship? Shouldn’t the phoenix.channel start the supervisor because it needs it to work and not the opposite?

The default configuration for umbrellas is to share all configuration from all projects. I am assuming that you have opted-out from this behaviour explicitly? You could use import_config on both apps and make them import the ecto configuration from state_app but given that you are running into issues, asking if using umbrellas is the way to go is indeed wise.

Umbrellas were designed for apps that share the same dependencies and configuration. So if you want to configure some apps differently or you want to update a certain dependency for one app or not the other, then it is indeed best to drop them.

The good news is that you can still keep a very similar structure. You can keep all projects in the same repository and use :path instead of :in_umbrella for your deps. The main difference is that you are explicitly saying those things are not coupled at the config and dependencies level, so other approaches may feel more natural.

#5

Thanks for your replies, everyone!

I suppose it’s not!

I’m new to elixir and correct organization is part of what I was struggling with. I think I felt like I was “adding phoenix in”, so it felt natural to add it to my supervision tree too. The way it sounds – I should still use my own supervision tree for the core app, but have Phoenix spin it up when it starts. Does this mean that Phoenix’s supervision tree is separate from my app’s?

I haven’t set up an umbrella yet, they all started as separate apps. I went through pragdave’s course, so I was trying to keep them separate.

I wasn’t aware of this function – assuming I keep the current structure, that definitely helps!

So it sounds like I should just start my app’s supervision tree from Phoenix, put the ecto config in the Phoenix config.exs and run everything with mix phx.server as @OvermindDL1 suggested. I see how that cleans things up. I’m still unsure about which modules go in which supervision trees. Is it loosely based on the interfaces/responsibilities in my code? If it’s standalone, it goes it its own tree? Should my Repo go in its own tree, too?

Thanks again, I really appreciate the input.

#6

Each application should have its own supervision tree. Then, if the phoenix app depends on the supervisor app, you need to list the supervisor app as a dependency in your mix.exs file. If you are using umbrella, this is a matter of adding {:supervisor, in_umbrella: true} to deps. By doing this, Elixir will automatically start the supervisor app and its supervision tree before it starts the Phoenix one.

I think it would be better if you actually use the shared configuration suggestion for umbrellas. Run mix phx.new foo --umbrella and you will see how it is designed. But in a nutshell every child application has a line in their mix.exs that points to the umbrella config files. As I said, the goal of the umbrella is to share deps and config, so if you keep them in separate applications, it gives the impression they are not shared while they do. Unifying everything in the config folders of the umbrella will be clearer.

#7

This helps a lot in clarifying how I should structure things. I’ll give the umbrella app a try. Thanks a ton.