How to set up Phoenix live code reloading in a not-quite-umbrella?

Phoenix Forum Questions / Help
, , ,
1

I am working on a project that is almost, but not quite, an umbrella application. Rather, it contains many mix-projects in subfolders of the apps/ folder, with multiple Mix projects depending on varying groups of these in subfolders of the rels/ folder.

A minimal example setup can be seen in this GitHub repo

The commands to create it were:

$ mkdir apps/
$ mkdir rels/

$ cd apps/
$ mix phx.new phoenix_app --no-ecto
$ cd ../rels
$ mix new main_app

Now edit rels/main_app/mix.exs to include the following in the deps() section:

  defp deps do
      {:phoenix_app, path: "../../apps/phoenix_app", env: Mix.env()}
  end

And edit rels/main_app/config/config.exs to include the Phoenix-related configuration of the dependency:

import_config "../../../apps/phoenix_app/config/config.exs"

This is the bare minimum to get Phoenix running in development (as well as in production) with this not-quite-umbrella setup, using iex -S mix phx.server from the main_app folder.

However, there are two gotcha’s:

  • Live code reloading is not working. Making a change to one of the files in apps/phoenix_app/lib/phoenix_app_web/templates/ for instance does not auto-refresh the page. Even manually refreshing the page will not pick up this change.
  • Webpack works, but only if someone went to apps/phoenix_app/assets and ran the npm install && node node_modules/webpack/bin/webpack.js --mode development there. That will work fine for the person who first set up the applications (because Phoenix asks you if you want to ‘fetch and install dependencies now’). But when another contributor fetches the application from Git later, they won’t have the same luck. They need to manually run those commands. They do get a warning when starting up the app to run cd assets && npm install, but this will not point to the path the Phoenix app itself is installed in, and thus be confusing.

I am trying to resolve these two issues.
What configuration am I missing to make this work without issue?
Feel free to look at and try things out with the minimal example application yourself :slight_smile:.

Thanks!~

1 Like
2

You would need to change Phoenix’ code reloader to look at path deps instead of umbrella children.

3

Thank you for your reply!

It is not currently running in an umbrella, i.e. the Phoenix app was not created using the --umbrella setting, and there is no in_umbrella: true in the deps().

In the configuration (apps/phoenix_app/config/dev.exs) the code reloader responds to the following regexps:

config :phoenix_app, PhoenixAppWeb.Endpoint,
  live_reload: [
    patterns: [
      ~r"priv/static/.*(js|css|png|jpeg|jpg|gif|svg)$",
      ~r"priv/gettext/.*(po)$",
      ~r"lib/phoenix_app_web/{live,views}/.*(ex)$",
      ~r"lib/phoenix_app_web/templates/.*(eex)$"
    ]
  ]

These seem to match any paths with the given suffixes, regardless of what appears in front (since the regular expression patterns do not start with a ^). Alas, that does not seem to work.

So what setting needs to be changed here exactly?

4

You might want to look at the reloadable_apps configuration of the CodeReloader: https://hexdocs.pm/phoenix/Phoenix.CodeReloader.html#reload!/1

(also make sure that you’re making a distinction in your head between the CodeReloader, which compiles and loads new versions of the beam files, and the phoenix_live_reload which triggers page refreshes when files change).

You might also want to take a look at exsync (which I am a maintainer of). You could probably use the reload_callback to trigger a page refresh:

Exsync will reload all path dependencies by default, unlike Phoenix.CodeReloader

4 Likes
5

That’s my point. Phoenix Code Reloader can do what you are asking for umbrellas, but not for path dependencies. Someone would have to generalize the existing code.

3 Likes
6

Thanks for bringing this up, and for the link to exsync. This has been an annoyance while developing the Oban ui for a long time now—it is an isolated set of live views and needs to be “hosted” in a proper Phoenix app. I resorted to symlinking to get reloading to work for modules, though “live reload” hasn’t worked at all.

I would imagine that the Phoenix live dashboard has similar development constraints.

1 Like
7

@sorentwo we start a single file Phoenix application for the dashboard and we configured it with code reloading, so things work fine there: https://github.com/phoenixframework/phoenix_live_dashboard/blob/master/dev.exs

1 Like
8

Thank you, it was not immediately clear to me that you expected changes to the code itself to be required.

After looking at the implementation, especially here(where the Endpoint’s :reloadable_apps configuration is read) and here (where the default fallback is set, that has a special case for an umbrella application) I figured out that manually setting this configuration to [:phoenix_app] (e.g. the name of your OTP application that runs phoenix) will make the code reloader work:

config :phoenix_app, PhoenixAppWeb.Endpoint,
  # ... other settings here,
  # ... with at the end:
  reloadable_apps: [:phoenix_app]

(See also this commit on the example repository.)

This does not do the live code reloading yet (and I’ll investigate further to figure that out), but at least it will load the new code after a page refresh, which might be good enough for some applications.

2 Likes
9

Of course you did https://www.youtube.com/watch?v=TslkdT3PfKc

Great idea! I did something very similar for test mode, but I neglected to try it for dev. So much to learn from phoenix_live_dashboard :yellow_heart:

2 Likes
10

Edit:
I posted what may be a better solution in a follow-up post, so you might want to skip this one and read the follow-up first.

I have been suffering from this annoyance as well.

Today I decided to try and figure out if I could do something better and tinkered with the phoenix_live_reload pakcaage in my _deps to see what is receives.

My setup is similar to the one described in the post, including various applications that depend on one another. This includes core apps, a phoenix front-end, APIs, and meta-apps that compose them into releases.

Often, when developing the Phoenix-based app I would run it directly in order to get live reload working.
However, it was an incomplete compromise, as I wanted some of the APIs that live in other apps, or the entire distribution, to be running as well.

For the sake of example, I have 3 apps:
:my_app_core (MyApp.Core namespace), :my_app_phoenix (MyApp.Phoenix namespace) that is the Phoenix UI, and :my_app_app1 (MyApp.App1 namespace), that is intended for distribution.

app1 depends on phoenix, and phoenix depends on core.

I wanted to get live-reload when running app1.

When running :my_app_phoenix directly, the live_reload socket would see file changes in the form of
lib/my_app_phoenix/components/layout_components/navbar.heex, the configured reload patterns would match and the pages would reload.

When runnign app1:
The dependencies between the apps are library path deps (i.e, {:my_app_phoenix, path: "../my_app_phoenix"} and it’s not an umbrella app.
This means that in the scope of my_app_app1, changes to my_app_phoenix get compiled into app1’s _build directory, and that is what live_reload reported when checking for file changes.

For example, /Users/<snip>/my_app_app1/_build/dev/lib/my_app_phoenix/ebin/Elixir.MyApp.Phoenix.Components.LayoutComponents.beam, for the module that contains the template in the previous example.

Therefore, I changed the patterns in my_app_app1’s dev config to include:

  ~r"my_app_phoenix/ebin/Elixir.MyApp.Phoenix.(Components|Layout|Views).*.beam$",

and now changes to templates that get compiled into those modules, or those modules themselves, are causing the page to reload. The appropriate pattern depends on the app’s module structure, of course.

One thing to note is that those are full-page reloads, and that live_reload may be able to do more subtle refreshes to the UI. The live_reload socket had reported {"asset_type": "heex"} when I ran my_app_phoenix directly and the changed to the template file was directly picked up, and {"asset_type": "beam"} when I ran my_app_app1 and it picked up the change to the compiled module.

I didn’t test this thoroughly yet, and may need to enhance my setup later, but as a first step this is a significant improvement for me.

11

After a bit more exploration, I noticed that (as I suspected), changes to css were not being picked up.

I dug some more and looked at live_reloader’s application. It has a dirs option, which is used for setting up filesystem watches.

I configured it to watch the my_app_phoenix app directory, and commented out the beam watch for now.

Reloading assets (js, css), as well as file chagnes seems to work as it does when running my_app_phoenix directly.

Here is the current config I’m using:

config :my_app_phoenix, MyApp.Phoenix.Endpoint,
  live_reload: [
    patterns: [
      ~r"my_app_phoenix/priv/static/.*(js|css|png|jpeg|jpg|gif|svg)$",
      ~r"my_app_phoenix/priv/gettext/.*(po)$",
      ~r"my_app_phoenix/lib/my_app_phoenix/(live|views|components)/.*(ex)$",
      ~r"my_app_phoenix/lib/my_app_phoenix/templates/.*(eex)$",
      # ~r"my_app_phoenix/ebin/Elixir.MyApp.Phoenix.(Components|Layout|Views).*.beam$",
    ]
  ],
  reloadable_apps: [:my_app_phoenix]

config :phoenix_live_reload, :dirs, ["", "../my_app_phoenix"]

The :phoenix_live_reload config causes live_reload to watche both my_app_app1 and my_app_phoenix directories for changes. I may remove the "" value to watch only the phoenix app.
I also suspect that removing the beam watches may cause a race condition where changes to a file would cause a page reload before the file is able to complete its compilation, such that the old version would be rendered on reload, but so far, the browser was displaying the desired results.

2 Likes