Phoenix.PubSub in an umbrella app -- Phoenix in Action 1.4

I’m working my way through “Phoenix in Action 1.4” for the second time. The instructions have you create an umbrella app with two apps: auction and auction_web, and following the instructions I created the auction_web app with --no-ecto. Unfortunately, --no-ecto seems to be the root cause of continuing PubSub warnings/errors. I was successful squelching the PubSub warnings/errors for a few days while I was working on getting the auction_web app to display a list of items on a web page, but then I moved inside the auction app to create a Postgres database, and when I tried to execute iex -S mix, then PubSub, like a Phoenix, rose from the ashes to bite me again:

...phoenix_apps/second_auction/auction_umbrella $ iex -S mix
...
...
...
> 14:25:30.946 [notice] Application eex exited: :stopped
> ** (Mix) Could not start application auction: Auction.Application.start(:normal, []) returned an error: shutdown: failed to start child: Phoenix.PubSub.Supervisor
>     ** (EXIT) shutdown: failed to start child: Phoenix.PubSub.PG2
>         ** (EXIT) shutdown: failed to start child: Auction.PubSub.Adapter
>             ** (EXIT) exited in: :gen_server.call(Phoenix.PubSub, {:join_local, Auction.PubSub.Adapter, #PID<0.4916.0>}, :infinity)
>                 ** (EXIT) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started

Am I in the wrong directory? If I change into apps/auction/ and try again, I get:

auction_umbrella/apps/auction$ iex -S mix
Generated auction app
15:13:27.742 [notice] Application auction exited: exited in: Auction.Application.start(:normal, [])
    ** (EXIT) an exception was raised:
        ** (ArgumentError) The module Phoenix.PubSub was given as a child to a supervisor but it does not exist
            (elixir 1.16.0-rc.1) lib/supervisor.ex:797: Supervisor.init_child/1
            (elixir 1.16.0-rc.1) lib/enum.ex:1700: Enum."-map/2-lists^map/1-1-"/2
            (elixir 1.16.0-rc.1) lib/supervisor.ex:783: Supervisor.init/2
            (elixir 1.16.0-rc.1) lib/supervisor.ex:707: Supervisor.start_link/2
            (kernel 9.1) application_master.erl:293: :application_master.start_it_old/4
15:13:27.747 [notice] Application ecto_sql exited: :stopped
15:13:27.747 [notice] Application postgrex exited: :stopped
15:13:27.747 [notice] Application db_connection exited: :stopped
15:13:27.747 [notice] Application ecto exited: :stopped
15:13:27.748 [notice] Application decimal exited: :stopped
15:13:27.748 [notice] Application telemetry exited: :stopped
15:13:27.748 [notice] Application eex exited: :stopped
** (Mix) Could not start application auction: exited in: Auction.Application.start(:normal, [])
    ** (EXIT) an exception was raised:
        ** (ArgumentError) The module Phoenix.PubSub was given as a child to a supervisor but it does not exist
            (elixir 1.16.0-rc.1) lib/supervisor.ex:797: Supervisor.init_child/1
            (elixir 1.16.0-rc.1) lib/enum.ex:1700: Enum."-map/2-lists^map/1-1-"/2
            (elixir 1.16.0-rc.1) lib/supervisor.ex:783: Supervisor.init/2
            (elixir 1.16.0-rc.1) lib/supervisor.ex:707: Supervisor.start_link/2
            (kernel 9.1) application_master.erl:293: :application_master.start_it_old/4

And, if I switch into apps/auction_web/ I get:

auction_umbrella/apps/auction_web$ iex -S mix
Erlang/OTP 26 [erts-14.1.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit]

Interactive Elixir (1.16.0-rc.1) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> 

I’m using:

Phoenix 1.7.10
Erlang/OTP 26.1.2
Elixir 1.16.0-rc.1-otp-26

Here are all the places where the name PubSub occurs in my umbrella app:

  1. apps/auction/lib/auction/application.ex:
defmodule Auction.Application do
  # See https://hexdocs.pm/elixir/Application.html
  # for more information on OTP Applications
  @moduledoc false

  use Application

  @impl true
  def start(_type, _args) do
    children = [
      # Starts a worker by calling: Auction.Worker.start_link(arg)
      # {Auction.Worker, arg}
      {Phoenix.PubSub, name: Auction.PubSub},
      {Auction.Repo, []}
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: Auction.Supervisor]
    Supervisor.start_link(children, opts)
  end
end
  1. apps/auction_web/lib/auction_web/application.ex:
defmodule AuctionWeb.Application do
  # See https://hexdocs.pm/elixir/Application.html
  # for more information on OTP Applications
  @moduledoc false

  use Application

  @impl true
  def start(_type, _args) do
    children = [
      {Phoenix.PubSub, name: AuctionWeb.PubSub},
      AuctionWeb.Telemetry,
      # Start a worker by calling: AuctionWeb.Worker.start_link(arg)
      # {AuctionWeb.Worker, arg},
      # Start to serve requests, typically the last entry
      AuctionWeb.Endpoint
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: AuctionWeb.Supervisor]
    Supervisor.start_link(children, opts)
  end

  # Tell Phoenix to update the endpoint configuration
  # whenever the application is updated.
  @impl true
  def config_change(changed, _new, removed) do
    AuctionWeb.Endpoint.config_change(changed, removed)
    :ok
  end
end
  1. auction_umbrella/config/config.exs:
# This file is responsible for configuring your umbrella
# and **all applications** and their dependencies with the
# help of the Config module.
#
# Note that all applications in your umbrella share the
# same configuration and dependencies, which is why they
# all use the same configuration file. If you want different
# configurations or dependencies per app, it is best to
# move said applications out of the umbrella.
import Config

# Stuff I added:

config :auction, ecto_repos: [Auction.Repo]

config :auction, Auction.Repo,
  database: "auction",
  username: "7stud",
  password: "",
  hostname: "localhost",
  port: "5432"

##############################

config :auction_web,
  generators: [context_app: false]

# Configures the endpoint
config :auction_web, AuctionWeb.Endpoint,
  url: [host: "localhost"],
  adapter: Phoenix.Endpoint.Cowboy2Adapter,
  render_errors: [
    formats: [html: AuctionWeb.ErrorHTML, json: AuctionWeb.ErrorJSON],
    layout: false
  ],
  pubsub_server: AuctionWeb.PubSub,
  live_view: [signing_salt: "cOmLEVHn"]

# Configure esbuild (the version is required)
config :esbuild,
  version: "0.17.11",
  default: [
    args:
      ~w(js/app.js --bundle --target=es2017 --outdir=../priv/static/assets --external:/fonts/* --external:/images/*),
    cd: Path.expand("../apps/auction_web/assets", __DIR__),
    env: %{"NODE_PATH" => Path.expand("../deps", __DIR__)}
  ]

# Configure tailwind (the version is required)
config :tailwind,
  version: "3.3.2",
  default: [
    args: ~w(
      --config=tailwind.config.js
      --input=css/app.css
      --output=../priv/static/assets/app.css
    ),
    cd: Path.expand("../apps/auction_web/assets", __DIR__)
  ]

# Sample configuration:
#
#     config :logger, :console,
#       level: :info,
#       format: "$date $time [$level] $metadata$message\n",
#       metadata: [:user_id]
#
import Config

# Configures Elixir's Logger
config :logger, :console,
  format: "$time $metadata[$level] $message\n",
  metadata: [:request_id]

# Use Jason for JSON parsing in Phoenix
config :phoenix, :json_library, Jason

# Import environment specific config. This must remain at the bottom
# of this file so it overrides the configuration defined above.
import_config "#{config_env()}.exs"

What was involved in that? It shouldn’t have needed any “squelching”…

In particular, did you change anything in mix.exs like applications? That list is maintained automatically nowadays, but you’ll still see a lot of old advice to fiddle with it and it can flat-out break your app if misconfigured.


This error is coming from PG2Worker, which is trying to connect to a pg process in pg_join:

The message suggests that the application supervisor for phoenix_pubsub didn’t start correctly, since that’s what makes the pg process:


On the other hand, this error suggests that running the app in apps/auction isn’t loading Phoenix at all. :thinking:


A general tip: having separate PubSub instances is explicitly against the design goal of the Phoenix pubsub system. The idea is that code in the “not-web” application can’t explicitly call code in the “web” application (because the dependencies don’t point that way), but it can push messages into pubsub that are handled by the “web” application.

1 Like

Do you know of a blog post that summarizes this? I have the same observation as you.

Past me FTW :smile:

Thanks for taking the time to look at my question.

In apps/auction_web/lib/auction_web/application.ex, I had to add {Phoenix.PubSub, name: AuctionWeb.PubSub}, and somewhere I read that it has to be above AuctionWeb.Endpoint :

 def start(_type, _args) do
    children = [
      {Phoenix.PubSub, name: AuctionWeb.PubSub},
      AuctionWeb.Telemetry,
      # Start a worker by calling: AuctionWeb.Worker.start_link(arg)
      # {AuctionWeb.Worker, arg},
      # Start to serve requests, typically the last entry
      AuctionWeb.Endpoint
    ] 

There are other posts I found about having to add that, so I think that is correct.

In particular, did you change anything in mix.exs like applications?

No, I did not add anything to the application function in any of the mix.exs files. Of course, I did add things to the deps() functions.

The good news is that I started over in another directory, and I recreated the umbrella app, and now everything is working without any errors/warnings. The only difference I can discern is that my apps/auction/lib/auction/application.ex file doesn’t have {Phoenix.PubSub, name: Auction.PubSub} in it:

defmodule Auction.Application do
  # See https://hexdocs.pm/elixir/Application.html
  # for more information on OTP Applications
  @moduledoc false

  use Application

  @impl true
  def start(_type, _args) do
    children = [
      # Starts a worker by calling: Auction.Worker.start_link(arg)
      # {Auction.Worker, arg}
      {Auction.Repo, []}
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: Auction.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

I can’t remember whether I added that PubSub line when I was trying something, or whether that PubSub line was generated by one of the mix commands that I used. In any case, when I recreated auction_umbrella there was no PubSub line in that file. Now, I can start the server in the directory /auction_umbrella without any errors or warnings, and I can run iex -S mix in the apps/auction directory and in the apps/auction_web directory and in the auction_umbrella/ directory, and I don’t get any errors. I can also query the database in iex.

Onwards.

Thanks for your help.

For whatever it’s worth, the default umbrella generator puts the PubSub in the other place:

Here are the new commands that I ran:

phoenix_apps/third_auction$ mix new --umbrella auction_umbrella
third_auction/auction_umbrella/apps$ mix new auction --sup
third_auction/auction_umbrella/apps$ mix phx.new.web auction_web --no-ecto --no-mailer

The new auction_umbrella command only created these files:

$ mix new --umbrella auction_umbrella
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating apps
* creating config
* creating config/config.exs

The next two commands are what created the application.ex files for each app. When I created my latest auction_umbrella app, I added the --no-mailer flag in an attempt to quiet the Swoosh warnings that I got in my previous auction_umbrella app, and that has worked so far. In my previous auction_umbrella app, I solved the Swoosh warnings by adding swoosh (and hackney) to my deps in mix.exs in the auction_web app:

  {:swoosh, "~> 1.14.3"},
  {:hackney, "~> 1.20.1"},

Edit: In a new directory, I just ran the three new commands listed above in succession, and I can confirm that PubSub does not appear in either app’s application.ex file, e.g. apps/auction/lib/auction/application.ex. The only place that PubSub appears in the whole auction_umbrella app is in auction_umbrella/config/config.ex:

import Config

config :auction_web,
  generators: [context_app: false]

# Configures the endpoint
config :auction_web, AuctionWeb.Endpoint,
  url: [host: "localhost"],
  adapter: Phoenix.Endpoint.Cowboy2Adapter,
  render_errors: [
    formats: [html: AuctionWeb.ErrorHTML, json: AuctionWeb.ErrorJSON],
    layout: false
  ],
  pubsub_server: AuctionWeb.PubSub,
  live_view: [signing_salt: "qR/Mri3l"]

In my opinion, there is a bug in one of the new commands because it generates a group of files that causes errors. The new command:

mix new --umbrella auction_umbrella

created the auction_umbrella/config/config.exs file before the auction_web app was even created, yet there is stuff about AuctionWeb in there. That means the mix command to create the auction_web app must have written to config.exs and configured its endpoint AuctionWeb.Endpoint. Yet, mix did not add AuctionWeb.PubSub to the list of applications that need to be started before the auction_web app, which are specified in the auction_web/lib/auction_web/application.ex file:

defmodule AuctionWeb.Application do
  # See https://hexdocs.pm/elixir/Application.html
  # for more information on OTP Applications
  @moduledoc false

  use Application

  @impl true
  def start(_type, _args) do
    children = [
      {Phoenix.PubSub, name: AuctionWeb.PubSub},  # I HAD TO ADD THIS LINE
      AuctionWeb.Telemetry,
      # Start a worker by calling: AuctionWeb.Worker.start_link(arg)
      # {AuctionWeb.Worker, arg},
      # Start to serve requests, typically the last entry
      AuctionWeb.Endpoint
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: AuctionWeb.Supervisor]
    Supervisor.start_link(children, opts)
  end

...
...

I would guess the bug is in the command:

mix phx.new.web auction_web --no-ecto --no-mailer