Start Cache/Load Data in APP.Application.start

About applications:

See https://hexdocs.pm/elixir/Application.html

for more information on OTP Applications

Straightforward code to show what I did.

defmodule APP.Core.Application do
  # See https://hexdocs.pm/elixir/Application.html
  # for more information on OTP Applications
  @moduledoc false

  use Application

  alias Bargain.Cache

  def start(_type, _args) do
    import Supervisor.Spec, warn: false
    # List all child processes to be supervised
    children = [
      supervisor(ConCache, [[], [name: :cache]])
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: APP.Core.Supervisor]

    Supervisor.start_link(children, opts)
    |> post_start()
  end

  defp post_start(callback) do
    Cache.start()

    callback
  end
end

The function post_start/1 is a workaround, because this will not be load in case of Cache failure by supervisor.
Elixir have a solution for this?

Or I can simple implement:

Ex:
in my get_cache method, something like this: (like an CDN when load data for first time and cache it)

case ConCache.get(cache, :data) do
  nil -> ConCache.set(cache, :data, get_data())
  data -> data
end
1 Like

If I understand your problem correctly, you can add a task to start Cache before supervisor(ConCache, [[], [name: :cache]]).

children = [
  worker(Task, [fn -> Cache.start() end], restart: :transient),
  supervisor(ConCache, [[], [name: :cache]])
]

or maybe just add Cache before ConCache

children = [
  worker(Cache, []), # would call Cache.start_link
  supervisor(ConCache, [[], [name: :cache]])
]

Don’t know if it’s of any help, but in a small project where I used ETS tables to cache some data in Postgres, I populated the cache with a task in the app which dealt with Postgres.

defmodule Datastore.Application do
  @moduledoc false
  use Application

  def start(_type, _args) do
    import Supervisor.Spec

    children = [
      Datastore.Repo,
      worker(Task, [&populate_usernames_cache!/0], restart: :transient)
    ]

    opts = [strategy: :one_for_one, name: Datastore.Supervisor]
    Supervisor.start_link(children, opts)
  end

  @spec populate_usernames_cache! :: true | no_return
  defp populate_usernames_cache! do
    Datastore.Users.get_all_usernames() |> Cache.Usernames.put_many!()
  end
end
5 Likes

TL;DR: Try to keep your start up encoded as much as possible with the standard facilities of sub-supervisors, children (the order of them) as well as the restart strategy and you’ll guarantee that your system looks like any other. If ConCache is actually doing logic, don’t make it a supervisor. Catastrophic failure should be encoded in the init to prevent the system/supervisor and the rest of the children from starting.

A bit of meditation on the things in this post, written by the great Fred Hebert: It’s About the Guarantees

I think maybe more information about everything is in order.

Ordinarily, a supervisor wouldn’t be a process that actually does logic, so I’m weary of this being started and called :cache in your supervision tree.

What is it that can fail when starting the supervisor and why do we need to start all our children before post_start? Usually when we require a specific ordering of started processes we can simply put them before/after other processes in the children list. A good model is to have processes that are tied to each other in functionality start under the same supervisor with their relationship encoded in the restart strategy. This might not be needed here, but it’s worth looking at in the future when your supervision tree grows.

Encoding catastrophic failure into your startup is best done by setting up whatever requirements you have in a process’ init (that would be the ConCache.init call here). If there is something that simply will not allow for your application to start without, put it there and the BEAM will make sure your app just doesn’t start if this condition isn’t met.

If a process is needed for ConCache to be running, put its startup before ConCache and that will have to start up properly for ConCache to be started. Conversely, if ConCache needs to be running for something else to function, putting it before it in the children list will ensure that it needs to start properly for the other thing to start. This requires you to bail out if things aren’t actually working as they should, so detection of that property is needed in the init of the process.

This applies to all processes, so it’s good to think of your startup process as potentially containing information about what’s important. If something is pivotal to the whole system and it simply can’t run without it, put that first in your children list, though you’ll have to have the dependencies it has start before it, obviously.

With regards to general cache behavior, I favor the idea of checking a cache for data and if it doesn’t have it, populate the cache with that data.

Edit: If you need to pre-populate the cache you have to make the choice whether or not to make that hold up your startup (if the cache is important enough) or delay that by, for example, sending yourself a message in the init callback in order to load the cache data in a handle_info. Both behaviors will likely die on error, so if the cache is important enough you very likely want it to die on init instead of later.

3 Likes