Including data files in a Distillery release

I’m trying to build a release of a project I’m working on which has several CSV, JSON and ETS files. There files are used at runtime to import ETS tables and at compile-time to dynamically generate certain modules. However, Distillery does not seem to include them in the release.

At the beginning I had them in a directory inside the Elixir project (alongside lib). I also tried moving the files inside the lib dir (and changing the config accordingly) but the results were the same.

I’ve been reading the docs but don’t find anything about this. There’s the following reference:

  • code_paths (list of strings);
    a list of additional code paths to use when searching
    for applications/modules

but it does not seem to be related to extra needed files.

2 Likes

The traditional place to put non-code resources that are needed at runtime is the priv folder. All the tools are aware of this convention and preserve proper paths.

You can access the files at runtime using Application.app_dir(app_name, "priv/path/to/file")

11 Likes

Thank you @michalmuskala. I’m going to try this and report back!

EDIT: Michal’s answer does work. Marking it as the solution.

1 Like

You can use :code.priv_dir(app) to get the priv directory directly as well, it’s what I typically use in my applications.

3 Likes

How can I access the priv path in dev.exs file? When I do it as below:

"#{Application.app_dir(:app)}/priv/ssl/ca.pem"

I got:

** (Mix.Config.LoadError) could not load config config/dev.exs
    ** (ArgumentError) unknown application: :app
    (elixir) lib/application.ex:428: Application.app_dir/1
    (stdlib) erl_eval.erl:670: :erl_eval.do_apply/6
    (stdlib) erl_eval.erl:269: :erl_eval.expr/5
    (stdlib) eval_bits.erl:81: :eval_bits.eval_field/3
    (stdlib) eval_bits.erl:65: :eval_bits.expr_grp/4
    (stdlib) erl_eval.erl:474: :erl_eval.expr/5
    (stdlib) erl_eval.erl:878: :erl_eval.expr_list/6
    (stdlib) erl_eval.erl:236: :erl_eval.expr/5

I’m setting ssl_opts for Ecto, putting those pem files under priv/ssl, but the application can’t access them after building with Distillery.

Quick question about this: Does this directory stay the same for all releases, or will each new version have its own priv_dir?

As long as you install each release into the same directory then the priv directory ‘should’ always remain the same (unless using a custom OTP build or so).

New versions of your application will have a slightly different path (as the path includes the version of your app in a release), but the same version will always be the same path, regardless of OTP version, Elixir version, etc.

1 Like

Thank you. This has tripped me off before. I stored some general app-state-information in a file (because the database would be ridiculously overkill for this), but it turned out that the file was not found after upgrading my release, with the app thus restarting with a blank slate every time after an upgrade.

Yeah my general advice along these lines is to use priv for anything version-specific and/or static data/assets and to rely on something more well-known of your own creation in the filesystem for stateful data, i.e. /var/myapp/. You could also use the var directory created by the release, which you can obtain via the RELEASE_MUTABLE_DIR environment variable exposed to the release (which is used for temporary files like pipes, generated configs, and log files). That said, I would still tend towards using a directory like /var/myapp myself, since you can avoid clutter and manage it’s permissions and so on separately.

1 Like

I’m having the same problem as @chensan I’m trying to set some ssl_opts in a config file and it says:

** (ArgumentError) unknown application: :app
    (elixir) lib/application.ex:654: Application.app_dir/1
    (elixir) lib/application.ex:681: Application.app_dir/2
    (stdlib) erl_eval.erl:680: :erl_eval.do_apply/6
    (stdlib) erl_eval.erl:888: :erl_eval.expr_list/6
    (stdlib) erl_eval.erl:240: :erl_eval.expr/5
    (stdlib) erl_eval.erl:232: :erl_eval.expr/5
    (stdlib) erl_eval.erl:888: :erl_eval.expr_list/6
    (stdlib) erl_eval.erl:240: :erl_eval.expr/5

The app is named :app in mix.exs

Config files are probably evaluated before your app is loaded.

You can set ssl opts in you application’s start callback, probably.

Thanks, I ended doing this:

defmodule App.Repo do
  use Ecto.Repo, otp_app: :app

  @doc """
  Dynamically loads the repository url from the
  DATABASE_URL environment variable.
  """
  def init(_, opts) do
    opts =
      opts
      |> put_env(:ssl_opts)
      |> put_env(:url)
      |> put_env(:pool_size)

    {:ok, opts}
  end

  defp put_env(opts, key) do
    case load_env(key) do
      nil -> opts
      val -> Keyword.put(opts, key, val)
    end
  end

  defp load_env(:ssl_opts) do
    [cacertfile: "#{:code.priv_dir(:app)}/rds-combined-ca-bundle.pem"]
  end

  defp load_env(:url), do: System.get_env("DATABASE_URL")
  defp load_env(:pool_size), do: String.to_integer(System.get_env("POOL_SIZE"))
end