This is a problem I’ve come across now and then… in an app, I have a few directories containing .json files (e.g. for example requests or responses). I put these files into directories inside of test/support/ and then in my mix.exs I reference the test/support/ directory as follows:
def project do
[
# ...
elixirc_paths: elixirc_paths(Mix.env()),
# ...
]
end
# Specifies which paths to compile per environment.
defp elixirc_paths(:test), do: ["lib", "test/support"]
defp elixirc_paths(_), do: ["lib"]
The problem revolves around the __DIR__ variable because it is resolved at compile time. I can include files in my setup fixtures doing something like this:
# example setup inside `test/support/conn_case.ex`
defp append_mutation(%{mutation: mutation} = context) do
data =
"#{__DIR__}/mutations/#{mutation}.json" # <-- resolves to /full/path/to/test/support/mutations/something.json
|> File.read!()
|> Jason.decode!()
%{context | mutation: data}
end
When I run tests locally, the path gets converted to paths that make sense on my local machine, so tests pass. However, if I then try to run tests in a containerized environment (e.g. via docker-compose), the paths are not the same, so tests fail.
One option is to do mix compile --force before running the tests, but that’s slow – I have to recompile everything every time I switch environments.
I seem to remember something about the https://hexdocs.pm/elixir/Application.html#app_dir/1 working in certain cases here… but I don’t see how it’s working its magic. Even with my mix.exs set to compile my test/support/ folder, none of my supporting .json files end up in the _build/ folder, so I don’t think I can reference them there.
Can someone educate me on a better way to dealing with this?
That just specifies that *.ex files in test/support should be compiled into *.beam files on disk.
This does not cause any JSON to be compiled into the modules. They JSON is still loaded during runtime.
The approach I am proposing does the following:
defmodule Foo do
@mutations ~s[mutation1 mutation2 mutation3]a
@json_data @mutations
|> Stream.map(fn mutation -> {mutation, Path.join([__DIR__, "mutations", "#{mutation}.json"]) |> File.read! |> Jason.decode!} end)
|> Map.new
@mutations |> Enum.each(fn {mutation, json} ->
@external_resource Path.join([__DIR__, "mutations", "#{mutation}.json"])
def json_data(unquote(mutation)), do: unquote(json)
end
defp append_mutation(…) do
data = json_data(mutation)
%{context | mutation: data}
end
end
This will only work though if the JSON file is not written to after compilation and if the names are knwon statically. On the other hand side, touching any of the files will cause the module in question to recompile as necessary.
If though the files are subject to change during the test as part of the testrun, you should not store them in the project folder at all, but somewhere in /tmp, making sure they won’t interfere in subsequent testruns.
Yes, __DIR__ will get evaluated at compiletime. But no file access will happen at runtime. The JSON will be parsed at compile time and put as an immediate value into the compiled module.
So instead of having some path in your sources pointing to a JSON file containing {} on the build but not existing on the run time host, you now have a module that not contains that path at all, but instead a function that returns the immediate value %{} when asked for that mutations name.
That looks promising – it resolves to /path/to/my_app/_build/test/lib/my_app/priv/, which in my case is empty.
I see that gettext has some extra files that end up in there (its .po files)… but I don’t see how to get those included… and I can’t find docs as to how mix is supposed to handle this.
From gettext 's mix.exs:
def project do
[
# ...
app: :gettext,
package: hex_package(),
# ...
]
end
def hex_package do
[
maintainers: ["Andrea Leopardi", "José Valim"],
licenses: ["Apache 2.0"],
links: %{"GitHub" => @repo_url},
files: ~w(lib src/gettext_po_parser.yrl mix.exs *.md)
]
end
the .po files are compiled into functions in the module that use Gettext, otp_app: ___ so they don’t need to be in a release.
The behaviour of the priv dir is controlled by the build_embedded: Mix.env() == :prod, line in mix.exs. In development the priv dir is symlinked to the _build directory and in production it is copied. So I would expect your /path/to/my_app/_build/test/lib/my_app/priv/ is a symlink on your dev machine?
For example, for one of my libs:
$ ls -al _build/test/lib/ex_cldr/priv
lrwxr-xr-x 1 kip staff 16 8 Jan 06:09 _build/test/lib/ex_cldr/priv -> ../../../../priv
Interesting. So it looks like it would be possible to put some extra files into the priv/ directory (in my case .json files), but that feels a bit odd because they are only relevant to testing…
So, as they are only relevant to testing, and during testing usually your full sources are available from where you run mix, why not just use relative path from CWD?
I would be very unusually to put test files in priv, yes. For accessing support files in tests I typically just rely on the fact that tests are being run when the working directory is the root of the project. Therefore just "test/support/mutations/#{mutation}.json"
Yes, thats my normal strategy. It breaks if you have tests that change working directory - and there are some functions in Mix that do that if you are messing around with dependencies (which is not something I’d recommend and wouldn’t normally be an issue).
