Using targets for deps in umbrella project

hectorsq · 2019-03-01 02:53:36 UTC

I am developing an umbrella application with one ecto app and multiple web apps. This application has to run against mssql and postgres.

Currently, I have two deps functions inside each app, one for postgres and the other for mssql.

When I want to change the database, I have to change by hand the call to deps from deps(:postgres) to deps(:mssql) in all my mix.exs files.

I want to use targets so I have changed my deps function to something like this:

  defp deps() do
    [
      {:phoenix, "~> 1.4.0"},
      {:phoenix_pubsub, "~> 1.1"},
      {:phoenix_ecto, "~> 4.0", targets: :postgres},
      {:phoenix_ecto, "~> 3.0", targets: :mssql},
      {:phoenix_html, "~> 2.11"},
      {:phoenix_live_reload, "~> 1.2", only: :dev},
      {:gettext, "~> 0.11"},
      {:jason, "~> 1.0"},
      {:plug_cowboy, "~> 2.0"},
      {:medik_ecto, in_umbrella: true}
    ]
  end

However, I am stuck because I do not how to handle the environment variable when I run MIX_TARGET=postgres mix deps.get from the command line.

hectorsq · 2019-03-01 23:25:48 UTC

I get the following error:

$ MIX_TARGET=postgres mix deps.get
Resolving Hex dependencies...

Failed to use "phoenix_ecto" (version 4.0.0) because
  apps/admin_web/mix.exs requires ~> 3.0
  apps/admin_web/mix.exs requires ~> 4.0

cohawk · 2019-03-02 02:19:30 UTC

I was unaware of and haven’t used targets yet, but taking a quick look it appears in your deps() you should use target: instead of plural targets:?

hectorsq · 2019-03-05 05:02:54 UTC

I tried using target: with no success

cohawk · 2019-03-05 16:20:50 UTC

What version of Elixir are you running? And do you have an example repo online to try and reproduce?

I had a need for this a while ago, but worked around it - so I am kind of blindly making suggestions. I will try to reimplement this afternoon in a project of mine and let you know here what I find.

hectorsq · 2019-03-06 01:46:34 UTC

@cohawk I am using Elixir 1.8.1.

I will create an example repo as soon as possible.

Thank you!

cohawk · 2019-03-06 05:52:12 UTC

Thanks for the reminder - got kinda busy today

No need to create an example repo - it’s easily enough reproducible.

So the digging around the Mix.target PR commit: https://github.com/elixir-lang/elixir/commit/2e576f67a00164d88e2e242e523dc3b5c1a8a5db

targets: is the correct syntax (since you can also pass a list targets: [:postgres, :absinthe]). I think before I was looking at some pre-merged code.

Anyways:

$ mix deps.get --target postgres

is the correct syntax to fetch target deps instead of MIX_TARGET=postgres mix deps.get

And with that it does give us a couple warnings:

warning: the dependency :phoenix_ecto is duplicated at the top level, please remove one of them

But it does appear to pull the correct target version:

$ mix deps.tree --target postgres
warning: the dependency :phoenix_ecto is duplicated at the top level, please remove one of them
warning: the dependency :phoenix_ecto is duplicated at the top level, please remove one of them
phx_targets_test
...
└── phoenix_ecto ~> 4.0 (Hex package)
    ├── ecto ~> 2.2 or ~> 3.0 (Hex package)
    ├── phoenix_html ~> 2.9 (Hex package)
    └── plug ~> 1.0 (Hex package)

and then for the mssql target:

$ mix deps.get --target mssql
warning: the dependency :phoenix_ecto is duplicated at the top level, please remove one of them
Resolving Hex dependencies...
Dependency resolution completed:
Unchanged:
    ...
Downgraded:
  phoenix_ecto 4.0.0 => 3.5.0 RETIRED!
    (invalid)
warning: the dependency :phoenix_ecto is duplicated at the top level, please remove one of them
* Updating phoenix_ecto (Hex package)

and then confirm with the target deps.tree:

$ mix deps.tree --target mssql
warning: the dependency :phoenix_ecto is duplicated at the top level, please remove one of them
warning: the dependency :phoenix_ecto is duplicated at the top level, please remove one of them
phx_targets_test
...
└── phoenix_ecto ~> 3.0 (Hex package)
    ├── ecto ~> 3.0 (Hex package)
    ├── phoenix_html ~> 2.9 (Hex package)
    └── plug ~> 1.0 (Hex package)

Then you execute using the MIX_TARGET variable:

$ MIX_TARGET=mssql mix phx.server

and where I thought it would get murky about remembering the version from which target I had last fetched the deps for, it reminds you at compile time:

$ MIX_TARGET=postgres mix phx.server
...
Unchecked dependencies for environment dev:
* phoenix_ecto (Hex package)
  the dependency does not match the requirement "~> 4.0", got "3.5.0"
** (Mix) Can't continue due to errors on dependencies

Good stuff.

hectorsq · 2019-03-09 03:29:59 UTC

@cohawk, I appreciate a lot your support.

Where is what I’ve done…

I added an IO.inspect(Mix.target()) inside each of my deps functions. I have multiple functions because I am working in an umbrella project.

  defp deps() do
    IO.inspect(Mix.target())
    [
      {:phoenix, "~> 1.4.0"},
      {:phoenix_pubsub, "~> 1.1"},
      {:phoenix_ecto, "~> 4.0", targets: :postgres},
      {:phoenix_ecto, "~> 3.0", targets: :mssql},
      {:phoenix_html, "~> 2.11"},
      {:phoenix_live_reload, "~> 1.2", only: :dev},
      {:gettext, "~> 0.11"},
      {:jason, "~> 1.0"},
      {:plug_cowboy, "~> 2.0"},
      {:medik_ecto, in_umbrella: true}
    ]
  end

Running mix deps.get using an environment variable I get

$ MIX_TARGET=postgres mix deps.get
:postgres
:postgres
:postgres
:postgres
:postgres
:postgres
Resolving Hex dependencies...

or

$ MIX_TARGET=mssql mix deps.get
:mssql
:mssql
:mssql
:mssql
:mssql
:mssql
Resolving Hex dependencies...

However when I pass the target as an option

$ mix deps.get --targets postgres
:host
:host
:host
:host
:host
:host
Resolving Hex dependencies...

In any case I always get the same error:

Failed to use "phoenix_ecto" (version 4.0.0) because
  apps/admin_web/mix.exs requires ~> 3.0
  apps/admin_web/mix.exs requires ~> 4.0
  apps/patients_web/mix.exs requires ~> 4.0

** (Mix) Hex dependency resolution failed, change the version requirements of your dependencies or unlock them (by using mix deps.update or mix deps.unlock). If you are unable to resolve the conflicts you can try overriding with {:dependency, "~> 1.0", override: true}

I will test on a non umbrella project over the weekend.

cohawk · 2019-03-09 04:51:58 UTC

You have targets instead of target - from the CLI you can only setup for a single target.

:host is the default target when you do not set the target flag

You want:

$ mix deps.get --target postgres

hectorsq · 2019-03-09 05:16:25 UTC

$ mix deps.get --target postgres
:host
:host
:host
:host
:host
:host
Resolving Hex dependencies...

$ mix deps.get --target mssql
:host
:host
:host
:host
:host
:host
Resolving Hex dependencies...

cohawk · 2019-03-09 05:54:09 UTC

Yeah, you might be right that the ‘target’ does not propagate through umbrella apps.

Here is my repo from the tests in the above posts - https://github.com/cohawk/phx_targets_test

I can try with an umbrella app sometime this weekend - time permitting.

Cheers

cohawk · 2019-03-09 22:46:45 UTC

I see what you mean with the different results from Mix.target/0 wtih deps.get when passing MIX_TARGET as an environment variable, opposed to an option --target

That should probably be standardized, but:
https://hexdocs.pm/mix/master/Mix.html#module-targets
This feature is considered experimental and may change in future releases.

In any case, yes - it appears that the --target option does not get propagated when running deps.get from an Umbrella project root. When running a targeted deps.get from the child project itself it get even more confused since the Umbrella root holds the mix.lock file.

I’ve updated my example repo to an Umbrella project with some example output, but I am off to do other things now.

I think there would need to be some more dependency resolution logic implemented for targets to work with Umbrella apps.