Mix.Task.run "deps.compile", ["dep_name", "--force"] not compiling

Questions / Help
,
1

I have a dep that might need recompiling when the host app configuration changes. That requires a forced compile since by design deps aren’t automatically recompiled (which is the right strategy).

At the command line I can force recompile fine. But from IEx calling Mix.Task.run/2 I cannot. From the command line:

$ mix deps.compile ex_cldr --force
==> ex_cldr
Compiling 2 files (.erl)
Compiling 31 files (.ex)
Generating Cldr for 6 locales named ["en-001", "it", "pl", "root", "ru", ...] with a default locale named "en-001"
Generated ex_cldr app

From IEx shell:

$ iex -S mix
Erlang/OTP 20 [erts-9.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

iex(1)> Mix.Task.run "deps.compile", [:ex_cldr, "--force"]
:ok

…but no compilation takes place.

Any advice on how I specify the args to Mix.Task.run/2 so that it force compiles would be most appreciated.

2

This seems to be an Elixir bug. When the --force flag is given the compile task for the dependency should be reenabled, by default a task is called only once per project/dependency and in this case it seems to have been called earlier at some point.

You have the same bug in your example, you should call Mix.Task.rerun since the task may have already run when mix was started.

1 Like
3

@ericmj, thanks for your reply. I’ve tried a few combinations without success so far, including reenabling, running and rerunning. All seems to return :ok without actually compiling. Anything else you might suggest>

Interactive Elixir (1.5.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Mix.Task.rerun "deps.compile", ["ex_cldr", "--force"]
:ok
iex(2)> Mix.Task.reenable "deps.compile"                     
:ok
iex(3)> Mix.Task.rerun "deps.compile", ["ex_cldr", "--force"]
:ok
iex(4)> Mix.Task.reenable "deps.compile"                     
:ok
iex(5)> Mix.Task.run "deps.compile", ["ex_cldr", "--force"]  
:ok
4

After looking at this more closely we decided this is a wont fix in Elixir. Mix is not intended to be called from iex or inside of your application. The primary interface is the CLI and because of this tasks are allowed to only run once.

You have to compile the dependency from the command line and restart the application. An application shouldn’t be recompiled after it has started, since it can have new modules, changed environment, or even changed supervision tree. If you don’t want to restart your application it is safest to use the OTP application upgrade mechanisms.

5

Thanks @ericmj. My IEx demo was an example only. My use case is actually in a Mix compiler.

I think that calling a Mix task from a Mix compiler would be a reasonable expectation? I can work around it if tasks aren’t intended to be called at compile time but it does seem a little strange to me.

6

If it’s in a Mix compiler then it’s a different question :). But since you are doing something uncommon by compiling dependencies after the parent project some more details about your problem would be helpful. It is possible we can solve it another way than compiling deps after the parent, for example would it be possible for you to alias the deps.compile or deps.precompile tasks to run your task before deps compilation? Can you elaborate on why the host app config changes in a compiler task?

7

If anyone passing by needs to do something similar, I managed to get a mix task to recompile deps: bonfire-app/lib/mix/tasks/localise at main · bonfire-networks/bonfire-app · GitHub (feedback welcome)

8

I do in fact need to do this, but that link is broken for me @mayel

EDIT: I think I found its new location bonfire_common/lib/mix_tasks/localise/localise_extract.ex at main · bonfire-networks/bonfire_common · GitHub

9

I haven’t looked at this in a while (or tried it with the last Elixir version) but hopefully it can help! There’s also this task: bonfire_common/lib/mix_tasks/extension/deps_compile.ex at main · bonfire-networks/bonfire_common · GitHub

10

Trying to isolate what about this task is different from the standard mix task is proving a bit of a challenge :sob: I feel like surely there must just be some kind of cache I can clear or some piece of state I can modify to make this work. Like before running mix deps.compile ...

11

Yeah I remember feeling the same but ended up with that less than ideal code by trial and error. Thinking about it fresh, I’m not sure why chaining mix deps.clean --build dep1 dep2 wouldn’t do the job? But maybe I’m not remembering some details.

12

It looks like it may not do anything because it doesn’t know about these deps yet?

warning: the dependency spark is not present in the build directory
13

I have a question, why you do not compile the dependencies in another folder and move all ebin file to your project?

for example using: Code.prepend_path()

def prepend_compiled_apps(files_list) do
    files_list
    |> Enum.map(
      &{String.to_atom(&1),
       Path.join(get_build_path() <> "/" <> &1, "ebin") |> Code.prepend_path()}
    )
    |> Enum.filter(fn {_app, status} -> status == {:error, :bad_directory} end)
    |> case do
      [] -> {:ok, :prepend_compiled_apps}
      list -> {:error, :prepend_compiled_apps, :bad_directory, list}
    end
  end

  defp application_ensure({:ok, :prepend_compiled_apps}, app, :add) do
    with {:load, :ok} <- {:load, Application.load(app)},
         {:sure_all_started, {:ok, _apps}} <-
           {:sure_all_started, Application.ensure_all_started(app)} do
      {:ok, :application_ensure}
    else
      {:load, {:error, term}} ->
        {:error, :application_ensure, :load, term}

      {:sure_all_started, {:error, {app, term}}} ->
        {:error, :application_ensure, :sure_all_started, {app, term}}
    end
  end

for updating

  defp application_ensure({:ok, :prepend_compiled_apps}, app, :force_update) do
    Application.stop(app)

    with {:unload, :ok} <- {:unload, Application.unload(app)},
         {:load, :ok} <- {:load, Application.load(app)},
         {:sure_all_started, {:ok, _apps}} <-
           {:sure_all_started, Application.ensure_all_started(app)} do
      {:ok, :application_ensure}
    else
      {:unload, {:error, term}} ->
        {:error, :application_ensure, :unload, term}

      {:load, {:error, term}} ->
        {:error, :application_ensure, :load, term}

      {:sure_all_started, {:error, {app, term}}} ->
        {:error, :application_ensure, :sure_all_started, {app, term}}
    end
  end

for scanning project version

with {:ok, tokens, _} <- :erl_scan.string(String.to_charlist(bin)) do
      :erl_parse.parse_term(tokens)
    end

for phoenix in runtime

If you are using Phoenix as developer mode, please disable live_reload in dev.exs . Please add reloadable_apps: [:mishka_installer] to your endpoint config in config.exs file.

And with Port you can monitor the command line real time! I think it can fix your problem? Am I wrong?

Update

I just do not know, this commit effects on my way or not

14

Was just about to post back here with my current solution which is probably a very very ugly hack, but it does work:

      case Mix.shell().cmd("mix deps.get") do
        0 ->
          Mix.Task.reenable("compile")
          Mix.Task.run("compile")

          Mix.Project.clear_deps_cache()
          Mix.Project.pop()

          "mix.exs"
          |> File.read!()
          |> Code.eval_string([], file: Path.expand("mix.exs"))

          Mix.Task.run("deps.compile", Enum.map(install_list, &to_string/1))

          Mix.Task.reenable("compile")
          Mix.Task.run("compile")

        exit_code ->
          Mix.shell().info("""
          mix deps.get returned exited with code: `#{exit_code}`
          """)
      end
1 Like
15

Thank you @zachdaniel, it is for runtime or need to reset the app? I think you totally change the mix and the phoenix returns alert and you need to do this.

I will try it. it is so much clear than my code it does for runtime

16

Not sure I fully understand the question, sorry.

A brief explanation of what I need to do is: I add a dependency to the mix.exs file, and I need to fetch it and look for a mix task defined by it. So I need to add a dep, get it with mix deps.get and then compile it with mix deps.compile. My use case is quite limited, as I know that the app will shut down after running my mix task, so it’s not really a problem if it ends up in a weird state. But this is happening at runtime, not at compile time (since its in a mix task.

Does that answer your question?

1 Like
17

Thank you for your time and efforts.

My lib downloads hex tar file (and extract it) or get from GitHub and after that I run a Port.open function and run system command(mix deps.get and etc), after building I move all ebin file to my for example build/prod and do all the code I put above.

Another way my lib can, he/she changes the mix file and your code can re-compile in runtime without any downtime? yes?

Another way for my first option I can do your code in diffrent directory with diffrent project and build and move all the ebin to my current project again?

18

I’m sorry, I’m having trouble understanding your question :frowning:

Another way my lib can, he/she changes the mix file and your code can re-compile in runtime without any downtime? yes?

I believe the answer to this question is yes. You can run this to install any deps added to the mix.exs file without downtime. But if the app is currently running I’m not sure if there will be weird behavior from the Mix.Project.pop/1 call. Hard to say.

1 Like
19

Thank you. Aha, this is the answer I need.

So I should re-check it. I do not care about state if lost! because hot coding is very hard for full project, hot coding just useful for one or 2 modules manually and it is very harder in release mode!

Thank you and sorry for my english I am not native, it wasted your time :face_holding_back_tears::pray:t2:

Anyway, Ash means bear in my mother tongue lang :joy:

1 Like