Issue with igniter recompiling deps on the fly

zachdaniel · January 6, 2025, 12:14am

Igniter needs to do some crazy stuff, but one of the craziest things it does is recompiles dependencies, using this bastardized version of mix deps.compile. Anyone familiar with the internals of mix who can help I would be eternally grateful, because this has stumped me so far.

github.com

ash-project/igniter/blob/main/lib/igniter/deps_compile.ex

defmodule Igniter.Util.DepsCompile do
  @moduledoc false

  # Copied from core mix task deps.compile, with small alterations
  # the primary difference is reenabling the mix tasks

  # Compiles dependencies.

  # By default, this task attempts to compile all dependencies.
  # A list of dependencies can be given to compile multiple
  # dependencies in order.

  # This task attempts to detect if the project contains one of
  # the following files and act accordingly:

  #   * `mix.exs` - invokes `mix compile`
  #   * `rebar.config` - invokes `rebar compile`
  #   * `Makefile.win`- invokes `nmake /F Makefile.win` (only on Windows)
  #   * `Makefile` - invokes `gmake` on DragonFlyBSD, FreeBSD, NetBSD, and OpenBSD,
  #     invokes `make` on any other operating system (except on Windows)

This file has been truncated. show original

I’ve honestly stumbled my way through making it work, but for some reason, sometimes (only when I’m using local dependencies in certain configurations) I just get an unceremonious exit that crashes the whole show. It seems to be related to having a dependency that is a path dependency, but I can’t say for sure.

like the entire vm goes down as far as I can tell

To reproduce:

mix archive.install hex phx_new --force
mix archive.install hex igniter_new --force
git clone git@github.com:ash-project/ash.git
git clone git@github.com:ash-project/igniter.git

ash mix igniter.new helpdesk \
  --install ash@path:../ash,ash_authentication_phoenix \
  --with phx.new --verbose --local igniter

and accept y all of the questions

I have beat my head against the wall on this problem for so long that I think I may be literally going a bit crazy about it

Is there anyone here who may be able to help me diagnose this? Or help me find a better way to recompile a dependency “live”?

I know recompiling a dependency comes with some strange behavior regardless (like if the new version of the dependency redefines a module, or no longer defines a module, etc). but honestly I think its “fine” because what we’re doing this for is access to relatively simple scripts, nothing with a runtime, no application tree etc.

zachdaniel · January 6, 2025, 7:27pm

Thanks to some prodding from José I was able to clean up a vast majority of the “jank” in this process and resolve the issue in the meantime

github.com

ash-project/igniter/blob/231ebf7c35f9b260dd41017aa78aa1abf746ac76/lib/igniter/util/install.ex#L142


      
            |> Enum.reduce(igniter, fn {name, task}, igniter ->
              igniter = Igniter.compose_task(igniter, task, argv)
              Mix.shell().info("`#{name}.install` #{IO.ANSI.green()}✔#{IO.ANSI.reset()}")
              igniter
            end)
            |> Igniter.do_or_dry_run(Keyword.put(options, :title, title))
          
            :ok
          end
          
          def get_deps!(igniter, opts) do
            case System.cmd("mix", ["deps.get"], stderr_to_stdout: true) do
              {_output, 0} ->
                igniter =
                  case List.wrap(opts[:update_deps]) do
                    [] ->
                      igniter
          
                    [:all] ->
                      System.cmd("mix", ["deps.update", "--all" | opts[:update_deps_args] || []])
                      %{igniter | rewrite: Rewrite.drop(igniter.rewrite, ["mix.lock"])}