gpb dependency not fully compiled when migrating to Elixir 15 and OTP 26

Questions / Help
1

Hello!

I am currently working on upgrading the Elixir and Erlang versions of an app from:

elixir 1.12.2
erlang 24.2.1

to:

elixir 1.15.6-otp-26
erlang 26.1.1

I’m using asdf as a dependency manager and I configured the new versions to be set at a local level, which creted a .tool-versions file.

Using the older versions, I was able to compile the following dependency:

defp deps do
    [
      # ...
      {:gpb, "~> 4.17", override: true},
      # ...
    ]
  end

As a result of compiling the dependency, its ./deps/gpb/ebin folder had these files:

-rw-r--r--  1 X  staff     735 Oct  3 14:36 gpb.app
-rw-r--r--  1 X  staff   86292 Oct  3 14:36 gpb.beam
-rw-r--r--  1 X  staff   39024 Oct  3 14:36 gpb_analyzer.beam
-rw-r--r--  1 X  staff   27368 Oct  3 14:36 gpb_codegen.beam
-rw-r--r--  1 X  staff   50508 Oct  3 14:36 gpb_codemorpher.beam
-rw-r--r--  1 X  staff  131500 Oct  3 14:36 gpb_compile.beam
-rw-r--r--  1 X  staff   37764 Oct  3 14:36 gpb_compile_descr.beam
-rw-r--r--  1 X  staff   17356 Oct  3 14:36 gpb_decoders_lib.beam
-rw-r--r--  1 X  staff   87360 Oct  3 14:36 gpb_defs.beam
-rw-r--r--  1 X  staff  438812 Oct  3 14:36 gpb_descriptor.beam
-rw-r--r--  1 X  staff   82868 Oct  3 14:36 gpb_gen_decoders.beam
-rw-r--r--  1 X  staff   62812 Oct  3 14:36 gpb_gen_encoders.beam
-rw-r--r--  1 X  staff   53124 Oct  3 14:36 gpb_gen_introspect.beam
-rw-r--r--  1 X  staff  101252 Oct  3 14:36 gpb_gen_json_decoders.beam
-rw-r--r--  1 X  staff   71140 Oct  3 14:36 gpb_gen_json_encoders.beam
-rw-r--r--  1 X  staff   33968 Oct  3 14:36 gpb_gen_mergers.beam
-rw-r--r--  1 X  staff   92652 Oct  3 14:36 gpb_gen_nif.beam
-rw-r--r--  1 X  staff   36428 Oct  3 14:36 gpb_gen_translators.beam
-rw-r--r--  1 X  staff   35784 Oct  3 14:36 gpb_gen_types.beam
-rw-r--r--  1 X  staff   47320 Oct  3 14:36 gpb_gen_verifiers.beam
-rw-r--r--  1 X  staff   45952 Oct  3 14:36 gpb_lib.beam
-rw-r--r--  1 X  staff   35376 Oct  3 14:36 gpb_names.beam
-rw-r--r--  1 X  staff   42292 Oct  3 14:36 gpb_parse.beam
-rw-r--r--  1 X  staff  197944 Oct  3 14:36 gpb_parse_old.beam
-rw-r--r--  1 X  staff   22632 Oct  3 14:36 gpb_scan.beam
-rw-r--r--  1 X  staff  202588 Oct  3 14:36 gpb_scan_old.beam

However, after upgrading Erlang and Elixir to the new versions and gpb to 4.20, the result of the dependency compilation ended up being different. The dependency is still compiling, but the ./deps/gpb/ebin folder now contains less files:

-rw-r--r--  1 X  staff   37252 Oct  3 15:05 gpb_compile_descr.beam
-rw-r--r--  1 X  staff  449744 Oct  3 15:05 gpb_descriptor.beam

The problem is that the main app requires gpb_compile.beam to be present in the specified folder in order to perform custom proto generation of some files. Otherwise, the app will not compile.

Things that I’ve tried, with no prevail:

  1. Upgrading protoc from 3.21.9 to 24.3;
  2. Making sure there are no whitespaces in the paths;
  3. Running mix local.rebar or with --force.

Please advise on how to properly compile this dependency given the new Elixir and Erlang versions. Thank you!

2

Sounds like somebody used private undocumented behaviour – never recommended.

Still, if you want to try and upgrade, it might be worth trying gradually incrementing Erlang versions first, and then Elixir ones, until you find the exact version that “breaks” it.

3

Managed to figure it out. What worked for me was to run make in the /deps/gpb folder as part of the compilation pipeline.

In my case a Makefile is executed as part of the compile step via GitHub - elixir-lang/elixir_make: A Make compiler for Mix, so adding a make -C deps/gpb in it has solved the issue.

4

That’s much better handled by doing rm -rf _build deps && mix do deps.get, compile btw.