Request for testers: Gleam support in Mix

Hi!

I have a PR on the elixir repo that adds support for gleam in Mix.

There you’ll find the instructions to test the feature before it’s released. It will allow you to add gleam path dependencies in your mix.exs file without the need for the mix_gleam archive.

Please try it out and report back with your findings, if any.

Thanks,
Rodrigo.

17 Likes

Thanks for working on this PR Rodrigo! It’s coming in at a perfect timing :smiley:

Today I was trying to set up a new umbrella repo and import an external Gleam project. I had started with gleam_mix but was clearly having issues given my umbrella + external Gleam project setup. Then I found this post and I’m super grateful for the help to get this working.

I just compiled your branch (I’m using Nix with src overlay) and rewound the gleam_mix config. At first I got this error:

12:19:03.397 [error] beam/beam_load.c(583): Error loading function ‘Elixir.Hex.Netrc.Cache’:fetch/1: op make_fun2 u: please re-compile this module with an Erlang/OTP 27 compiler or update your Erlang/OTP version

That was easy to fix by wiping out project and home folder caches (e.g. rm -rf ~/.mix).

Now when I run mix deps.get, I’m getting this:

zsh ❯ mix deps.get
error: unrecognized subcommand 'package-information'

  tip: a similar subcommand exists: 'package-interface'

Usage: gleam export <COMMAND>

For more information, try '--help'.
** (Mix) Command "gleam export package-information --out /dev/stdout" failed with reason:

I believe that is because I’m running on Gleam 1.9.1. Should it be using the ‘package-interface’ given this is the newest version?

I believe that is because I’m running on Gleam 1.9.1. Should it be using the ‘package-interface’ given this is the newest version?

No, export package-information is a new command I had to implement in the gleam compiler precisely for Mix integration: it exports the gleam.toml file as JSON so Mix can use it.

As described on the PR comment you need to install gleam nightly to have this new command, which will be released in gleam 1.10.0. That should fix the issue you are having.

Let me know if you hit any other hurdle after that or if it works as expected. Thanks!

Amazing! Just using the latest main branch from Gleam and mix deps.get and mix compile are working fine. I can see Gleam logs during the compilation and the artifacts under _build. Now I’m struggling to find how to import anything from the Gleam package into Elixir. I couldn’t find any module or atom referencing it.

I tried to add gleam_stdlib in the elixir deps list, mix compile works fine but now when I run iex -S mix I get:

** (Mix) Could not start application gleam_stdlib: could not find application file: gleam_stdlib.app

Not sure if this was expected. I was trying to import stdlib and run a basic test.

Yes, that’s expected because gleam_stdlib does not have an .app file, so you need to add app: false to the dependency in mix.exs. In fact this is true for any hex dependency as well.

It makes sense. And it is all working now. Thank you so much @Papipo!

One observation is that adding app: false leads to a conflict:

Dependencies have diverged:
* gleam_stdlib (Hex package)
  the dependency gleam_stdlib in apps/buzz_agent/mix.exs is overriding a child dependency:

  > In apps/buzz_agent/mix.exs:
    {:gleam_stdlib, "~> 0.34 or ~> 1.0", [env: :prod, hex: "gleam_stdlib", app: false, repo: "hexpm"]}

  > In /code/claude/corev2/gleam.toml:
    {:gleam_stdlib, ">= 0.34.0 and < 2.0.0", [env: :prod, hex: "gleam_stdlib", repo: "hexpm"]}

  Ensure they match or specify one of the above in your deps and set "override: true"
** (Mix) Can't continue due to errors on dependencies

Adding override: true did fix it (though felt a bit weird having to override it). Also realized I could have used it as a transitive dependency from the gleam package which would have avoided the conflict but led to having direct code usage on transitive packages.

Another improvement, though possibly unrelated to this, is making the gleam dependency atoms (.e.g. :gleam@list or :gleam@json) visible or searchable in iex. It seems there is some lazy loading because they don’t show up for tab completion in iex, but after you use them once, then they show up.

Yeah, I am aware of the need for override, already looking to fix it.

What you say about iex autocompletion is interesting but I don’t know to do it, I’ll investigate.

Thanks for the feedback!

Can you pull the latest changes on the elixir branch, recompile and try again?

app: false was being forced and I’ve just removed it. This fixed the issue for me.

After switching to the latest commit, mix compile is failing with:

Unchecked dependencies for environment dev:
* gleam_javascript (Hex package)
  could not find an app file at "_build/dev/lib/gleam_javascript/ebin/gleam_javascript.app". This may happen if the dependency was not yet compiled or the dependency indeed has no app file (then you can pass app: false as option)
* gleam_crypto (Hex package)
  could not find an app file at "_build/dev/lib/gleam_crypto/ebin/gleam_crypto.app". This may happen if the dependency was not yet compiled or the dependency indeed has no app file (then you can pass app: false as option)
* gleam_regexp (Hex package)
  could not find an app file at "_build/dev/lib/gleam_regexp/ebin/gleam_regexp.app". This may happen if the dependency was not yet compiled or the dependency indeed has no app file (then you can pass app: false as option)
* gleam_yielder (Hex package)
  could not find an app file at "_build/dev/lib/gleam_yielder/ebin/gleam_yielder.app". This may happen if the dependency was not yet compiled or the dependency indeed has no app file (then you can pass app: false as option)
* ranger (Hex package)
  could not find an app file at "_build/dev/lib/ranger/ebin/ranger.app". This may happen if the dependency was not yet compiled or the dependency indeed has no app file (then you can pass app: false as option)
* birl (Hex package)
  could not find an app file at "_build/dev/lib/birl/ebin/birl.app". This may happen if the dependency was not yet compiled or the dependency indeed has no app file (then you can pass app: false as option)
* gleeunit (Hex package)
  could not find an app file at "_build/dev/lib/gleeunit/ebin/gleeunit.app". This may happen if the dependency was not yet compiled or the dependency indeed has no app file (then you can pass app: false as option)
* gleam_json (Hex package)
  could not find an app file at "_build/dev/lib/gleam_json/ebin/gleam_json.app". This may happen if the dependency was not yet compiled or the dependency indeed has no app file (then you can pass app: false as option)
* json_blueprint (Hex package)
  could not find an app file at "_build/dev/lib/json_blueprint/ebin/json_blueprint.app". This may happen if the dependency was not yet compiled or the dependency indeed has no app file (then you can pass app: false as option)
** (Mix) Can't continue due to errors on dependencies

with the following gleam deps:

      {:gleam_stdlib, "~> 0.34 or ~> 1.0", app: false, override: true},
      {:gleeunit, "~> 1.0", only: [:dev, :test], runtime: false},
      {:corev2, path: "../../../corev2", app: false}

Only got it “working” by removing the local dep and adding app: false to gleeunit:

      {:gleam_stdlib, "~> 0.34 or ~> 1.0", app: false, override: true},
      {:gleeunit, "~> 1.0", only: [:dev, :test], app: false, runtime: false},
      # {:corev2, path: "../../../corev2", app: false}

I need to add more tests and ensure all cases are covered.

Why doesn’t gleam include .app files? Those are important for releases and embedded mode, so the runtime knows which modules to load. It may be that releases with gleam apps may not even work then? :thinking:

App files even when present are also missing some modules. I attempted to fix this, but my testing after the next release still seemed to have them missing: Include erlang ffi modules on .app file for releases to work with gleam apps · Issue #3710 · gleam-lang/gleam · GitHub

In fact I think that the “fix” here is to always include app files, that’s it. I can include a minimal one if there are no applications to start, etc, right?

We should probably have a discussion with the Gleam team, as I would say it is the job of their build tool. I wouldn’t want to step in and generate the .app on their behalf.

They said:

If you’re using the compile-package API it does not because it cannot tell all the beam modules until after they are compiled, but that’s not the job of Gleam in that case.

And I can’t just create a minimal one because

It would be incorrect if there are Elixir modules in the package

I think once files are compiled there is probably something in mix I can use to generate the .app file? I can then inject Erlang options from gleam.toml

2 Likes

The Gleam compiler only generates Erlang from Gleam modules, but it doesn’t know what any Elixir modules in the package expand to, so it cannot create .app files unless it owns the whole project and so will be performing Elixir and Erlang compilation. This means that any build tool that uses the Gleam compiler will need to generate the required BEAM .beam and .app files.

edit: Oops, I saw Papipo beat me to it!

1 Like

I’ve have commited a change that leverages Mix’s task for App file generation (unless app: false has been passed as opt to the gleam dependency).

1 Like

This resolved the .app issue.


A new minor issue I’ve spotted is that the compiled dependency is not detected until mix compile is rerun.

For example, if I remove _build/dev/lib/gleam_stdlib and run mix compile, I get:

==> gleam_stdlib
Generated gleam_stdlib app
** (Mix) Unknown dependency gleam_stdlib for environment dev

If I rerun mix compile, the dependency is then detected by Mix.


We can, of course, do mix deps.compile then mix compile. I’ve never needed to compile twice when working in Elixir though.