Elixir 1.11 Application boundaries and umbrella apps

I migrated my app to Elixir 1.11 and I receive some warnings that I need to fix for the build to pass on CI.

I have a bunch of umbrella applications. Since all umbrella applications depend on common libraries I have decided to have an umbrella where I list all the common dependencies and the other umbrella to depend only on this one. This worked fine prior to Elixir 1.11. Now it seems that all the dependencies need to be listed as direct child not sub-child.

warning: Jason.encode!/1 defined in application :jason is used by the current application but the current application does not directly depend on :jason. To fix this, you must do one of:

  1. If :jason is part of Erlang/Elixir, you must include it under :extra_applications inside "def application" in your mix.exs

  2. If :jason is a dependency, make sure it is listed under "def deps" in your mix.exs

  3. In case you don't want to add a requirement to :jason, you may optionally skip this warning by adding [xref: [exclude: Jason]] to your "def project" in mix.exs

Found at 3 locations:
....

Elixir release readme says:

This new compiler check makes sure that all modules that you invoke are listed as part of your dependencies, emitting a warning like below otherwise

But I would not want to copy paste all common dependencies since it will be overkill to maintain and upgrade all dependencies. How can I fix this warning without moving my common deps (50+) to each umbrella app??

Instead of copy pasting all 50+ of so called common deps, only add those handful to each project you actually use directly.

Sounds like a good opportunity for a some housekeeping.

2 Likes

This should not be necessary. Indirect dependencies are ok. If you are seeing different, please open up a bug report with a minimal case that reproduces it. Thanks!

2 Likes

We also see hundreeds of error like this one in ElixirLS (which makes it unusable). They all come from indirect umbrella dependencies, eg: A <- B <- C (which calls functions from A). The solution is to add A to C’s extra_applications but it does not make a lot of sense to me.

If C depends on A, then add A to the list of C’s dependencies.

:extra_applications is for applications that are part of the elixir or erlang distribution

1 Like

I know, but ElixirLS for some reason does not understand that. I’ve manually added transient umbrella deps to those apps and It removed all the warnings related to deps (leaving a hundred other false positives). I guess it’s just not ready for use on large codebases.

If you could create a sample repository that demonstrates the errors and submit it as an issue on https://github.com/elixir-lsp/elixir-ls/issues that would be appreciated.

4 Likes

A PR fixing that would also be appreciated

This is unfair. The fact it is not working for your codebase does not mean it is overall an issue for large codebases. It may even be an issue specific to your application, or specific to Elixir, especially because the feature it is emitting warnings on is fairly new to Elixir.

5 Likes

I’m sorry for using strong words there, but for me, it still feels like to use it you need to dance over the hoop to at least start using it. Someone who is just starting with Elixir may be not able to install it properly (especially if it’s not a plugin for VSCode which is okay, but not everyone is using that editor). I’ve spent a lot of time trying to set it up for us and here is my story:

  1. We all are using Atom. ide-elixir plugin for atom kinda works but I needed to manually release elixir-ls and replace it in the plugin folder. UI did not feel great as it did not highlight issues for me, instead, it kept poping up a list with 150 false entries and no scroll. It’s not maintained either, eg. there is no such package as nuclide any more and atom-ide-ui + ide-elixir makes Atom feel a bit slow when saving files. I was not able to make “format on save” feature work.

  2. Similar story with SublimeText, but I dig into that one less than Atom.

  3. Then I decided to switch to VSCode (because it’s the only one that is actively promoted) and started dealing with all the issues.

  • It just did not work. Because It was telling me that elixir is not installed. And I’m not the only one (https://github.com/elixir-lsp/elixir-ls/issues/408). I manually patched that by setting PATH in that script. I would love to see that day when plugin self-updates and it breaks again.

  • Then you just wait until it compiles, it’s good that I checked logs. I did not have any UI notification that it’s not ready to be used. A small thing that would make beginners extremely confused.

  • Next, I got hundreds of reports (lots and lots of project files were marked “red”). It was resolved by recompiling elixir-ls that was shipped with ElixirLS plugin for VSCode and manually replacing files in that plugin directory. (shipped elixir-ls was compiled with older Elixir and Erlang versions and it was breaking for some reason for me). I did not find a way to configure path to external elixir-lsp launch.sh. So I’m still waiting for the day with plugin auto-update.

  • And now I’m getting hundreds of dialyzer errors because it did not respect plain old Dialyzer ignore files: https://github.com/elixir-lsp/elixir-ls/issues/412.

  • In addition, sometimes server just crash and I have a feeling that VSCode plugin does not restart it. So after something like 18:23:14.332 [error] Process #PID<0.24932.7> raised an exception in the ElixirLS output I need to close and open editor again.

Maybe that is because our codebase is not just very large (it takes a reasonable amount of time to compile all the stuff) but also a very complex one (we have a legacy app in submodule back from Elixir 1.6 times and deps override on ENV variable). (For shocked people: we working hard to remove that app but this is the way for us to gradually migrate codebase, data, and users.)

I still value all the work contributors are putting into it, I know it would be better, but I won’t tell anyone that it was a pleasant experience for me now.

1 Like

What you have posted is much more valuable to everyone in this thread. There are issues you pointed that others can contribute and work on and, most importantly, of all the issues that you linked, there is active maintainer feedback.

It is fine to say that it doesn’t work for yourself and the complex structure you have right now, but extrapolating that to all projects based on your data point is definitely not ok. As I always say, for a fairer assessment, I recommend you to put yourself in the shoes of the maintainers for a couple months, help people on the issues tracker, Slack, in the Forum, and then you will certainly have a better assessment and appreciation for the work being done.

4 Likes

True, you raise a lot of valid issues but provide little in return. May I remind you that this is OSS maintained by the community members in their free time with no warranty nor commercial support.

We all are using Atom. ide-elixir plugin for atom kinda works but I needed to manually release elixir-ls and replace it in the plugin folder. UI did not feel great as it did not highlight issues for me, instead, it kept poping up a list with 150 false entries and no scroll. It’s not maintained either, eg. there is no such package as nuclide any more and atom-ide-ui + ide-elixir makes Atom feel a bit slow when saving files. I was not able to make “format on save” feature work.

If something is no longer maintained then feel free to fork it and take up the maintainer’s mantle (if the license permits that).

It just did not work.

If it’s impending your team’s work then please take your time to debug and fix it. I use elixir-ls and a lot of other projects for my day to day commercial work and if something is broken/missing, I try fix it and contribute.

Then you just wait until it compiles, it’s good that I checked logs. I did not have any UI notification that it’s not ready to be used. A small thing that would make beginners extremely confused.

That would be an easy first contribution. Here are some docs to get you started Specification

2 Likes

How much should I provide in return to feely share my opinion on a forum that something, in my opinion, is not ready? Didn’t I give enough feedback to clarify all issues I have faced? Did I ever tell that ElixirLS for VSCode is bad? Did I ever tell you that contributors are doing a bad job or something? I agree with Jose that extrapolating myself is not right but with a list of issues I’ve described people can extrapolate themselves.

I value your work, and Jose work, and everyone else who is spending personal time developing open source. I have a hundred OS repos myself. Fortunately, they don’t require as much maintenance as before. If someone would tell me that one (or all) of my libraries smell :poop: I would not go the “what did you do to make it better?” route but would try to clarify what is wrong with it and either agree (and have good points to make it better / ask for a PR if it’s appropriate) or disagree. And this comes from my first touch with Elixir: nobody expected me to fix everything I find when there is an issue people (mostly Elixir Core team) would do an extra step to help to solve it and since then I’m trying to do that too.

And I don’t even expect it from you, we all are free people, I believe you are not paid to solve every single issue opened in GitHub that that is totally fair to not spend time on stuff you don’t want to deal with. Just please don’t start judging who is providing little or not in public.

It’s really sad when valued and well-intentioned people clash about a misunderstanding.

Opinions are dime a dozen, and I guess due to lots of quite upsetting things happening right we might be quicker than usually to give harsher feedback than needed. I really think AndrewDryga meant no harm and his comment is based on his personal (frustrating) experience. He did not claim absolute correctness and made an open assumption that EllxirLS did not work as expected for his codebase. Because well… It was a confusing experience and one has a right to express his/her frustration. I hope we stay true to core values in the Elixir community and don’t let this misunderstanding grow out of proportions. We need every bit of support and “good vibrations” in current times.

Hope this comment did not offend anyone and gave a bit different perspective to the whole situation :purple_heart: :purple_heart: :purple_heart:

9 Likes

Where should such a bug report be raised? I am seeing this same warning but with dialyzer, which is an indirect dependency of my project, pulled in by dialyzex, which is declared as a dependency of my project.

Assuming you have a minimal case that reproduces the issue and the warning comes from Dialyzer (i.e. when running the dialyzer command), then it should be reported to Dialyzex first. :slight_smile:

1 Like