Dialyxir - Recent Updates and Request for Feedback

I think the main difference is that dialyze is controlled by command line switches, and dialyxir is driven more by configuration. dialyxir does take a couple of args, and you can pass any dialyzer arg and it gets passed on. dialyze is a single task and it will just do whatever is required to setup the PLT when you run it. When you are just doing analysis with dialyze you probably want to pass --no-check so that it doesn’t check for updated apps every time you analyze your project. With dialyxir you have to run the separate dialyzer.plt task to create/update/check the PLT. This lets it default to --no-check.

The implementations are also different; dialyze uses the erlang :dialyzer module to perform all operations. dialyxir builds a command-line string and shells out to dialyzer. It prints this string which is handy for debugging; several issues (usually user environment/project specific) that have been reported to the issue tracker I was able to figure out just by reading the command string it generated.

dialyxir was started first but probably lacked some features when dialyze came out. I think dialyze has had more sensible defaults, for example it has always included all direct dependencies in the PLT. I do not think it lets you change that though; dialyxir gives you full control of what goes in the PLT. The thing is I do not know if anyone wants that. dialyxir has lots of configuration and I have little idea of how much of it is used.

2 Likes

Instead of adding it to my 30+ *_view.ex modules I added it to the *.Web.view/1 with a TODO: to remind me to check if this has been fixed upstream yet and remove it if so. Much easier having it in one place, and it works. :slight_smile:

1 Like

Found another for your wiki page, Phoenix.Presence is also apparently bugged as this code:

  use Phoenix.Presence, otp_app: :my_server, pubsub_server: MyServer.PubSub # Line 75

Generates this dialyzer issue:

help_presence.ex:75: Expression produces a value of type {'ok',pid()}, but this value is unmatched
help_presence.ex:75: The inferred return type of init/1 ({'ok',#{}}) has nothing in common with {'error',_} | {'ok',pid()}, which is the expected return type for the callback of 'Elixir.Phoenix.Presence' behaviour
help_presence.ex:75: The inferred return type of track/3 ({'error',_} | {'ok',binary()}) has nothing in common with 'ok', which is the expected return type for the callback of 'Elixir.Phoenix.Presence' behaviour
help_presence.ex:75: The inferred return type of track/4 ({'error',_} | {'ok',binary()}) has nothing in common with 'ok', which is the expected return type for the callback of 'Elixir.Phoenix.Presence' behaviour
help_presence.ex:75: The inferred return type of update/3 ({'error',_} | {'ok',binary()}) has nothing in common with 'ok', which is the expected return type for the callback of 'Elixir.Phoenix.Presence' behaviour
help_presence.ex:75: The inferred return type of update/4 ({'error',_} | {'ok',binary()}) has nothing in common with 'ok', which is the expected return type for the callback of 'Elixir.Phoenix.Presence' behaviour

And this fixes most of them:

  @dialyzer [
              {:nowarn_function, 'init': 1},
              {:nowarn_function, 'track': 3},
              {:nowarn_function, 'track': 4},
              {:nowarn_function, 'update': 3},
              {:nowarn_function, 'update': 4},
            ]

However it leaves this:

help_presence.ex:75: Expression produces a value of type {'ok',pid()}, but this value is unmatched

And I am not sure what is happening here, any help? Is this one or the others my bug instead?

Is it possible to share an example?

The stock PhoenixFramework Presence example should do it as that is mostly all that mine is:

defmodule MyServer.HelpPresence do
  use Phoenix.Presence, otp_app: :my_server,
                        pubsub_server: MyServer.PubSub

  def fetch(_topic, entries) do
    entries
  end
end

I wrote dialyze because I wanted a dialyzer task to make very different decisions to dialyxir. I would have liked to have contributed these changes but I wanted to change nearly everything.

dialyze is designed for zero/minimal configuration, repeatable/accurate warnings, smart PLT creation, sane defaults and to automatically make decisions it knows should be done without manual user commands.

Zero/minimal configuration:

In the majority of cases all the information required to build a PLT already exists if an application adheres to OTP principles. The PLT should contain the dependencies of the project and on application should have all it dependent applications listed in its applications. This means dialyze won’t work if not creating OTP compliant applications. I think a lot of people don’t. For example I think phoenix users can’t use this task very well because phoenix does not generate the correct applications list.

Therefore dialyxir will likely be more appropriate if you are a new user because it will show unknown warnings when run and then the relevant application(s) can be manually added to the configuration. I am probably more anal than average about OTP principles so I was confident for my own use that dialyze wouldn’t be a problem here. However if you need the configuration then dialyxir is your only option.

Repeatable/accurate warnings:

Since the modules added to the PLT are completely controlled by the dialyze task it can change the PLT to reflect changes in dependencies. It does this by adding 2 extra phases to --check-plt that dialyxir doesn’t do. It builds the application dependency tree (by using the zero configuration PLT approach above) and works out which files should be in the PLT. Then it removes any modules that shouldn’t be there, then it checks the modules in the PLT are up to date with the latest versions of the modules and then it adds any missing modules. The 3 stage “check plt” is run every time by default (and only takes about a second if no changes to be made) so the PLT should always be the correct one. This feature might crash for a number of reasons prior to OTP 18.3 due to bugs in dialyzer itself.

Even though dialyxir works at the application level it will only perform the middle change using modules that are already in the PLT. Therefore if deps change adds/removes modules the PLT is out of sync which can cause false positives and negatives. I think the only way to avoid this with dialyxir is to completely rebuild the PLT when a dep changes but please correct me if this is wrong.

I could be wrong but I imagine the very vast majority of dialyxir users aren’t doing this and are missing warnings and getting invalid warnings.

Smart PLT creation:

dialyze will create 1 PLT per project per Elixir version per OTP version. The reason for this is that different projects may use different dependencies and different versions of dependencies. This may sound like it would be very slow but dialyze does smart caching of PLTs so if another project has already created a PLT for that version of Elixir and OTP, or just for OTP, all or part of the PLT can be reused. This means a project that doesn’t have any dependencies except for Elixir (and dependencies of Elixir like :kernel and :stdlib) it will build in a matter of seconds. Without this caching it will probably take a long time, maybe more than 10 minutes. This means getting started using dialyzer on a new project is fast. Because of the smart “check-plt” for repeatable warnings feature (above) the minimal changes are done to a PLT automatically when a dependency is added and the PLT does not need to rebuilt.

On the other hand dialyxir uses a global PLT per OTP version, which means you would can only use one global version of elixir and deps. Otherwise you will need to rebuild the PLT or end up using the incorrect PLT! It is configurable to avoid this but it is not the default. This default could mean using another project could break the PLT usage for different project.

I could be wrong but I imagine for many dialyxir users they don’t realise that PLTs from one project can interact with another. And for all dialyxir users that have realised this and set local PLTs they will have had to spend a lot of extra time waiting for PLTs to build.

Sane defaults:

As well as the (forced) defaults above dialyze does not add any extra warnings that dialyzer does not run. Apart from opting out of checking the PLT or opting out of running analysis this is the only part of dialyze that can be configured and requires command line arguments to turn warnings on and off.

However dialyxir adds extra default warnings: "-Wunmatched_returns", "-Werror_handling", "-Wrace_conditions", "-Wunderspecs". These warnings are not on by default because success typing should not produce false positives. All of these except -Wrace_conditions can cause them. These warnings are extra features that dialyzer is able to produce.

The first "-Wunmatched_returns" is a useful warning but perhaps a little to strict for a beginner, in most cases of this warning it is fine to ignore the return of a function - false positive. Requiring _ = call() in these cases is of questionable style. I like it but some don’t.

The second "-Werror_handling" can lead to false positive because it might be intentional that a function will always raise. I am unsure of a case where dialyzer won’t producer other warnings when this is not intentional.

The third "-Wrace_conditions" tries to find race conditions in the code. I am unsure what the warnings are because it has never found any in my code. Please let me know if it has found any for you. However it can add a lot of extra time to running dialyzer, I think it tries brute force. Some people have even reported it get stuck in an apparent infinite loop on the erlang mailing list. Therefore I would advise against running this warning. If you need race condition testing try concuerror it is much better at this job.

The fourth "-Wunderspecs" shows when specifications are too allowing. It may not be practical or helpful to use the more specific specification, i.e. false positives. For example a macro may return a subset of Macro.t and this warning will appear if dialyzer a subset that it can produce. However to a user of the macro it should only be applicable that the macro returns Macro.t. Enforcing such a strict spec often leads to requiring a backwards incompatible change in spec for what would otherwise be a backwards compatible change.

Automatically run commands that need to be run:

If a computer can work out that something needs to run then it should run it for the user without being prompted. dialyze will automatically do everything required: ensure the PLT is using the current deps and is up to date unless explicitly told not.

dialyxir requires manually building and checking the PLT. Even if checking was automatic the check would not be enough to ensure the PLT is up to date with deps.

Another difference is that dialyzer runs dialyzer inside the same VM. Whereas dialyxir starts another VM. I am unsure if there are bugs here like getting the wrong version of Erlang but if dialyer is slow and the user decides to kill mix, the extra VM will keep running and wont exit until the analysis finishes. You will need to look up the processes and kill it manually. There is a benefit to using the CLI though, unknown warnings are shown by default, whereas for the erlang API there are not and have to be turned on. Therefore dialyze does not show unknown warnings by default but dialyxir does.

Importantly dialyze is no longer maintained because no one was contributing and only people complaining. Instead I maintain the rebar3 provider which has all the features of dialyze (but is faster) plus the same (and more) configuration that dialyxir has. The rebar3 provider also adds various colours to different parts of the warnings to make them easier to understand and groups warnings by module.

I would suggest that any one interested in using dialyzer in elixir gets behind dialyxir and resolves the issues I have mentioned above. Otherwise the experience of using dialyzer will remain poor (and IMO broken) in elixir: Wasting ~10 minutes per PLT build (after the first global) that is not going to be managed properly and generate false positives and false negatives. Once that has been handled people may want to look at formatting the errors in a clearer style.

6 Likes

Also it is not enough to just add the direct dependencies to the PLT (as I think both dialyze and dialyxir), you will likely need to add deps of deps. dialyzer won’t know about the types of deps of deps and may not be able to infer the spec of dependency functions without the deps of deps if they dont have @spec. A previous version of dialyze handled this but some people had memory issues.

Edit: Ah dialyxir can handle this situation I think if it gets all transitive deps by walking the whole deps tree. This should probably be the default otherwise false negatives due to dialyzer being unable to infer type information.

1 Like

Thanks for the reply, the thinking behind dialyze is really fascinating, glad you shared it.

This is partially correct; dialyxir has always had functionality to check for missing deps and add them if necessary, but the user has to run the mix dialyzer.plt task after they make dependency changes. Until recently there was an ommission where it did not check for updates in dependencies that were added to the PLT, but it does do that now. For a medium sized project this whole process takes several seconds, at least the way I’m doing it - so I’ve kept it as a separate task.

I agree, if I can do it quickly enough, and presently I haven’t been able to - just checking if there are missing apps takes a couple of seconds. Perhaps there could be a quick way to see if any dependencies of the project has changed, like a hash of the mix.lock file, that should trigger a recheck.

I just added support for doing this, and actually this is the reason I opened the thread is to solicit feedback on if it should be the default or not, I’m presently inclined towards making it so.

I do plan to adopt the dialyze method of PLT management, as I mentioned in the original post.

I’m open to changes in the default warnings, particularly removing -Wrace_conditions. The genesis of the default warnings used is an elixir-lang-core email from April 2013 which was my introduction to Dialyzer and one PR which added -Wunderspecs. I never made any changes as I didn’t think it was right to silently stop checking something that I used to check, even if it happened to be listed in a changelog. But my intention with the 0.4 release is clean-up some of these mistakes that have been left in the name of “compatibility”.

This whole experience has given me a very different perspective on worse-is-better. I wrote Dialyxir before I knew hardly anything about Dialyzer, and despite never mentioning it to anyone (until this OP) it sort of took on a life of its own and started getting PRs to fix some of the most glaring issues and to maintain compatibility with the evolving Elixir that I had lost touch with for a couple of years. Now for better (and often worse) a lot of people are trying to use it so I’ve taken a new interest in fixing it. Thanks for your feedback, it really does help.

2 Likes

Unfortunately I think this is fully correct but perhaps there is a miscommunication. If a dep is updated and this changes that dep so that modules are added or removed dialyxir does not add/remove these modules. Not that if a dep application is added. Dialyxir also never handles removal of removed deps. I think this will cause dialyxir to crash and require a rebuild of the PLT from scratch.

I did edit before your reply but you probably missed it, please see the edit, dialyxir is not adding transitive deps as I described in the edit.

Yes but I was trying to explain that the current PLT usage needs a lot of work and that is going to generate false negatives and false positives in the hope people will contribute. I decided to officially declare dialyze as not maintained in response to this post in the hope that you will stand up and produce a good task that I believe neither of us have been able to do so far.

The current defaults have beginner users wasting a lot of time. I have seen a lot of people struggling to deal with these warnings which shouldn’t be displayed and they get put off using dialyzer. Elixir can maintain the -Wunderspecs and other advance warnings because it has the contributors to hold itself to a higher standard than most other libraries, and it is not expected for beginners to use that, maybe not even more advanced users.

1 Like

Dialyxir will now (as of latest release) invoke the dialyzer --check_plt task which should handle updating apps modules when dependencies have had code changes. It is correct it does not remove apps, this is a liability of the global PLT and should definitely be fixed when I move to the dialyze style PLT management.

dialyxir gets transitive dependencies using Mix.Project.deps_paths - basically the same contents as the mix.lock file. I am not pulling in OTP applications from dependencies applications list and perhaps I should - is that what you see is missing?

Yes but I think we are talking passed each other as I was only talking about when a dep adds or removes modules.

Right! You may also had deps that aren’t needed using this method. dialyze uses .app files to do this. I am unsure if dialyze and dialyxir use the same license, if not I can provide a copy of dialyze under a matching license sans contributions from others, so you can re use what you’d like to.

The license is the same, apache 2.0, so there is no issue there.

You are right, I reread your original post and see that I missed you pointed out this is in addition to what --check_plt already does. I guess I’m not surprised --check_plt does not remove modules, but I am surprised it doesn’t add them - that would also cause problems for CLI users of dialyzer. I agree then I should also use this approach and your code for it in dialyxir.

Huh, is it possible to run rebar3 dialyze on a compiled Elixir _build?

I’m a happy user of dialyze and I made one (however small) contribution :slight_smile:

I think it’s a great tool and it pretty much just works as advertised.

One thing I’d like to have is some project config to include additional apps in PLT. The main case for this is mix. At my company we have a couple of custom mix tasks, and dialyzer complains since mix is not in the PLT. Obviously, we don’t want to add mix as a runtime dep, so our current solution is to suppress these warnings. IMO a better solution would be to explicitly request dialyze to include mix in the PLT via project config. I’ll be happy to make a PR if you’re open to this addition.

Has anyone discussed this with the Phoenix team? If Phoenix generator added plug and poolboy in the application list (but not compile-time deps), then this issue would be solved, right? And FWIW, I agree that it’s the proper way to go.

It is not possible. It’s very tedious to maintain the same features in mix and rebar3. Rebar3 got more attention because:

  • dialyzer warnings are in Erlang terms, Elixir won’t have a native feel without patching dialyzer and its only in the last year did we manage to remove compiler generated warnings in Elixir.
  • Testing was much easier (rebar3 had a better test framework at time) and orders of magnitude faster to test (don’t need to add :elixir to the PLT)
  • I tend to use Erlang for complicated projects that don’t require Elixir-only features to take advantage of its simpler syntax and thats when I want dialyzer the most
  • People contributed to it

Edit: Oops this was supposed to be a reply to @christopheradams

1 Like

At present there are two dialyzer tasks which lack the features of each other and in my opinion we have two poor tasks. As @jeremyjh is copying features (that are missing in :dialyxir) from :dialyze to :dialyxir there is no need to have :dialyze any more. The feature you want is present in :dialyxir so perhaps you should help @jeremyjh move the features you want across. The license is the same so nothing prohibits this.

I have brought it up on IRC several times where phoenix team members are involved in the conversation.

The question is whether it should be the other way around. I actually opted for dialyze mostly because of some differences you mentioned earlier. It seemed to me (and still does) that it’s pretty much plug-and-play, with sensible default choices and optimized workflow. So from where I stand, it seems that dialyze should be the base, and a few features (e.g. customization of project PLT) should be merged from dialyxir.

But TBH, I took only a brief glance at dialyxir , so I’d like to learn what’s in dialyxir that’s missing from dialyze?

And what was the answer? :slight_smile:

Both projects are under apache 2.0 so nothing stops anyone doing this. I think it is best for the users of both if we get behind one library in the long run and @jeremyjh wants to do the work so I think we should let him :smile: and get behind dialyxir. I do not want to do this work.

I wrote a long reply above to explain the differences and help @jeremyjh to move forward, and I got some differences wrong :smiley: so I’ll let someone else answer that.

I don’t think they saw it as an issue because phoenix will start plug. However if transitive mode is enabled on dialyxir then plug will be added anyway, and hopefully this will become the default in the next version.

Dialyxir permits configuration of many different things, and while its defaults are not good it can be made to work acceptably well for almost any project due to this. I think ability to have mix file configuration of warnings (both disabling default and enabling the extended checks) is a good idea; this should be captured somewhere in your projects configuration management, and not left to command line parameters unless those parameters are managed in some higher-level script.

It has other configuration which is maybe more dubious - for example it will let you configure the location of the PLT file; this is needed in dialyxir because its default is stupid. But even for dialyze which has a really smart way to handle this, there was a long open request in its issue tracker to support configuration of this. It even lets you specify the path to the beam files to analyze…does anyone use it? I don’t know if they do now- a few people did between the time mix began using the _build directory and when dialyxir was updated for that. Maybe someone, somewhere will want it.

Being able to configure the plt apps is handy - right now dialyze isn’t pulling transitive dependencies due to a memory problem it caused in some projects. Probably if I add that code back someone will have the same problem in dialyxir and they’ll be able to solve it in configuration by specifying only direct project dependencies plus adding specific transitives that they know they need. dialyxir supports this right now and because of it, its reasonably easy to dialyze a Phoenix project. Its just not defaulting the best way presently.

So my thinking now is we need something to be smart like dialyzer and configurable like dialyxir.

Right now this is the list of changes I’m pretty sure I need. Most of these things are pretty easy, I don’t see it taking quite a long time.

Dependencies

  • Include OTP applications from current mix & dependency .app files
  • Transitive by default
  • Include prod dependencies only by default?

PLT

  • dialyze style PLT file management, e.g. erlang/elixir core file copied into _build, app dependencies added only to this project local file
  • dialyze style plt update functionality (add/remove individual modules from dependencies)
  • warn users of :plt_file about new file scheme (allow suppression of warning with :no_warn) e.g. plt_file: {:no_warn,“local.plt”}
  • use mix_home for core storage - config via :plt_core_path
  • warn users who have the old ~/.dialyxir*.plt files but no new core file about backward incompatibility in the 0.4.0 release

Warnings

  • remove all default warnings
  • needs to be in the compatibility notice
  • -Wunmatched_returns ?

dialyzer task

  • invoke plt task if plt file is not found
2 Likes