Erlang OTP 28
Elixir 1.20.0-rc.3 (9b80ab5) (compiled with Erlang/OTP 28)
I am having a difficult to fathom issue with developing on 1.20.0-rc.3. Yesterday I switched from 1.19.4 to developing on that version and all seemed to be fine. Everything recompiled and tests all passed.
Today I added a field to an Ecto schema for a module - for the purposes of this issue the name appears to not be relevant so I called it xx
```
 field(:xx, :string)
``
After I did that I did a mix compile and got dozens of errors. The errors all referred to the structure in my schema, and the struct breakdown in the errors did NOT include that added field in the field list. The references in the warnings all looked odd (the references in the structure to a title of type :string displayed like they were a :map sometimes).
If I removed the newly added field and did a mix compile again then I got no errors.
Adding it back again caused all the type warning errors everywhere I referenced this structure. Mind you I did nothing other than add that one line - I expected NO warnings/errors.
Then I tried doing a mix compile –force
ALL my errors were now gone!!!
So I tried removing the newly added field. Did a mix compile and all my errors came back. Interestingly the type warnings included the field I had just removed in the structure breakdown in the error output - but I had just REMOVED IT! So how was it displaying a field in the list that did not exist in the source?
So I did a mix compile –force and like before all my type warnings disappeared.
My inference from the above is two-fold under 1.20.0-rc.3:
- Mix compile may not be compiling all of the code it should be under rc.3 (yes - it is very fast, but perhaps incomplete).
- When it fails to compile everything and then type checking occurs, the type checking is comparing a PRIOR version of my structure to the current version and finding the structs do not match.
I have not taken the time to create a simplified example of this, but this appears to be quite serious if my inferences (or some variation on them) is accurate.
This seems on the surface easy to reproduce - assuming a structure reference in a parameter to a function which is where all the type warnings originated.
warning: incompatible types given to pick_items_needing_reviewed/2:
pick_items_needing_reviewed(cd, participant)
given types:
(
dynamic(%Sct.Solve.CollaborativeDesign{}),
dynamic(
%Sct.Align.Participant{person: %Sct.Network.Person{is_user?: nil or boolean()}} or nil or
(%Sct.Align.Participant{active: true} and
not (%Sct.Align.Participant{person: %Sct.Network.Person{is_user?: false}} or
%Sct.Align.Participant{person: %Sct.Network.Person{is_user?: nil}} or
%Sct.Align.Participant{person: %Sct.Network.Person{is_user?: true}}))
)
)
but expected one of:
(
%Sct.Perform.Fd{active: true} or %Sct.Perform.RoadMap{active: true} or
%Sct.Perform.TopicScorecard{active: true} or
%{
__meta__: term(),
__struct__: Sct.Solve.CollaborativeDesign,
active: true,
cd_details: term(),
created_at: term(),
description: term(),
id: term(),
issues: term(),
pds: term(),
permalink: term(),
published_state: term(),
restricted?: term(),
solutions: term(),
strategic_collaboration: term(),
strategic_collaboration_id: term(),
tasks: term(),
theme: term(),
theme_id: term(),
title: term(),
type: term(),
updated_at: term(),
xx: term()
},
term()
)
where "cd" was given the type:
# type: dynamic(%Sct.Solve.CollaborativeDesign{})
# from: lib/sct/misc/changebars.ex:55:41
%Sct.Solve.CollaborativeDesign{} = cd
where "participant" was given the type:
# type: dynamic(
%Sct.Align.Participant{person: %Sct.Network.Person{is_user?: nil or boolean()}} or nil or
(%Sct.Align.Participant{active: true} and
not (%Sct.Align.Participant{person: %Sct.Network.Person{is_user?: false}} or
%Sct.Align.Participant{person: %Sct.Network.Person{is_user?: nil}} or
%Sct.Align.Participant{person: %Sct.Network.Person{is_user?: true}}))
)
# from: lib/sct/misc/changebars.ex:56:8
show_changebars?(participant)
type warning found at:
│
57 │ pick_items_needing_reviewed(cd, participant)
│ ~
│
└─ lib/sct/misc/changebars.ex:57:7: Sct.Changebars.changebars/2
Above is one of the warnings AFTER I removed xx from the schema. Note xx appears in the breakdown of the schema - even as it has been removed. Also note that the breakdown appears like a map format with the internal __struct__ reference as expected.
I wanted to note I have been able to reproduce this behavior on other of my Ecto Schemas.
I have been able to reproduce this on a structure referenced as a parameter in 4 modules. Interesting part is when I compile I get only a warning re: one function, even as the parameter signature looks similar in all the modules. If I merely do a second mix compile I get no indication of anything compiling but I continue to get that single warning reference to one function in one of those modules. So after reconfirming that I added a blank line to the file containing the module that has the function with the type checking warning to try and trigger a recompile of that module. Another mix compile and it indicated 2 modules compiled. And my warning disappeared!
My inference from the above test is that not all of the modules that should be being recompiled are being recompiled. As that is likely a complicated process to minimize the recompilation of ONLY what needs to be recompiled, I infer that is where the issue lies - that not all of the modules needing a recompilation are on the list when I make changes elsewhere in the set of 1300 files I have.
@nulltree I am nearly 100% that this is an issue re: incomplete compilation selection. There are a couple nuances that concern me:
-
I would not have known about this “issue” were it not for the type checking in 1.20. I have done similar schema changes over the last few weeks using 1.19.5 as my Elixir version and never ran into a warning, because the warnings are a result of the type checking that is in 1.20.
-
The issue presents itself in “some” modules and not others. In fact I do not get type check warnings against functions in the same “un-re-compiled” source file with have a similar signature. So one function may elicit a type checking warning when the next with a similar reference does not. I cannot explain that.
-
Most importantly while I do believe this is a failure to recompile, when I look at the code reference to the schema module I do not see a “reason” for recompilation. EX:
def func(%Product{} = product), do: do_something(product)
if the structure of Product changes does this require recompilation? I frankly do not know.
- Finally due to all of the above I have not been able to ascertain how to make a simple example of this. When I try and simplify it, the issue disappears. So I have yet to figure out the minimum assembly of code to repro in a “simple” example.
I am trying other examples by adding the field(:xx, :string) to the Ecto schema and then doing a mix compile . Am seeing multiple cases of the same behavior - incompatible type warnings that list an unchanged schema map. I do not know specifically how to tell if a module was recompiled, so I am looking in the _build/dev… folder for the modules associated .beam file. I can confirm that the files indicated with errors also show a last updated many hours prior to my testing - approximately the same time as my last “mix compile –force” . So my inference is that those modules reporting the type warning have not been recompiled after my schema change.
On the theory that the underlying issue is lack of recompile of associated modules when a structure changes, I decided to test that theory using this process:
- Return to 1.9.4 Elixir
- mix compile –force (force was probably unnecessary…)
- Make same change - add
field(:xx, :string) to same Ecto Schema as before
- do
mix recompile
I did not get any type warnings although I believe those may have been introduced with the type-checking in 1.20 so that is not surprising.
However when I previously did a recompile in 1.20 the logging indicated
\[14:08:32\] (base) ➜ sct_elixir git:(development) ✗ mix compile
Compiling 194 files (.ex)
Compiling 3 files (.ex)
With 1.9.4 of Elixir
\[15:02:49\] (base) ➜ sct_elixir git:(development) ✗ mix compile
Compiling 193 files (.ex)
Compiling 26 files (.ex)
So a significant number of additional modules were recompiled. In addition checking the modules I had gotten the warnings from in 1.20 in 1.9.4 compile it was recompiled according the the .beam last updated time.
From this testing I conclude that the type warnings are surfaced because the modules that should be recompiled are not all being recompiled on certain types of changes. I do not know how to look at the manifest that drives the interdependencies on modules and therefore compilation.
My conclusion is this is not a type checking issue, but a recompilation issue when changes are made.
It is a bug indeed, it will be fixed shortly.
1 Like
