where is your struct ‘Protocols.TimeSeries’ defined?
your module ‘Brain.Core.Utils.ToProtocols’ should know (alias or import) it
Protocols.TimeSeries is defined inside the :protocols library.
I have tried variations using alias or import with no success:
defmodule Brain.Core.Utils.ToProtocols do
@moduledoc false
alias Protocols.TimeSeries
def to_timeseries(forecast) do
%TimeSeries{points: to_points(forecast)}
end
end
or
defmodule Brain.Core.Utils.ToProtocols do
@moduledoc false
alias Protocols
def to_timeseries(forecast) do
%Protocols.TimeSeries{points: to_points(forecast)}
end
end
leads to the same error.
Meanwhile,
defmodule Brain.Core.Utils.ToProtocols do
@moduledoc false
import Protocols
def to_timeseries(forecast) do
%Protocols.TimeSeries{points: to_points(forecast)}
end
end
leads to
== Compilation error in file lib/brain/core/utils/to_protocols.ex ==
** (CompileError) lib/brain/core/utils/to_protocols.ex:4: module Protocols is not loaded and could not be found
‘alias’ does not throw an error if the argument is missing, but ‘import’ does.
The problem is that it does not find your ‘Protocols’ parent module/directory.
How is the directory structure of the ‘lib’ folder where the ‘Protocols.TimeSeries’ module resides? You can list it with ‘tree lib/’ if the ‘tree’ tool is installed
Given that Protocols.TimeSeries is in another library, the file is located at deps/protocols/protocols-elixir/lib/primitives/time_series.pb.ex. Others folders in the :protocols lib have the same structure or file extension.
defmodule Protocols.TimeSeries do
@moduledoc false
use Protobuf, protoc_gen_elixir_version: "0.10.0", syntax: :proto3
field :points, 1, repeated: true, type: Protocols.Point
end
When compiling, a corresponding _build/dev/lib/protocols/ebin/Elixir.Protocols.TimeSeries.beam file is built.
strange, I have no idea.
How does it behave with other modules from the ‘:protocols’ package?
Are there other modules like ‘TimeSeries’ that work fine?
maybe somebody else can help soving the issue?
seeing the file name /ebin/Elixir.Protocols.TimeSeries.beam …
is it possible that you have to do a
alias Elixir.Protocols.TimeSeries despite the module id declared with defmodule Protocols.TimeSeries ?
its just an idea.
Others modules in the lib have the same consistent faulty behavior. It depends on which module the app is trying to compile first.
Thanks for the help!
I tried adding Elixir. in front of aliases or imports with no successful effect.
Can you show the source tree of the library as it is in its own directory (and not how it is in your deps)?
The library tree looks like this
lib/
├── commands
│ ├── current_command.pb.ex
│ ├── power_command.pb.ex
│ └── setpoint_command.pb.ex
├── events
│ └── thermal_zone_updated.pb.ex
├── primitives
│ ├── point.pb.ex
│ └── time_series.pb.ex
├── services
│ └── predictive_control.pb.ex
├── signals
│ ├── constraint.pb.ex
│ ├── forecast.pb.ex
│ └── objective.pb.ex
└── state
├── storage.pb.ex
└── zone.pb.ex
All pb.ex files are generated using protoc. The mix.exs looks like
defmodule Protocols.MixProject do
use Mix.Project
def project do
[
app: :protocols,
version: "0.1.0",
elixir: "~> 1.13",
deps: deps(),
package: package()
]
end
defp package do
[
organization: "world",
files: [
"lib/**/*.pb.ex",
"mix.exs"
]
]
end
def application do
[
extra_applications: []
]
end
defp deps do
[
{:protobuf, "~> 0.8.0"}
]
end
end
maybe it is a protoc internal but I wonder that the module Protocols.TimeSeries is inside the primitives folder.
Do the other modules also have Protocols as a parent module even if they are in different parent folders? For example:
/lib/commands/power_command.pb.ex -> defmodule Protocols.PowerCommands
/lib/state/storage.pb.ex -> defmodule Protocols.Storage
Yes, all the modules are under the form Protocols.<Module>, like Protocols.PowerCommand, Protocols.Constraint or Protocols.Storage. Protocols as parent and only one child level.
I don’t understand how Protobuf can influence the compilation, as bumping versions down and up fixes the problem temporarily. Quite an investigation.
ok, I already suspected it … hmmm …
Does the problem occur if you delete the generated _build folder? If deleting the _build folder helps, then you may investigate under which condition the deletion is needed
Yes, rm -rf _build/ deps/ & mix deps.get & mix run --no-halt generates the same error. I found the same behavior while building the Dockerfile without using any cached layers.
Bumping once or twice the versions is the only way to make the app working. Editing one module where Protocols is needed, or used in another module using Protocols, and the compilation goes crazy.
as a last resort … can you talk to the creator of the :protocols library?
I can, that is me. Sorry if it hasn’t been clear ! 
We are using this lib (in another repo) because it is generating “protocols” and packages from protbuf definitions usable for our Elixir and Python services that needs to speak together.
(Thaks for all the help.)
anyway, I hope I could help a bit.
1 Like
If you fire iex -S mix in the directory of the library – not the app – can you then reference these modules e.g. with exports(Protocols.TimeSeries)?
1 Like
Looks like the thread is going to be very long without any progress. I would recommend to invest a bit of your time to create a minimal git or zip project. In many cases when doing so you can easily find an issue on yourself and if that would not be a case then you can share such sample project with community, so others may try to reproduce the problem and investigate it separately without your further responses.
The other option may be a temporary invite somebody to your private project. That would require less work from you, but obviously you would need to trust that person. I’m senior developer with many bug reports submitted from time to time in elixir, ecto, phoenix and few other projects, for example Mix test fails in umbrella (Ecto, Phoenix): db couldn't be dropped
If interested here’s my profile: Eiji7 (Tomasz Marek Sulima) · GitHub
1 Like
I found the error.
Protobuf was not fully building the structs, as all structs were available from the Protocols library but not defining their __struct__ method. With this method missing, all constructs under the form %Module{} were failing but not others forms.
Bumping the version of Protobuf from 0.8 to 0.11 was (kindly) enough to fix the problem, but as I found no changelogs, it is hard to find what changed between the two versions.
The exports() advice and merging and shirking down the projects to the minimal number of files (2) were very useful . Thanks!
2 Likes
