This hardly seems more complex than “don’t use modules with @modulesoc false” or “tag with @moduledoc false all modules you don’t want others to use”.
tmbb
tmbb
I dislike implicit naming conventions in modules (like what Phoenix does to match controllers and views). I’d prefer if this was as explicit as possible.
brightball
Shouldn’t say solved. I was just thinking if distillery had explicitly listed version requirement for Elixir.
In thinking it through further, I can see a lot of other problems that would cause though, so never mind. That was a hasty comment.
Shikada
I would argue that this isn’t a technological solution at all, but exactly what you advocate. It’s a communication tool designed to help with “people problems”.
And as far as learning, most people are accustomed to working in a language that has some form of private/public parts in it. If they then learn how Elixir does it currently, it would seem like nothing short of a really hacky solution.
Again, this proposal isn’t here to stop determined attempts to use private code as it were public. It’s about the clear expression of intent. I just don’t understand how some people see this as a negative. Conventions and warning are at worst hacky, at best easy to miss or ignore.
dimitarvp
True, but in my experience with people they always ignore any warning signs but even a slight slap on the wrist immediately gets their attention.
josevalim
Creator of Elixir
This isn’t the only problem this proposal solves. As many others have said, even within the same application, maintained by the same team, this proposal helps outline boundaries for future uses of the code. Edit: I have updated the proposal to list this as well.
Also, I have had in multiple instances to maintain code that I was really not supposed to, nor didn’t want to, to not break other people’s code that were accidentally using private APIs. It is easy to see how this is not healthy in the long term: it discourages the team to bring more improvements while increasing the maintenance burden.
So saying this only aids with the public perception is not true. Especially because, it is not about even about perception, since those applications are actually broken on updates due to transitive dependencies. This is mentioned directly in the proposal: “Even more worrying, is that this practice in the long term can be really harmful as systems grow in size”.
Isn’t improving the language so we can better communicate between teams and users about our boundaries a way to use technology to improve what is inherently a people’s problem? What are the other suggestions then to solve this problem? How can we better reveal intent about the use of “private” modules given the current mechanisms are clearly not enough?
10
baldmountain
I’m not sure I accept the original premise/reason for this proposal. I understand the desire to have a demarcation between the exposed API and internal implementation. but I prefer to rely on the structure of the library and it’s documentation rather than forcing something explicit. I like the fact that everything is public. Most libraries are structured with a main module with the exposed API, all other modules are implementation. If I use something in an implementation module I do so with the understanding that my code will break in the next version and I’ll need to fix it.
As to the issue if Elixir is updated and breaks package X, which then breaks package Y and Z, I’m okay with that. Package X will get updated and package Y and Z will get updated. If one of them doesn’t, say Y, then folks will either stay on the previous version of Elixir if they HAVE to use Y, or package A will come along and replace Y. This is just the natural state of Open Source Software.
Having said all that, I’d prefer option D then C if I have to choose one.
dimitarvp
There are literally maintainers of the biggest and most used Elixir packages in this thread who tell you that this is not working. I sympathize with the sentiment but practice shows that many people simply rush their way through tasks and do not notice these nuances.
Agreed. But that’s not helping corporate teams. We have enough work as it is and we can’t be eternally tracking The-New-Hotness®. We have stuff to do. If we cannot keep on top of things by re-reviewing the ecosystem once a month then we’ll eventually move our business elsewhere.
tmbb
I’ve been thinking and maybe my main objection is that any package that used defmodulep will be instantly impossible to use in older Elixir versions. Because defmodulep is a macro, you can’t simple define it in a dummy module somewhere in your app, and just use it. You have to import your dummy module, and that propagates through your source in a distracting way. But that’s the cost of innovation, I guess…
The more I think about this proposal, the more I think there should be both a requirep and an aliasp macro. An aliasp macro would play well with the idea that private modules should start with :"Elixirp." and would avoid the cost of requireing the module (because alias calls can be made in parallel).
Even if we decide to have a stronger level of name mangling, such as starting the private module names with :"Elixirp.<hash_of_the_module_name>.", aliasp would work as long as the hash is deterministic. So no need to require the module anyway.
Another possibility would be to rename aliasp as something like expose. That way it would read quite naturally:
defmodulep MyLib.Hidden do
@visible_to: [MyLib.Public]
# ...
end
defmodule MyLib.Public do
expose MyLib.Hidden
# or
expose MyLib.Hidden, as: Hidden
# instead of
aliasp MyLib.Hidden
# ...
end
require could continue to be require instead of requirep in this case.
But ultimately, I guess I prefer the consistency of having everything end with a p: defmodulep, aliasp, requirep, defp, defmacrop, defguardp, no matter how bad it sounds while reading these aloud.
So I’d vote for defmodulep, requirep and aliasp.
I still prefer the mangled module names instead of the ones starting with the fixed Elixirp prefix, but for practical reasons I might settle for Elixirp.
gregvaughn
As I said, I fully accept that we will end up with private modules in Elixir. I have no illusions that I will change anyone’s mind. I continue the conversation only for those who wish to understand the contrary opinion.
Within a team, if some part of the code they own is dependent on some other private part that they also own, then it’s up to them to resolve that before refactoring the private part. If/when a team chooses to try to prevent this situation, then they have choices like pair programming, code reviews, linting tools, etc.
Effects within a team are less important than third party dependencies though. I’ve lived through projects with bloated node_modules, 1000 line Gemfiles, and heavyweight maven dependencies. The more of those you have, the harder it is to upgrade. And, yes, I’ve even debugged into some of those libraries that intentionally work around the language’s privacy features. The more dependencies you have, the greater the chance at least one of them has something like this happening. Private modules will change how frequently it happens, but it does not completely prevent it.
Third party libraries are not “free as in beer” but more like “a free puppy.” Puppies are great! But, you may not be able to make impromptu trips out of town any more because the dog needs to be fed, or you may have to buy new shoes because the puppy chewed them, etc. The point is that third party libraries change your “lifestyle.” Teams should take a more cautious approach to using them than is the norm today. This is the direction toward reduced brittleness in software in my opinion, but I’m still trying to figure out best how to encourage it. Thinking that private modules solves the problem once and for all works against this goal. I’m not suggesting anyone specifically thinks that, just that it would be contrary to my goal.
I didn’t mean to imply perception was the only value, just that it was the primary one in my mind. Elixir is a growing community, and developers have plenty of other choices of language communities to invest their time in instead. We’d like Elixir to reach a critical mass that we can continue (or begin to) enjoy our time working in this wonderful language professionally. If people who are considering investing time with Elixir hear about too much brittleness it might send them elsewhere and limit that critical mass.
I wish I had a great answer, but I don’t. But my main point is this is not so clear to me. And on top of that, I’m not sure we’re weighing the costs of the feature enough. Aside from what I mentioned earlier about needing to learn and use it, what about operational debugging when calling a private module is a huge help, or how sad will it be to announce the compiler has slowed down x%, or what if OTP introduces a low level feature in the future that would give us an even better way to communicate privacy?
It boils down to how we weigh the “current mechanisms are clearly not enough” with the potential intangible cost of private modules in the long run.
So, I’ve shared my opinion. I don’t have the time or energy for a big debate. I"m not upset about it. I expect to see private modules in Elixir, and I expect I’ll grumble a bit to myself but mostly my life will go on with few changes.