Proposal: Private modules (general discussion)

[imetallica]

I think privateing modules will be a cure worse than the disease.

1. It breaks the semantics of the language. defp (an alias for defprivate) doesn’t allow access to functions outside the module, but defmodulep allows explicit visibility to some modules. Another naming is required.
2. I think that we should use an annotation, such as: @visible_to: [Foobar] or @scoped_to: [Bar] instead (maybe other names), to tag that this module is not for public usage. This should solve (1).
3. Although already a rejected idea, what we want to do is to guarantee that private modules are not accessible outside the application/package/library/younameit they are included, which is exactly what we are trying to do here.

Given my ideas, commenting on the proposals:

A. I like this idea, but with another name: @visible_to [MyApp, Foo, ...]. And if any module not in that list tries to use, import or whatever it will be a compilation error, much like what happens when you try to call a defp function outside the module you are defining it.

B, C & D. No because of breaking semantics: explained in (1).

[webdeb]

Defining private modules and functions is like defining blacklists. The other approach would be to define whitelists. So instead of defining private modules / functions. Only those which are whitelisted (public) would be exposed to outer world.

I think this would be the way to go. Maybe its something for elixir 2.0, but I think it would make the language more stronger for the future.

just my 2c on this topic in general

[josevalim]

@imetallica

A. I like this idea, but with another name: @visible_to [MyApp, Foo, ...] . And if any module not in that list tries to use, import or whatever it will be a compilation error, much like what happens when you try to call a defp function outside the module you are defining it.

We can’t have compilation errors with A because it is best-effort. It would be jarring to have compilation fail only from to time or in a later optional pass. If you want errors, then you need the require mechanism and a separate namespace.

@imetallica

It breaks the semantics of the language. defp (an alias for defprivate) doesn’t allow access to functions outside the module, but defmodulep allows explicit visibility to some modules. Another naming is required.

That’s a good point and we can discuss other names. However, that would be too specific for now. So instead of worrying about the names themselves (such as scope/defmodulep), can you please share your thoughts on the proposal in considering the merits of the mechanisms instead, regardless of names? Thanks!

[blatyo]

My preference would be A, D, C, then B. B is out because of the reasons already stated.

I’m concerned about this being an error, because it’s effectively a backwards incompatible change. This changes the meaning of private from, it’s not documented to I’ve explicitly marked it as private. Which means that everything that was considered private before isn’t. I may have been calling something that was private before and now because B or C were chosen, my code fails to compile. This is quite similar to using a private API and Elixir changing it and my code breaking.

[lpil]

@blatyo

because it’s effectively a backwards incompatible change.

It wouldn’t be a backwards incompatible change as all existing forms and macros would remain the same. It would be a new macro with new functionality.

[blatyo]

@lpil

It wouldn’t be a backwards incompatible change as all existing forms and macros would remain the same. It would be a new macro with new functionality.

If someone changes defmodule MyApp.Foo to defmodulep MyApp.Foo, ..., that is backwards incompatible.

[NobbZ]

I do not think, that introducing defmodulep would invalidate the former meaning of @moduledoc false.

The latter is “You use this at your own risk” and will ever be, the former is “You are not allowed to do this, and I will try the best I can to not let you pass beyond this point”.

And if a @moduledoc false module becomes defmodulep, well, you have been warned before… There never was a promise or contract to not remove that module…

[lpil]

@blatyo

If someone changes defmodule MyApp.Foo to defmodulep MyApp.Foo, ... , that is backwards incompatible.

That would be a backwards incompatible change in the library in question, not in the language itself. It would be the same as the library author renaming a function, or removing a module.

[blatyo]

@josevalim

let’s update Elixir! Unfortunately, updating Elixir breaks package X because X used a private API. So we have to update package X too but wait! That breaks Y and Z.

If people start making their modules private, won’t it lead to the same problem this is trying to address?

@lpil

That would be a backwards incompatible change in the library in question, not in the language itself. It would be the same as the library author renaming a function, or removing a module.

I was assuming Elixir would start using private modules as well.

[josevalim]

@blatyo

If people start making their modules private, won’t it lead to the same problem this is trying to address?

You can choose it happening one last time or it happening for the next 2 decades of the language.

[NobbZ]

@blatyo

I was assuming Elixir would start using private modules as well.

They probably will, but only in places that were @moduledoc false before.

[blatyo]

@josevalim

You can choose it happening one last time or it happening for the next 2 decades of the language.

I could see it as a warning now and a hard error in Elixir 2.0. Effectively a deprecation warning.

[Cochonours]

@imetallica

It breaks the semantics of the language. defp (an alias for defprivate) doesn’t allow access to functions outside the module, but defmodulep allows explicit visibility to some modules. Another naming is required.

As it is explicit, I do not think it is a problem. It looks like in C++ where you can have private members accessed by friend classes.

[NobbZ]

@blatyo

I could see it as a warning now and a hard error in Elixir 2.0.

So you want to deprecate the use of things that shouldn’t have been used from the start, because the developer tried their best to hide the function or module from you?

Again, we are only talking about making it harder to access things you shouldn’t have accessed at all, we are not talking about magic that breaks everything in existence…

[dimitarvp]

@blatyo

If someone changes defmodule MyApp.Foo to defmodulep MyApp.Foo, ... , that is backwards incompatible.

That is exactly what “backwards incompatible” is not. You consciously choose to use a new addition to the language. No legacy code will be broken.

[josevalim]

@blatyo

I could see it as a warning now and a hard error in Elixir 2.0. Effectively a deprecation warning.

If someone is using a private API (which is @doc false today), they are in no place to put a burden of the maintainers of that library. The author of said libraries are free to move the private code around, rename private functions, or remove private modules as they wish. This is a non-discussion for me.

[dimitarvp]

I heavily lean towards option C:

@josevalim

C. Provide defmodulep where modules must be explicitly required+aliased and it will error if you break its boundaries. However, the namespace the module will be assigned to is public , which means you can access it directly, without any visibility check, by using its long name. For example, defmodulep Foo.Bar would be accessible directly via :"Elixirp.Foo.Bar" .

However, I’d also prefer if next Elixir version adds a function like Module.get_private/1 that hardcodes the implementation detail logic of :"Elixirp.Foo.Bar". Better for the core team not to bind themselves to a hardcoded logic which might not endure the test of time. But maybe I am overthinking it. So far they always knew very well what they were doing.

In any case, having the option to do iex debugging in prod environments has been, and must continue to be, one of the best selling points of Elixir.

[scarfacedeb]

I’m ok with C.

Although I’d prefer having @visible_to as module attribute, instead of an option to defmodulep.

I got used to @moduledoc, @behaviour, @type and other special attributes. And having another extra one seems more natural than an option for me.

[dimitarvp]

@scarfacedeb

Although I’d prefer having @visible_to as module attribute, instead of an option to defmodulep .

It’s too easy for people to forget to put a module attribute in. Requiring it in defmodulep leaves them no such option.

[NobbZ]

@dimitarvp

However, I’d also prefer if next Elixir version adds a function like Module.get_private/1 that hardcodes the implementation detail logic of :"Elixirp.Foo.Bar" .

No need to, as you would require Foo.Bar, as: Private.

:"Elixirp.Foo.Bar" is a fallback hatch, a back door if you want, to be able to access the module during development or debugging sessions in prod through iex.