I think it would be a very welcome feature, currently it’s about discipline in any bigger app, so I’m excited about this proposal.
I like option B the most but would also be pretty happy with C.
Will do.
I don’t disagree. But it still requires you to do something special in order to circumvent the private nature of the module anyhow; only it will be a well-known mechanism in the stdlib API and it will have better readability.
50/50 though, would not insist on it. Plus adding stuff to the stdlib is a really heavy-handed measure, that much is true.
1 Like
About point A:
I’m not sure if this is a good idea at all but, what if instead of being an exposed module, we inline all the functions/macros/etc… of the private module, inside the modules listed in @visible_to: [Foo, Bar, Baz]? Is that possible? Also, how could we guarantee that the use X from the private module doesn’t conflict with the use Y that uses the private module?
It’s just some libraries in our app, not a standalone app. Each module has its own test file. Just functions grouped by modules, nothing really fancy. But because they contain some arcane formulas, readability and testing is really necessary here for each module. We just couldn’t test only the top calling function and expect some magic to work.
Anyway, I also work in Ruby on Rails code in my fulltime job, having service pattern (something like this: Service Objects in Ruby on Rails…and you | HackerNoon) and form classes really helps me and my team to do testing and splitting the code for maintainability, it has been a breeze. But these service and forms classes are public, and they are reusable for anyone. (But this pattern also raises some significant problems that I will explain below.)
What comes to my mind, is that Elixir private module is a sort of contract for code reuse in internal app/library, but not for global/public. “You want to use this? Well just whitelist/make visible the caller.” This is probably intended for several people working together in a close ‘vicinity’. Probably the one who create a private module will be the one who will call (whitelisting) it as well. Something like: Jose created a private module, the farthest people who is going to call it is probably Chris McCord, not me, not you, not most Elixir users. CMIIW.
I’m kinda thinking that Elixir private module (if it is implemented) is best used for hiding inner complexity for libraries. “Yes you can test it, but please don’t call it if you can. Let us handle and maintain this private module. You don’t need to. You don’t have to.” This is in contrast with above service pattern, where everyone can use it, and hence when somebody needs to change (or hack it 
) it, it will affect all other codes which call it, it will end up with many flag variables if we don’t immediately refactor/duplicate it.
Now after some moment of thinkings, this private module proposal made me (and my friend) to rethink “among 18 modules that we got here, which one should go public for other developers to reuse? Should they be reusable? 
”. This in turn raise a question to me, which module should go public and reusable, which module should be private.
I like option C the best.
1 Like
Option C for sure, for the ergonomic and pragmatic reasons described earlier. I like the addition of the feature generally as well.
1 Like
I vote Option C.
Just to be super clear, this would be a compile time error, right? For example, will the following code compile?
defmodule MyApp.Other do
alias MyApp.Private
Private.hello()
end
Also, will private modules be able to be imported?
defmodule MyApp.Other do
require MyApp.Private, as: Private
import Private
hello()
end
And what about useing a private module?
Yes and yes, after it is required.
That’s exactly the productive thoughts such language features are supposed to provoke!
It’s OK to have a lot of internal modules that deal with various aspects of something big and ugly. They usually have to be private – and this proposal will still allow you to test them. But they usually shouldn’t be exposed as a public API inside your project.
OCaml does it via mli files, which are like h/hpp files in C/C++, they define an interface to a given module, and they can be different between internal use and external distribution (though that’s not a common pattern).
C++ namespaces can be hidden, that’s what an unnamed namespace is:
namespace MyApp {
namespace {
// Private functions here, entirely not accessible outside of this file and this file only
}
// Public functions here
}
You can’t even ‘craft’ a function pointer to such a function (not unless it’s returned via some other method).
You can ‘expose’ methods only to specific other (static) classes using friend, which lets you have ‘scoped’ static functions or methods that can only be called be specifically marked other classes, though this is not a common pattern either.
And of course C/C++ has headers as well, of which some can be internal and not exported to users of the library, that means you can even have multiple headers for the same ‘implementation’ with different things exposed in them, however the user of the library can ‘expose’ them manually by explicitly declaring them (which is definitely a red flag).
I’m personally for C, a hard error should be raised for invalid private access along ‘normal access patterns’, however there absolutely must be a way to bypass it as well for both live debugging purposes and working around issues in dependent libraries, and I wouldn’t want that via a :"elixirp.Blah".call() (though that would be nice for iex use, but its awful easy to abuse), thus I’d propose C with a slight extension being that in the mix.exs file a new attribute can be given to give a specific list of private modules to public module mapping overrides for the current project. Perhaps even hex.pm could disallow hosting of libraries that use any of such mappings. Perhaps it could look like:
def project do
[
...
private_access_overrides: [ # By this not being empty then this project is not allowed onto hex.pm
{SomeDep.Private, [MyApp.Some.Specific.Module, MyApp.Some.Other.Specific.Module]}
],
]
end
Maybe the tuple should even encode the SomeDep’s version number somehow as well, so it must be changed (or removed and properly fixed) each time it is updated.
EDIT: And iex access should be allowed via require SomeDep.Private, as: Private from within iex unbounded (with a warning printed each time it is used from within IEx too, just to make it obvious).
3 Likes
So when building a DSL, I often (following Ecto’s examples) will have an __attribute__/1 and similar functions that my DSL eventually calls into. Elixir does not generate documentation about these functions I’m wondering if there’s some parallel that can be made here, perhaps naming the modules defmodule __MyPrivateModule__ do ... end to convey similar intent. I’m not sure if we’re past that point already, but if it’s really just signalling to a user that something is private, that’s ugly enough to give a user pause, I think, and aligns with other conventions in the language, without requiring a new language concept, or requiring the private module to know its callers.
2 Likes
That’s a good question @asummers. Unfortunately __MyPrivateModule__ is not a valid alias name. Anything that starts with an underscore is either a function name or a variable. We could change the language to make __MyPrivateModule__ actually be an alias by checking the first letter after the underscores but that would not work with existing variables such as __MODULE__ and __ENV__. We would need to at least special case them in the parser (which is fine, they are already special cased in the compiler anyway).
The only downside of this approach is that it doesn’t say to which modules an underscored module would be visible to.
1 Like
The only downside of this approach is that it doesn’t say to which modules an underscored module would be visible to.
@josevalim I think that’s okay personally. I don’t think this is something the compiler should need to enforce if the problem is largely social. If someone’s using an underscored module, that’s their fault. They know the risks when they decided to use that module name, same as if they called an underscored function name. Perhaps that isn’t strong enough but to me not closing the private module (a la open/closed type semantics) seems more optimal.
I rather like having to use requirep insead od require + alias. It makes it explicit that we’re requiring a private module.
1 Like
In such case I would prefer to require using == when defining dependency version.
defmodule MyApp.Mixfile do
# …
def project do
[
# …
deps: [{:some_dep, "== 1.5.3"}], # fails if not using == version check
private_access_overrides: [ # By this not being empty then this project is not allowed onto hex.pm
some_dep: {SomeDep.Private, [MyApp.Some.Specific.Module, MyApp.Some.Other.Specific.Module]}
],
# …
]
end
# …
end
I like that idea, but note that in such case we would also need option like: package_namespaces: [MyApp, MyAppTest] and validate all defined modules inside lib and test. It’s because somebody could write defmodule SomeDep.Hack do … end. The problem with that is we can’t do it directly in compiler as @josevalim said. Doing it from some code analyse tools is also not a solution, because we could call something like: Code.require_file("hacks/some_dep/hack.ex").
The dependency resolver could use the information from the private access listing to try to acquire that specific version as well, but honestly I’d still prefer it to ignore it and for updates to ‘break’ it to encourage fixing it faster. It’s not like such a library that uses this construct would be allowed on hex.pm anyway.
Hmm, actually another idea, the private access overrides could go as options on the dependency tuple itself… Many places it could go.
And can’t forget that that could be entirely intentional, like say to expose a ‘private’ API only to a protocol’s implementations.
I’m not sure what an option would be for?
Though I don’t think the original C option to allow it to be an entire subtree match is right either, that should also be explicitly per-specific-module with an option for an entire subtree (great for protocols for example).
Ah, sorry - I though that you are talking about extra protection (when reading that part with hex checks), so I just started talking about possible hacks. 
I did not realized that you wanted to only extend access for specific private modules. It’s good idea too. 
1 Like
Option C gets my vote. I also quite like the defmodulep naming.
5 Likes
It would be nice if requirep Foo.Bar could automatically alias :"Elixirp.Foo.Bar" to Foo.Bar (I’m aware that elixir doesn’t support nested aliases and it probably won’t happen).
Anyway, option C is something that would be useful for me even today.
2 Likes
