Proposal: Private modules (general discussion)

I also like the name mangling features. I suggest a deterministic naming scheme like the following: Elixirp.<hash_of_the_module_name>.Module.Name. That

I wonder if it would be possible to have a make_all_private_modules_public/0 function or macro on IEx, which would alias all private modules somehow.

require + alias can be done in a single require call. The difference in usage between require and requirep is literally just the extra p.

When you alias, you can alias only to a single entity, like as: Bar, never Foo.Bar. Which is why we don’t implicit alias and you need to give it the :as bit.

You can’t because of the above. Aliases need to be explicitly given because you can only alias the ā€œlast bitā€, and doing for everything automatically would generate clashes.

1 Like

I know it’s the same call, I was just trying to say that I like the extra p.

Ok, but that’s how the normal alias works… We can define a new alias-like thing that works as I’m saying, can’t we?

I personally can’t see how such feature would work. Aliases perform expansion based on the first element and if we start to perform it based on subsets then it adds complexity both in the implementation and in understanding of the code, as Foo.Bar can now be Bar.Baz and Foo.Baz can be something else completely such as Enumerable.

2 Likes

You are right and I’m stupid for not seeing the obvious problem. It seems like name mangling is a hard problem to crack… :confused:

2 Likes

C - ā€œYou have my swordā€. For me, defmodulep seems to be Elixir idiomatic.

1 Like

Would this be valid?

defmodulep MyApp.Private, visible_to: [MyApp.*] do
end

I could see it becoming a tangled mess if we have to update visible_to every time we want to reference this module from another one of our private implementation modules.

Here’s a contrived example:

defmodulep MyApp.EmailWorker, visible_to: [MyApp] do
  alias MyApp.Repo
  alias MyApp.EmailService

  def run(user_id) do
    user = Repo.get(MyApp.User, user_id)
    EmailService.send_welcome(user)
  end
end

defmodulep MyApp.Repo, visible_to: [MyApp, MyApp.EmailWorker, MyApp.KafkaJob] do
end

defmodulep MyApp.EmailService, visible_to: [MyApp.EmailWorker] do
end

Then if I wanted to rename MyApp.EmailWorker to MyApp.WelcomeWorker, in addition to updating the places where EmailWorker.run/1 is called, I’ll have to go update MyApp.Repo and MyApp.EmailService as well. If this MyApp.EmailWorker interacted with even private modules this could be a much bigger task.

It seems like without visible_to: having a wildcard option, this will create tons of difficult to manage circular dependencies, that would serve as a huge deterrent to refactoring how private modules are laid out.

Personally, @moduledoc false and treating submodules as private implementation details of the parent modules seems like enough to me, but I get the pain people have caused by no respecting these conventions.

That’s how it works! :smiley: visible_to is always by namespace for the reasons you mentioned. I don’t want to refactor code in MyApp to an internal module and then have to update the visible_to list.

3 Likes

My choice is B, followed by C if people complain too much about the name mangling.

Out of those, I’d strongly favor A or D.

In my experience, private methods and modules cause more problems than they solve. They increase logic duplication, complicate debugging and if developers are trying to access them…the developers usually have a good reason for doing so.

While taking the approach of exposing APIs as the top level is great, when a developer is working deep with a particular module there are often needs to dive into the weeds beyond the original developer’s intent to achieve your goals. If an overzealous use of private methods is in place, you end up in situations where you’re left to fork the module the code base just so that you can access / modify the method that you need.

While the intent behind private methods is good, in practice they are heavily overused and create a lot of problems for a developer. If I’m running code locally that applies to my application…I don’t want anything off limits, regardless of the original developer’s intent because I’m the one who will be maintaining and responsible for the application that’s using the module.

If I’m calling code on somebody else’s server, I’m 100% fine with the barrier. If I’m calling code on my own, the barrier just means that now I’m going to have to devote additional time and effort to get around it…and I’m still going to get around it one way or another.

2 Likes

What would it look like to start using defmodulep in libraries that need to support older version of Elixir than whatever version it is released in?

Is there some way to conditionally use it or defmodule based on the version of Elixir? Would it just not be available until the minimum supported version hits the version it was released in?

I see little value in locking people out of being able to access ā€œprivateā€ modules.

If I have to do something hacky that is my decision. I do not want to be unable to do what I need to do because a developer decided that THIS module should be private.

1 Like

I’m solidly in favor of option C as well. I think it strikes a nice pragmatic balance.

1 Like

You could probably create a macro that flips things around depending on the Elixir version but it wouldn’t be supported by Elixir per se.

If you do not want a developer to decide what you need to do with the code written by said developer, maybe you shouldn’t be using their code? :slight_smile:

1 Like

Would a combination of B & C be an option? I like the idea of being sure that I can mark a module as completely private. This is from the perspective however of working in a large codebase wanting to enforce the designed boundaries to other teams and less so about packages. Or would this make the heuristics of private modules more complicated to use?

You got it right: having both would complicate the heuristic. However, note the reason we decided to not go with B is not because of enforcing the boundaries to other developers, but because sometimes you may want to debug that code yourself, and making it fully private would not allow that.

1 Like

Well, if defmodulep isn’t smooth to use with code that needs to support older versions of Elixir, I think I favor option A, which would allow simply updating private modules with that attribute, which (I believe) would then work if the version of Elixir was high enough and be ignored otherwise, which is a much smoother path to start using.

In terms of options C vs D, I think that if you want something to be possible (in this case, being able to ignore boundaries), it’d be better to just support it gracefully rather than make people jump through hoops to use it, adding cognitive overhead in the process (if it’s called as :ā€œElixirp.Foo.Barā€, it means x, if it’s called as Foo.Bar it means y).

I know that warnings are easy to ignore, but option C seems like a pretty hacky solution to imprisoning your cake but eating it too.

So my preference is, A, then D, then C in a far third.

1 Like

While I believe being able to use it sooner is nice, it should play a very small factor. Do you want to choose something that is less optimal forever just to get a 1-2 years head start? :slight_smile:

EDIT: to be clear. I am not saying you considered it as a big factor, just providing a counter argument.

1 Like

Agreed, something being easier to use sooner shouldn’t be a driving motivation, though I do think it’s worth thinking that adding syntax which is not backwards compatible with previous versions of the language is a negative trade-off, particularly when it’s syntax for something as central as defining modules.

Potential reasons why it’s a negative trade-off include that it requires overhead to remember when you can use syntax in relation to supporting previous versions, and people wondering why sample code they copied off the internet is erroring because defmodulep is unknown.

That doesn’t mean it’s not a worthwhile trade-off: I don’t think I could live without with.

However, since I alternately favor option D- which requires much more invasive changes to achieve warnings similar to what A achieves- it’s a larger factor in my consideration.

To expand a little on why I favor D over C, I agree with @brightball when he says:

In my experience, private methods and modules cause more problems than they solve. They increase logic duplication, complicate debugging and if developers are trying to access them…the developers usually have a good reason for doing so.

Warnings won’t prevent code from breaking if a developer chooses to access private functionality, but I don’t know that the benefits prevent access to private functionality outweigh the benefits of allowing access, especially when the benefits to allowing access are such that we’re already adding loopholes to work around the restrictions.

It is not a new syntax. It is just a new macro/function. For example, if we add a new function to Enum, you can’t invoke it on earlier Elixir versions either. Should we stop adding functions to Enum and other Elixir modules? :slight_smile:

There is even a proof of concept of the functionality working today.

Those are not the same warnings since we can guarantee those warnings with B, C, D, but not with A.

We are not allowing access because of external users, but for the authors of the private module themselves. It is a small distinction but it is an important one. Leaving the key under the carpet does not give permission for anyone else to get that key and enter your house - although you know it is a possibility when you do.