PSA: Do not use private APIs, request a feature instead

Also, it would really be nice to have at least a warning when a private API is being used by a module outside its application. I remember that being discussed somewhere else but don’t remember the outcome of it, maybe that’s not even possible, IDK.


Wouldn’t it be better to be able to have truly private APIs built into the language so that compilation against “private” modules and functions would fail? Seems like something the language owners could add.

Something like a defpmodule macro that accepted either as part of the module def or possibly a well known constant, a list of module friends able to access internally public functions.


Yes, that would be the best approach, although it is not trivial to implement because of how modules are dynamic. Unless we take some liberties, such as, private modules always need to be required before using or something of sorts. Somebody would have to write such proposal. We should do it in another thread though.


Maybe, if you want to catch things like String.Tokenizer.tokenize(list). But if you’re happy with only detecting modules that are imported/used/required/aliased, then it’s pretty easy:

defmodule AvoidPrivateAPIs do
  require Logger

  def hidden?(module) do
    match?({:docs_v1, _, :elixir, _, :hidden, _, _}, Code.fetch_docs(module))

  def private_module_message(verb, this_app, that_app, this_module, that_module) do
    The module #{inspect(this_module)}, which belongs to application #{inspect(this_app)}, \
    has tried to #{verb} the hidden module #{inspect(that_module)}, \
    which belongs to application #{inspect(that_app)}.

  defmacro import(quoted_module, opts \\ []) do
    {that_module, _} = Code.eval_quoted(quoted_module)
    this_module = __CALLER__.module
    this_app = Application.get_application(this_module)
    that_app = Application.get_application(that_module)

    if this_app != that_app and hidden?(that_module) do
      message = private_module_message("import", this_app, that_app, this_module, that_module)
      Logger.warn(fn -> message end)

    quote do
      import(unquote(that_module), unquote(opts))

defmodule PrivateAPI do
  @moduledoc false

  def hey() do

defmodule TryToUsePrivateAPIsFromOutsideTheApp do
  require AvoidPrivateAPIs
  # Will log a warning, because String.Tokenizer is defined in the :elixir application
  AvoidPrivateAPIs.import String.Tokenizer

defmodule TryToUsePrivateAPIsInsideTheApp do
  require AvoidPrivateAPIs
  # Will not log a warning
  AvoidPrivateAPIs.import PrivateAPI

The example above shows only how to deal with imported modules, but the idea might work for other kinds of modules. There are probably some subtleties I’m not capturing here, of course…

I would say some of the issue stems from the fact that its not easy to be able to identify if a module is intended for public consumption or not. The elixir language is well documented and that makes it clearer, but it doesn’t make it black and white. How should one tell the difference between “maybe its missing documentation” and a module which is intended to be private?

Theres a larger issue to solve here, as has been mentioned. However, to address the problem at hand in the moment perhaps all internal modules should be marked as such. String.Internal.Tokenizer would make it plainly clear.

1 Like

Shouldn’t @moduledoc false just become the default behavior for undocumented modules? I’ve always wondered why documentation has been considered opt-out in ex_doc

1 Like

For the Elixir language itself it should be black and white. Elixir categorically marks all private functionality with @moduledoc false and @doc false. Those do not appear on the official documentation, which means their usage is being lifted directly from the source code.

To clarify, this has never been an issue in Elixir itself. Or we explicitly document it or we tag it as false. Sure, there are projects in the community where this issue will arise but it is definitely not the source of confusion in this case.


@josevalim: Please take a look at @lexmag response:

It was written in almost 2 years ago and still ExUnit.Diff has no public API. It would be helpful if you will be able to change it.

1 Like

Thanks @Eiji! This is a good example. I recommend you to start a discussion on Elixir issues tracker. We do not track comments or discussions on the forum as feature requests.

Note that it is unlikely that exporting ExUnit.Diff as is would be useful to you, as the current algorithm is recursive on itself, and you want to support more data types than the ones we do. There are different ways we could solve this, for example, by making the recursive function an argument on ExUnit.Diff, introducing a protocol, or by breaking ExUnit.Diff into smaller, reusable bits. So when opening the issue, please make sure that you provide a more complete proposal with examples on how you believe the ExUnit.Diff should be exported.


@josevalim would it be acceptable to detect the use of private modules only at the level of use/alias/import/require like I’ve done above?

I dont think you can handle the other cases (the use of modules that aren’t even aliased) without runtime checks.

I think this is a good example of why the pragmatic version of “Don’t use private APIs” is “Be aware of the risks of using private APIs”. Any feature that provides some kind of compile time check on private API usage should have an escape hatch, because there are situations where being locked to a specific version of the language and having a feature is sensibly considered a worthwhile trade off.

A tracking mechanism for private usage would actually make that sort of thing safer, since it’d be easier to audit.


While you really can’t stop someone from using “private” modules in the BEAM, you could do better compile time warnings by adding a new module attribute @use_in which has a list of modules in which the the “private” module is intended to be used.

This has the benefit of being explict.

That’s exactly what my code snipped above does, except instead of declaring explicitly where the module should be used, it assumes the module should only be used inside its application.

The big decision here is not how you specify where you can use a module, is whether catching only the places where it’s used, imported, etc is enough.

PS: I really like the @use_in idea. The only problem with the attribute name is that in elixir, use is a very loaded word.

1 Like

Wouldn’t it be more clear to explicitly tag it as private or unstable? Possibly with additional scary colors and “user beware” attribution in generated docs. This communicates exactly what you mean. It lets projects document private or unstable classes as well.

There’s a way other than @moduledoc false and @doc false to prevent docs from showing up in public distributions isn’t there? Like an exclusions list config for the mix task or something.

I know this is going to be a long shot, but has anyone considered releasing new versions of Elixir in a way similar to how the Rust team does it? They download and compile every single package on before they do a release. Not only does this help them make sure they don’t break things for everyone each release, but it also brings to light bad practices the community may be engaged in, like using Private apis. Findings like that can be discussed early on either to inform the community that they aren’t squared away, or to inform the dev team just how far reaching their changes really are.

If y’all are already doing this I apologize for being so far behind.


I’m fairly sure the Elixir team doesn’t do this, it seems like it would require a lot of extra work and processing power on their part. As of now, they are recommending that people add the master branch to their CI systems so that libraries will be built against master and the authors/Elixir core team can catch issues similar to what you are describing. I agree it would be nice, but without a big company like Mozilla backing Elixir, I’m not sure they have the infrastructure/time for it.

Fun idea. Made it my evening project.

Am running it on my laptop right now with Elixir 1.7.1 and Erlang 21.0.4 … let’s see what happens. If the community feels it would be actually useful, I could run it regularly on servers at work. Could also extend it with other checks, e.g. perhaps harness credo to do some static analysis.

It is pretty bare-bones … perhaps logging with json would be nicer for later usage. Also just noticed it fails for rebar projects, which should be easy enough to fix. So rough edges, to be expected for a few hours thrown at a project, I guess … improvements welcome :slight_smile:

It has already found packages with problems, e.g.:

 == 23:15:30.197
FAILED at compile: /home/aseigo/packages/ex_loader/0.4.1
errors: could not compile dependency :pre_commit_hook, "mix compile" failed. You can recompile this 
dependency with "mix deps.compile pre_commit_hook", update it with "mix deps.update pre_commit_hook" 
or clean it with "mix deps.clean pre_commit_hook"


edit: bonus round … Since I had them on disk already, I ran cloc over the entire set of packages.


It might be worth it to work something out with the guys where you can update project pages if you find vulnerabilities while doing the static analysis.

Yes, if I get this working sufficiently well I will definitely reach out to the team. It will be most useful if we can provide some feedback loops, including notice of build breakage, e.g.

1 Like