Boundary - enforcing boundaries in Elixir projects

Your Libraries & OS Mentoring Libraries
21

@justincjohnson Yup, umbrella allows to have multiple OTP apps in one meta-project with less ceremony (although opinions on that vary :)) than managing multiple projects altogether.

And indeed umbrella’s main concern isn’t validating cross-module deps i.e. what boundaries does - as stated in the official guide umbrella has drawbacks in that department.

What umbrella does that could be perceived as similar to boundaries is that it allows to split app code into higher level buckets compared to modules i.e. OTP apps and that it draws dependencies between those apps via deps with in_umbrella BUT it’s a very unreliable tool for guarding these dependencies because if app A always declares dependency on app B and app B doesn’t declare dependency on app A, it still can call code of app A without warnings because these apps are always present together in the build.

Therefore I rather see umbrella as tool for exactly what you wrote i.e. creating heterogenic releases made of different sets of OTP apps, than a code organization tool.

5 Likes
22

It would be a little bit hacky and I do not know if it would be good place for it, but you could “overuse” @moduledoc attribute with attributes (is it proper nomenclature?):

defmodule MyBoundary do
  @moduledoc exports: [
    # …
  ]
end
4 Likes
23

Folks,

I want to thank you all for providing the encouraging feedback and some great food for thought. Owing to your comments, I’m even more convinced that we can build a solid boundary enforcement tool for Elixir.

As a result of your comments, and the issues reported on the repo, I now realize that some of my original ideas were not on the right track. That’s actually a good thing! While I was a bit reluctant about sharing my work at such an early stage, I’m now happy that your review helped me discover some foundational problems so early on in the process.

So at this point, I’m increasingly leaning towards the following changes:

  • allowing partial coverage (not all modules have to be a part of some boundary)
  • switching to a decentralized solution
  • prioritizing nested boundaries (boundaries within boundaries)

I plan on writing a detailed proposal, but I probably won’t make it this week, so I just wanted to let you know that I’m intensively thinking about this.

25 Likes
24

This looks awesome!

To the tune of how boundaries can be awesome, we had a JavaScript project which was 1 codebase split across background and foreground pages (chrome extension). Implementing a boundary type system ensured that we never put background-only concepts in the foreground (where they would break). I caught several errors that I added in once this boundary checker was implemented.

Having a similar type of concept for organizing Elixir code would be really useful. I’m convinced that “people policies” are not enough, only tools can guarantee it. The compiler seems like the right place for it.

4 Likes
25

I finally found some time to make the proposal for the changes I’d like to do in boundaries. You can find it here. Looking forward to hear your feedback!

11 Likes
26

FYI, I started working on the proposed changes. See this comment for details.

9 Likes
27

The first draft of all proposed changes is now implemented in this PR. Note that some parts (most notably ignoring checks) are implemented in a somewhat different way. The relevant links are included in the PR description. I’ll leave the PR sit for awhile in hope of getting some feedback.

4 Likes
28

The PR has now been merged, and the repo has been renamed to boundary.

7 Likes
29

Now that Elixir 1.10 is released, I pushed the changed internals which rely on new compilation tracers. The new version requires Elixir 1.10, and the usage is slightly changed. See changelog and docs for details. I also decided to push the package to hex. The new version is 0.2.0. Those who are using the library but aren’t on Elixir 1.10 yet can use 0.1.0.

13 Likes
30

Started working on the support for managing external dependencies. The idea is that you can do something like this:

defmodule MySystemWeb do
  use Boundary, externals: [ecto: [Ecto.Changeset]]
end

to restrain ecto modules only to Ecto.Changeset (and Ecto.Changeset.* if such exist). Other external deps are unrestrained.

Demo:

image
image814×340 36.1 KB

The code is pushed to GH master, but not yet published on hex, since I’d like to get feedback on the interface. Docs for the new feature are available here.

8 Likes
31

I’m wondering a bit about the mixture of blacklists/whitelists. The keys of the :externals list form a blacklist, while the values are a whitelist of allowed modules per application. I would find for example use Boundary, externals: [ecto: [except: [Ecto.Changeset]]] to be clearer.

1 Like
32

Thanks! I agree that using only/except would make inclusion/exclusion property more explicit. I think in this case it should be ecto: [only: [Ecto.Changeset]], right?

3 Likes
33

Yeah, that’s even better.

34

From this call how does Boundary determine what calls are allowed? Does it look at the first part of the module (Ecto) or does it do something else such as look at all the modules in the OTP application that contains Ecto? For example if ecto defined EctoChangeset (instead of Ecto.Changeset) would it still be caught by Boundary?

35

First off, based on comments by @LostKobrakai, the interface is now ecto: [only: [Ecto.Changeset]] (or except: [...]).

This means that calls to, Ecto.Changeset as well as Ecto.Changeset.Foo, or Ecto.Changeset.Bar.Baz, etc are allowed.

Suppose you make the call to Ecto.Changeset.Foo.bar. Boundary will compare the callee module (Ecto.Changeset.Foo) with the allowed ones. If any of the allowed modules is a prefix of the callee, the call is allowed. Otherwise (e.g. EctoChangeset) it is reported as a warning (you can see this in the example screenshot I made earlier, where Ecto.Query.from is forbidden).

In the case of except it’s works the same, but if there’s a match the call is reported, otherwise it’s allowed.

36

But how does Boundary not flag a call to a separate dependency, such as a call to Plug.Conn.assign/3 as a warning. Does Boundary look at all modules contained within a dependency?

37

To make it less disruptive, boundary only considers calls to deps (apps) you’ve explicitly listed in the :externals list. So, considering the following example

use Boundary, externals: [ecto: {:only, [Ecto.Changeset]}]

Calls to modules from :ecto app are analyzed. Calls to non-listed deps (phoenix, plug, jason, etc) are always permitted. Finally, in-app calls are always analyzed.

Does that make sense?

1 Like
38

I would prefer it to be strict, thus to require us to list all external dependencies we want to use.

1 Like
39

FWIW, I’m actively thinking about strict, and would at least like to support it as a per-project opt-in. It would look something like:

defmodule MySystem.MixProject do
  def project do
    [
      boundary: [externals: :strict],
      # ...
    ]
  end
end

At which point you’d need to explicitly approve each external dep in every boundary.

I wouldn’t set this as a default b/c I feel it might turn out to be very disruptive and turn away most users. I’d like to keep boundary as relatively low ceremony for people starting with it, but at the same time I’d like to make it possible to introduce stricter rules for teams who want to enforce tighter control.

Another thing I’m considering is combining external boundaries together with regular boundaries. For example, consider the sample web boundary:

defmodule MySystemWeb do
  use Boundary,
    deps: [MySystem],
    externals: [ecto: {:only, [Ecto.Changeset]}]

I’m thinking about turning this into

defmodule MySystemWeb do
  use Boundary, deps: [MySystem, Ecto.Changeset]

So basically, we treat stuff from external apps as ad-hoc boundaries (in future we might also leverage deps boundaries meta if such exists, as proposed by @michalmuskala here).

The project-level :externals mode would determine what to do with unlisted boundaries. E.g. if you set externals: :relaxed (default), all calls to other apps are allowed, but if you include some boundaries from another app then only those boundaries are permitted. OTOH if you set externals: :strict, you need to explicitly approve every external dep. In most cases that would mean approving the top-level module (e.g. including Phoenix in the boundary list). Calls to Erlang modules would be completely ignored, at least in this phase of development.

Thoughts?

2 Likes
40

I can live with it ;), and yes I think it’s the most sensible option to go with, relaxed by default, strict by opti-in.

Despite I never used this Library it looks like a sensible approach to go with :slight_smile:

1 Like