pnezis

pnezis OP

Having to update multiple repositories (version bumps) for a change on a single package breaks the atomicity

If 2 people need to work on the same file they will need to work on the same file even if it is on a separate package.

We don’t apply any privilege control and I am not of the philosophy that employees should have limited visibility. All our employees get access to all our gitlab repos (>100). Also scaling is not an issue, even Google, Microsoft Facebook have mostly monorepos and their codebases are huge. A good read on monorepos: https://monorepo.tools

That’s correct

I agree and that is the reason I want to avoid an umbrella. And this is also the reason I don’t want to split it into multiple services on their own repos. With path dependencies on the same repo and independent mix projects you solve all of these issues. With the proper tooling you can also speed up the CI pipelines no matter how big the codebase is.

1
Post #11
D4no0

D4no0

Then the only way is just to put all projects in the same repo, using the domain logic as a library, referencing it by relative path and for runtime interaction to use any tools for communication.

As for keeping lock and deps in the same folder doesn’t seem to be the best idea, you will lose the flexibility of running some of the libraries with older versions and because elixir library dependency management sucks it will only get worse with time.

0
Post #12
c4710n

c4710n

Besides umbrella, there’s also poncho which is introduced by Nerves community. You can check it out.

There are also some topics in this forum, you can use the “site search” for relevant information. The keyword is “poncho”.

1
Post #13
mgibowski

mgibowski

Your set up seems sound to me.
I don’t have much experience with that, but if you wanted to take it further (and have time/resources for that), you could investigate adopting Bazel.
I imagine you could declare dependencies between your projects and build them in parallel + get caching benefits.

There is a related thread:

0
Post #14
pnezis

pnezis OP

Poncho is the closest to what I want to achieve, my only concern with this is the different deps and build paths per project.

What I want to achieve is something like the cargo workspaces in rust

Cargo offers a feature called workspaces that can help manage multiple related packages that are developed in tandem.
A workspace is a set of packages that share the same Cargo.lock and output directory.

3
Post #15
lafka

lafka

Did you end up using a monorepo structure?

I’ve just started an experiment exmonorepo to validate if mix can use a monorepo with directory structure similar to yours where the elixir apps/libraries are automatically discovered., I would be interested to learn about your experience implementing monorepos in elixir.

0
Post #16
pnezis

pnezis OP

We are still in the process of transforming a very big codebase to a mono-repo. So far it works great.

At the same time I am in the process of creating a library for managing elixir mono-repos which I plan to open-source soon.

0
Post #17
D4no0

D4no0

Have you decided to go with umbrella projects, or a different solution?

1
Post #18
pnezis

pnezis OP

Plain mix projects with path dependencies under a single git repo.

3
Post #19
lafka

lafka

ping me once you publish something. Are you manually adding the paths between the different monorepo applications or do you have some way of discovering apps? Does your monorepo manage non-beam code as well?

2
Post #20