I’m considering using a single Github repository that will contain the code for more than one mix project, where each mix project is hosted on hex. I think it is a good idea for my usecase but wondered if there were any negatives I had not considered. Rather than discuss my usecase which is a project very much under development It might be easier to explain in the context of a more established project such as ex_aws.
Currently ex_aws has a single lib directory in which there is the code for iteration with many amazon services such as SNS S3 EC2. If following SemVer then a breaking change to any one of these pieces of functionality will require a major version change. However the majority of users are probably only using a fraction of the services available.
What would be the benefits/costs of changing to the following structure?
In this case a breaking change to ex_aws_ec2 could be released to hex without changing anything about ex_aws_s3.
In addition keeping them all together allows a single readme etc for documentation about the whole project.
For those whole are familiar I think I am looking for a way to implement something similar to the features feature available in the Rust ecosystem
Hex doesn’t really care if the project is in source control at all. When publishing you’d cd into the appropriate directory project and run mix hex.publish from there. Other than that I believe it should “Just Work™”.
Off the top of my head, the cost would be maintaining dependencies of each mix project, and probably difficulty of having shared code/config without duplication (extra package like ex_aws_utils?). But that can be mitigated for example by having an ex_aws package and separating the ex_aws_s3 and other specific features (kinda like ueberauth, ueberauth_google, ueberauth_facebook, etc.)
The benefit would be what you said: the flexibility of independent project versioning.
All in all, I think it’s a matter of trade-off: which one are you willing to put heavier attention to. Hex allows you to do that, but should you?
I assume that one thing that might be lost is that you would not longer be able to use git dependencies. Unless someone can correct me and there is a way to depend on a sub folder of a git repo.
def deps do
[{:plug, git: "git://github.com/elixir-lang/plug.git"}] # anyway to make the dependency a sub directory?
end
Personally I like splitting up a project. I do it with benchee, there is a core and there are formatters which are separate packages and repos (html etc. ).
I like to have them in completely separate projects/repositories if they are optional and not highly dependent on each other especially if you may update one without the other. There mostly is one core library that had all the general information and the others just the specific one then. Little downside is you have to duplicate stuff like the CoC and License.
However, if the libraries are highly dependent on esch other and are mostly updated together, lime rails, then the mono repo is good imo. I also do that over at shoes4 (Ruby GUI).
I think that my case matches the one you described where the parts rely on each other tightly. In my case I have some core functionality for handling web request and second projects that are adapters(i.e. for cowboy, elli etc). I don’t want the adapters in the same mix project as that makes the core project depend on every library the adapters use and yet keeping them in the same repo is helpful as every time the core changes the adapters normally also need to change.
I haven’t tried it for a library project, but to me starting with a monorepo sounds like a pretty good idea. Especially if we’re fairly certain that we’ll end up with multiple distinct packages - but at the same time initially we want to be able to iterate very quickly (and not worry about syncing code, releases etc.) And then as the project matures we have an option to keep it as monorepo or break it up (e.g. for having separate issues, releases etc).
Speaking of releases, handling git tags in your monorepo might be problematic for your library project - either have just one version for the entire thing: v0.0.1, v0.0.2, or foo-v0.0.1, bar-v0.0.2 which seems unwieldy.
As far as git dependencies, indeed it’s a limitation of Mix AFAIK. Would be nice if apart from declaring dep as {:foo, github: "acme/foo"}, we could declare path, perhaps: {:foo, github: {"acme/acme", "apps/foo"}}
@benwilson512 I take your argument but let me just try the following counter arguments.
You are dog fooding the shared part of the API in a package called ExAwsApi for example
Nearly every other package depends on it.
ExAwsApi can be updated with a breaking change for the purpose of convenience or efficiency. Every other project in ExAws needs changing but as they do not change there own API need only a patch/minor version update.
@wojtekmach
My solution to the tags is probably to not use them. They are too coarse for tracking a monorepo, I’ve head people express that position when using mono repos in a variety of situations.
@OvermindDL1
I would happily tackle that PR over the coming holidays. Does that warrant an issue on the language repo? Is there a way to mark this forum topic as language/mix feature discussion
The argument for independent versioning seems inherently flawed to me. Looking at various projects that are maintained as several packages I see that they usually strive for having the same version anyway. Having different versions is extremely confusing for the consumer of the package. Starting with the fact they they can’t simply express what version they use when they find bugs or face some issues. Without git tags, I also can’t easily find the code for my particular version, which is very important in many cases.
I see this happening with ecto and phoenix_ecto all the time (ecto is at 2.0, phoenix_ecto at 3.0) - and those are just two packages. I can’t imagine what mess that would be with 10+ packages.
“Compatible” package versions is a headache to be sure… I agree that for tightly coupled ones they should be kept the same (so does rails and so we do at shoes).
If the use case is more of a plugin nature, aka I have one main library and different libraries that support its functionality I think different versioning is fine. Hope/goal being that the API between them becomes stable enough so that they don’t need to be updated just because (like it is often in rails with lots of gems being updated to a new version with no changes).
I.e. that’s what I try to do with benchee, there is a core library and the rest of them are plugins. Sometimes there need to be releases together as pre 1.0 breaking changes occur. Sometimes releases occur together because I want the new features from benchee “core” - the long term plan however is that they can evolve rather freely and independently in a 1.x world
