I’ve run into trouble trying to build a release of my Elixir app using Distillery.
There’s a restriction on the maximum length of a filename that can be added to the release .tar archive. The release archive is built using Erlang’s :systools.make_tar which delegates to erl_tar.
I’ve tracked down the source of the 100 character limit. It’s the th_name_len constant configured in erl_tar. This is called by split_filename when writing out the header for each file added to the archive in create_header. The documentation for erl_tar suggests that it should be possible to include filenames greater than 100 characters, as long as they are less than 256 bytes.
For maximum compatibility, it is safe to archive files with names up to 100 characters in length. Such tar files can generally be extracted by any tar program.
For filenames exceeding 100 characters in length, the resulting tar file can only be correctly extracted by a POSIX-compatible tar program (such as Solaris tar or a modern GNU tar).
So it may be a bug in Erlang’s tar implementation that you cannot exceed 100 characters.
During release each Elixir module is built into a .beam file. Therefore you are limited to modules with names containing 100 characters or fewer. Including the module’s namespace and the standard Elixir. prefix. This is a problem if you use deep namespaces or when you implement protocols due to the length of the generated filename format: Elixir.Some.External.Library.Example.Protocol.MyApp.Foo.Bar.Baz.Qux.Example.Protocol.Implementation.beam.
In my app I have many instances of filenames exceeding 100 characters. It appears that my options are as follows.
Reduce the length of module names to ensure they are less than 100 characters.
Attempt to fix and submit a patch for erl_tar in OTP to support 100 - 256 character filenames.
Don’t use Erlang’s release mechanism to build the .tar package.
Add an error message to Distillery warning users of the problem when encountered.
Has anyone else encountered this issue? How did you circumvent it? I would expect people to stumble into this problem too. Unless it’s because I’m using longer namespaces than is typical.
Having had no response to this issue, I’ve decided to opt for reducing the modules names below the 100 character limit. That seems to be the path of least resistance. I’d prefer to have support for long namespaces, but the ability to deploy is a pressing concern.
I’m building a web application following domain-driven design and using CQRS/ES. It uses data from Strava (social network for cyclists and runners).
With the Elixir. prefix and .beam suffix to every module’s name to make the filename, the actual limit for a module name is 87 characters.
Here are four such modules from the 64 modules I had in my application that exceeded the limit. I’ve also split the project up by using an umbrella app. So each module is prefixed by the main app’s name, and then by each umbrella app (e.g. SegmentChallenge.Contracts).
Implementing a protocol (such as Elixir.Vex.blank and Commanded.Serialization.JsonDecoder) causes problems as the protocol module name is prefixed to your own module’s name.
This may be an unusual approach to building Elixir apps. I’ve migrated from writing .NET where namespacing is used extensively to organise code.
As a minor note here, I don’t see a lot of value in using the umbrella application name as part of the namespace. It isn’t an actual Elixir application itself, and if you’re choosing unique names for your application names to begin with, it doesn’t provide much value in my view.
Some of the umbrella apps are just libraries (e.g. authorisation, contracts). Some have their own application module and supervision trees (e.g. challenges, projections, web).
Sorry, misunderstood your point about dropping the SegmentChallenge prefix. I was concerned that my umbrella app names might clash with third party libraries on their own.
Just for reference, mentioning that since posting the above @slashdotdash has decided to make the project’s code source-available (though unlicensed). It looks to be refactored to no longer be an umbrella app.
