Making sure libraries CI workflows is kept up-to-date with Elixir and Erlang/OTP releases

What do you guys think could be a good way to prevent the last official Elixir version (patch or minor) from not being add in the CIs after being launched due to forgetfullness?

The following are two of the simplest and more palatable approaches that I have considered:

  • a scheduled CI job whose sole purpose is to fetch the last official Elixir version (e.g., asdf list-all elixir | grep -vP "master|otp|rc" | tail -1) and check that the CI YAML (e.g., .travis.yml or .github/workflows/main.yml) contains it, failing with a message such as “please upgrade your pipeline to include Elixir version X (in a separate PR preferably)” if it does not
    • more explicit and generous with the user, instructing her to update the CI YAML
    • tends to keep conformity with the intermediary minor Elixir versions (IMEV) rule
    • could be adapted to include release candidate versions as well (better as a test of the language)
    • jobs can be scheduled on GitHub Actions, CircleCI, and Travis CI
  • include master as one of the Eliixr versions
    • less explict and does not give instructions regarding updating the CI YAML, leading the user to fix what caused the failure in case of failure
    • may lead to failure to conform to the IMEV rule if the CI YAML is not updated for long

PS: feel free to move (or suggest moving) this specific discussion elsewhere

1 Like

I think that there are a few suppositions baked into this question that need more discussion/evidence to support:

  • Compulsory support of the latest Elixir/Erlang versions is intrinsically good
  • Maintainers are overlooking those releases which are currently a few times per year
  • Libraries need to support new language versions as soon as possible even if not otherwise being maintained/revised
  • Changes will be needed to use a library that works on version N in order to work on version N+1

I find the whole idea a little bit puzzling, tbh. Is it just to get ahead of the requests by library users who are also early/immediate adopters of a new language version? Why not let those requests be driven by actual need and otherwise assume that the working code remains in a working state? The core team makes a lot of effort to uphold backwards compatibility and to clearly communicate when deprecations or breaking changes arise.


For my libs I work on a strategy that is:

  1. Test on the latest release of each major version of Elixir that is supported
  2. Plus Elixir master when it gets to Beta phase. I check on compatibility during the development cycle but only see if there’s any major incompatibilities in my code (usually deprecations I need to manage).

I don’t find it a big challenge since:

  1. Elixir guarantees compatibility for patch releases
  2. New releases don’t arrive as surprises or often

Therefore I don’t place a priority on automating updates to my test matrix.


Could you please reconcile your arguments with the fact that many of the most popular libraries, including ones mantained by the Elixir Core team, are not kept up-to-date and I do not mean patch, but minor version and major OTP versions for years?

@adrianomitre I’m not quite following you, can you give an example?

@kip Sure:

Testing on a particular version combination on CI implies some degree of support for those versions will be upheld, even if nothing is stated outright. Open source maintainers are humans with finite time and other responsibilities and hobbies too, so I can understand why they might limit their scope to avoid that implication of support.

It’s up to a given library’s author(s) to decide how much extra diligence they support for forwards and backwards compatibility. There’s no need to assume incompetence, negligence, or hostility if a particular library isn’t testing as comprehensively as you yourself would like. The neighborly way to react is to open issues when you find incompatibilities for environments that aren’t testing in CI.

If the authors are finding enough volume of these issues in aggregate, they will likely either make a clear stance on their compatibility guarantees or start to include more versions in their testing matrix, according to their needs and time. That’s the most we can ask of OSS maintainers, IMO.