Switching versions of released app breaks after updates

We use releases for deployment and could activate any version that was released (has a directory under releases) using RELEASE_VSN environment variable as documented here https://hexdocs.pm/mix/Mix.Tasks.Release.html.

That worked fine up until we upgraded the system runtime (erlang and elixir packages). After upgrade no versions that were compiled earlier are working anymore. The errors look like that:

(no logger present) unexpected logger message: {log,error,"~s~n",["beam/beam_load.c(1883): Error loading module 'Elixir.Connection':\n  This BEAM file was compiled for a later version of the run-time system than 22.\n  To fix this, please recompile this module with an 22 compiler.\n  (Use of opcode 169; this emulator supports only up to 168.)\n"],#{error_logger=>#{emulator=>true,tag=>error},gl=><0.0.0>,pid=><0.514.0>,time=>1602140276214009}}

The problem has nothing to do with the specific Connection module, we also had it with Elixir.MIME.

My assumtipn was, that releases encapsulate the runtime and I know it is at least partially true cause the binary that runs virtual machine is located in the releases directory. Should’t starting an older release be the same no matter if newer releases use newer runtime or was that a wrong expectaion? If that’s not normal then what might be a way to fix it? Also why “compiled for a later version of the run-time system” when it’s something that was compiled earlier and actually used older runtime before upgrade? The error seems a bit backwards.