I received a very helpful reply from @gregvaughn via Slack:
"you’re using releases? That call in prod.exs to System.get_env is happening when and where the release is built. Not on the server it is deployed on.
It’s important to understand that .exs files are not compiled and therefore have no beam files in a release. Mix manages config/*.exs and distillery converts them to another form. You can set it as “${SECRET_KEY_BASE}” and set REPLACE_OS_VARS to true in distillery to get the behavior you intend"