How to set environment variables and secrets in Google App Engine (GAE)?

woohaaha

I am deploying a Phoenix app to Google App Engine (GAE). I am using elixir releases and am following this tutorial (https://cloud.google.com/community/tutorials/elixir-phoenix-on-google-app-engine). Once I deploy it appears that GAE reads an app.yaml file and then builds a container with the release inside it.

Setting secrets and environment variables has been a bit tricky. I sure enjoyed the way Heroku does it. It seems that GAE recommends making an http request to get those variables/secrets from another service. I don’t know enough about elixir releases to know where/how I’d make this http request and set the variables in this App Engine build process. Has anyone here successfully set environment variables in an App Engine Phoenix app? I’d love to hear how you did it. (If I had a preference I’d prefer not to make an HTTP request, and I’d also prefer not to compromise on security by setting them in the app.yaml)

Thank you

9 comments

/phoenix #deployment #environment

1 3486 9

2021-07-08 12:19:00 UTC

Showing Posts 1 to 9

Show Best Posts
Show All (oldest first)
Show All (newest first)

Francesco

@woohaaha did you ever figure this out? I’ve run into this hurdle now and it’s really not clear what to do. The only way I see so far is to use the env_variables field in app.yaml and run gcloud app deploy but then you can’t check app.yaml into source control… I’m very confused about this whole thing.

Post #1

woohaaha

Not the answer you’re looking for but I decided to use Docker on AWS with ElasticBeanstalk since I could abstract the app from the platform in a way I could understand. I chose to go to AWS because Google gave me some sticker shock on running two instances, and also it seems AWS is the road better traveled (lots of docs/examples).

Post #2

hubertlepicki

so it’s my turn to ask the same question you, @Francesco . Did you figure it out?

I am thinking of not doing it but instead using Secret Manager Secret Manager | Google Cloud
This seems like preferred way of doing things there.

Post #3

sezaru

I don’t know about GAE, but you can run HTTP requests or whatever inside a runtime.exs, that is what I do to communicate with Hashicorp Vault to retrieve secrets, etc only when my node starts.

Post #4

Francesco

Hey I actually did sort something out a while ago.

In the application.ex start method I call a method called load_google_service_accounts. This then authenticates with Googles metadata server using the Goth package (as long as you are running on a google service such as GAE). Under the hood this uses the service account associated with the running service which it can do securely since you are on Googles infrastructure. Once authenticated you will get a token (from Goth) that you can use with the Google Secrets Manager API. You then fetch the secrets you want, and for easy access in the rest of the app you can store them in persistent_term.

  defp load_google_service_accounts() do
    # Load the secrets from Google Secrets Manager for use in relevant processes
    {:ok, %{token: token}} = Goth.Token.for_scope(@secret_manager_scope)

    {:ok, %{ payload: %{ data: secret_data }}} = GoogleApi.SecretManager.V1beta1.Connection.new(token)
    |> GoogleApi.SecretManager.V1beta1.Api.Projects.secretmanager_projects_secrets_versions_access(@google_project_id, @secret_name, "latest")

    decoded_data = secret_data |> Base.decode64! |> Jason.decode!

    :persistent_term.put(:secret_name, GoogleServiceAccountKey.new(decoded_data))
  end

Hope this helps

Post #5

hubertlepicki

Yep, this is perfect and very much what I was trying to do.

I think you could also do it in runtime.exs or you had problem with google apis lib not being a started app?

Post #6

Francesco

Glad to hear it. I vaguely remember trying to get this to work in runtime.exs first and having issues but this was a while ago so I can’t remember exactly what I had tried.

I call the load_google_service_accounts method before I call Supervisor.start_link(children, opts) in start so I think it essentially achieves the same thing as doing it in runtime.exs without the limitations of running code in that file.

Post #7

hubertlepicki

yeah, I suspect it’s the same issue I am having. This code gets executed early before the libraries/otp apps are loaded. Anyway, thanks for sharing, I’ll figure something out!

Post #8