Deployment to Google Compute Engine

azimlord · February 25, 2019, 1:48am

Hey guys

I’m trying to deploy to GCP Compute Engine by following this tutorial
https://cloud.google.com/community/tutorials/elixir-phoenix-on-google-compute-engine

I’m unable to connect to provided external IP after create firewall-rules
There are no errors in following the tutorial. But cannot connect to http://${external_ip}:8080 after creating firewall rules

I copy hello

gsutil cp _build/prod/rel/hello/bin/hello\
    gs://${BUCKET_NAME}/hello-release

instead of

gsutil cp _build/prod/rel/hello/bin/hello.run \
    gs://${BUCKET_NAME}/hello-release

instance-startup.sh

#!/bin/sh
set -ex
export HOME=/app
mkdir -p ${HOME}
cd ${HOME}

RELEASE_URL=$(curl \
  -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/release-url" \
  -H "Metadata-Flavor: Google")
gsutil cp ${RELEASE_URL} hello-release
chmod 755 hello-release

wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 \
  -O cloud_sql_proxy
chmod +x cloud_sql_proxy

mkdir /tmp/cloudsql
PROJECT_ID=$(curl \
  -s "http://metadata.google.internal/computeMetadata/v1/project/project-id" \
  -H "Metadata-Flavor: Google")
./cloud_sql_proxy -projects=${PROJECT_ID} -dir=/tmp/cloudsql &

PORT=8080 ./hello-release start

gcloud compute instances get-serial-port-output shows

...
Feb 23 18:02:35 hello-instance startup-script: INFO startup-script: + PORT=8080 ./hello-release start
Feb 23 18:02:35 hello-instance startup-script: INFO startup-script: + ./cloud_sql_proxy -projects= hello -dir=/tmp/cloudsql
Feb 23 18:02:35 hello-instance startup-script: INFO startup-script: 2019/02/23 18:02:35 Rlimits for file descriptors set to {&{8500 8500}}
Feb 23 18:02:35 hello-instance startup-script: INFO startup-script: ./hello-release: 31: exec: /app/hello_rc_exec.sh: not found
Feb 23 18:02:39 hello-instance startup-script: INFO startup-script: 2019/02/23 18:02:39 Listening on /tmp/cloudsql/hello:asia-east1:hello-db/.s.PGSQL.5432 for hello:asia-east1: hello-db
Feb 23 18:02:39 hello-instance startup-script: INFO startup-script: 2019/02/23 18:02:39 Ready for new connections
Feb 23 18:08:08 hello-instance ntpd[656]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized

firewall rules

NAME                     NETWORK  DIRECTION  PRIORITY  ALLOW        DENY  DISABLED
default-allow-http-8080  default  INGRESS    1000      tcp:8080           False
...

Im not sure what information is needed to fix this
Please do ask for information and I will provide them.
Thank you

stefanluptak · February 25, 2019, 7:53am

./hello-release: 31: exec: /app/hello_rc_exec.sh: not found is probably something worth looking at. Do you know what it is?
Also try running ps aux | grep erl on the instance, to check if something is even running. I would say it’s not.

azimlord · February 25, 2019, 8:07am

Hey EskinMag

I will look at ./hello-release: 31: exec: /app/hello_rc_exec.sh: not found
and im not sure what it is

hello_team@hello-instance:~$ ps aux | grep erl
hello_t+ 23166  0.0  0.0  12784  1032 pts/0    S+   08:04   0:00 grep erl

stefanluptak · February 25, 2019, 8:26am

Ok, it looks like your app is not even running, so there’s probably no problem with the firewall.
Also, try to run the app with the foreground command instead of start, to see what’s going on (line PORT=8080 ./hello-release start will be PORT=8080 ./hello-release foreground)

azimlord · February 25, 2019, 9:01am

Feb 25 08:48:40 hello-instance startup-script: INFO startup-script: 2019-02-25 08:48:40 (41.8 MB/s) - ‘cloud_sql_proxy’ saved [7955768/7955768]
Feb 25 08:48:40 hello-instance startup-script: INFO startup-script: + chmod +x cloud_sql_proxy
Feb 25 08:48:40 hello-instance startup-script: INFO startup-script: + mkdir /tmp/cloudsql
Feb 25 08:48:40 hello-instance startup-script: INFO startup-script: + curl -s http://metadata.google.internal/computeMetadata/v1/project/project-id -H Metadata-Flavor: Google
Feb 25 08:48:40 hello-instance startup-script: INFO startup-script: + PROJECT_ID=hello
Feb 25 08:48:40 hello-instance startup-script: INFO startup-script: + PORT=8080 ./hello-release foreground
Feb 25 08:48:40 hello-instance startup-script: INFO startup-script: + ./cloud_sql_proxy -projects=hello -dir=/tmp/cloudsql
Feb 25 08:48:40 hello-instance startup-script: INFO startup-script: 2019/02/25 08:48:40 Rlimits for file descriptors set to {&{8500 8500}}
Feb 25 08:48:40 hello-instance startup-script: INFO startup-script: ./hello-release: 31: exec: /app/hello_rc_exec.sh: not found
Feb 25 08:48:43 hello-instance startup-script: INFO startup-script: 2019/02/25 08:48:43 Listening on /tmp/cloudsql/hello:asia-east1:hello-db/.s.PGSQL.5432 for hello:asia-east1:hello-db

This still the same

hello_team@hello-instance:~$ ps aux | grep erl
hello_t+  1109  0.0  0.0  12784   968 pts/0    S+   08:58   0:00 grep erl

How do you check if app is running?

Still cannot connect to the server

Edit: Found out ./hello-release: 31: exec: /app/hello_rc_exec.sh: not found is
The file is generated after initialize Distillery. It is stored in _build/prod/rel/hello/bin/hello_rc_exec.sh

stefanluptak · February 27, 2019, 8:03am

I am just guessing a bit, I am too quite new to this, but I would check if you have server: true in the Endpoint part of your prod.exs file.

I would also check if you have your app included in

release :name_of_your_release do
  # ...

  set(
    applications: [
      # ...
      your_app: :permanent
    ]
  )
  # ...
end

azimlord · February 27, 2019, 8:48am

My prod.exs do have server: true

config :hello, HelloWeb.Endpoint,
    load_from_system_env: true,
    http: [port: {:system, "PORT"}],
    check_origin: false,
    server: true,
    root: ".",
    cache_static_manifest: "priv/static/cache_manifest.json",
    include_erts: true

But my release don’t have my_app: : permanent
Currently my release look like this

release :hello do
  set version: current_version(:hello)
  set applications: [
    :runtime_tools
  ]
end

If Im correct

release :hello do
  set version: current_version(:hello)
  set applications: [
    :runtime_tools,
    hello: :permanent
  ]
end

should look like this

I will try to deploy again and update you

dazuma · February 27, 2019, 9:05am

Hi all. I’m the one who wrote the tutorial. Apologies for not being able to engage sooner. I’m abroad on vacation and my internet access is terrible, but I should be back online in about 5 days and can help troubleshoot at that time. If a solution is found, I’ll be sure to update the tutorial accordingly. Thanks everyone, and apologies for any errors!

azimlord · February 27, 2019, 9:20am

Awesome. Have fun on your vacation!

dazuma · March 5, 2019, 2:54am

@azimlord Hey, so I’m back. Thanks for your patience. I notice you said you copied hello instead of hello.run. Is there a reason you did that? I think that’s the issue. hello is just a wrapper script but not the entire release. hello.run is an executable archive of the entire release, and that’s what you want.

I just retested the tutorial myself, and it is working for me (if I use hello.run).

azimlord · March 11, 2019, 2:21am

Hey, thanks for the reply

Because Im not able to run this
PORT=8080 _build/prod/rel/hello/bin/hello.run foreground

I will try again. And update you

azimlord · March 11, 2019, 3:27am

Hey @dazuma,

I tried again an it still failing to connect using http://${IP_ADDRESS}:8080
The progress of instance creation is not showing Finished running startup scripts

set -ex
export HOME=/app
mkdir -p ${HOME}
cd ${HOME}

RELEASE_URL=$(curl \
  -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/release-url" \
  -H "Metadata-Flavor: Google")
gsutil cp ${RELEASE_URL} hello-release
chmod 755 hello-release

wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 \
  -O cloud_sql_proxy
chmod +x cloud_sql_proxy

mkdir /tmp/cloudsql
PROJECT_ID=$(curl \
  -s "http://metadata.google.internal/computeMetadata/v1/project/project-id" \
  -H "Metadata-Flavor: Google")
./cloud_sql_proxy -projects=${PROJECT_ID} -dir=/tmp/cloudsql &
PORT=8080 ./hello-release start

I already have my firewall up

NAME                     NETWORK  DIRECTION  PRIORITY  ALLOW     DENY  DISABLED
default-allow-http-8080  default  INGRESS    1000      tcp:8080        False

The machine type that I use, both is in asia-southeast1
Cloud SQL : db-f1-micro
Compute Engine: f1-micro

When I try to run

PORT=8080 _build/prod/rel/hello/bin/hello.run foreground
_build/prod/rel/hello/bin/hello.run: line 26: /Directory/To/Project/hello_root/tmp/hello/bin/hello: No such file or directory
_build/prod/rel/hello/bin/hello.run: line 26: exec: /Directory/To/Project/hello_root/tmp/hello/bin/hello: cannot execute: No such file or directory

NobbZ · March 11, 2019, 6:49am

What exactly are you copying? A single file from the place you put the screenshot from?

I’d try copy and extract _build/prod/rel/$APP_NAME/releases/$APP_VSN/$APP_NAME.tar.gz.

That should contain everything you need.

azimlord · March 11, 2019, 7:05am

From the tutorial

gsutil cp _build/prod/rel/hello/bin/hello.run \
    gs://${BUCKET_NAME}/hello-release

So I’m copying the hello.run file

NobbZ · March 11, 2019, 7:13am

What tutorial?

BrightEyesDavid · March 11, 2019, 7:40am

It’s linked to in the original post.

I didn’t know about Distillery’s --executable option and the resultant .run file, which self-extracts. Good to know.

azimlord · March 11, 2019, 8:58am

This tutorial from @dazuma. Deployment to GCP Compute Engine
https://cloud.google.com/community/tutorials/elixir-phoenix-on-google-compute-engine

You can see in the “Perform a production build” section

NobbZ · March 11, 2019, 8:58am

Ah, haven’t seen that link before.

Okay.

I took a look at the generated *.run.

It basically does untar the attached binary data and then runs the included script.

The only reason I can think of right now, is that there are permission issues OR there is already a folder /Directory/To/Project/hello_root/tmp/hello, which would make the script assume it had already untared itself.

@azimlord please make sure to that no such folder exists or use the environment variable RELEASE_MUTABLE_DIR to change the place where the script untars itself. It either untars to ./tmp if that var is not set or $RELEASE_MUTABLE_DIR/tmp if it is set.

azimlord · March 11, 2019, 9:50am

Execute this
PORT=8080 _build/prod/rel/hello/bin/hello.run foreground is now working.
No problem connecting to the Cloud SQL (remote DB)

There are actually already generated tmp/hello from my previous try deploying to GCP Kubernetes.
I removed the /tmp file and also _build file and redo the mix release.init

I deleted the old instance and also bucket from GCP and redo again. Still the same.
I could not connect to http://${IP_ADDRESS}:8080 where ${IP_ADDRESS} is my instance external IP

NobbZ · March 11, 2019, 9:56am

When you start your phoenix server, what IP is it binding to? You should see it as one of the first couple of messages logged by phoenix.

PS: Please also check netstat -lnpt | grep 8080 if the socket is bound properly and if your firewall is configured to accept connections on 8080.