🎬

Google Compute Engine: Getting Started

🎬

Watch it on YouTube

Google Compute Engine is one of the fundamental building blocks on Google Cloud. It lets you run Virtual Machines, autoscaling groups of Virtual Machines, create reusable templates, and more.

Is it good though? It depends. Google tries to create simpler products for the most commonly used cases, so Compute Engine is almost always not the optimal solution. However, Compute Engine is probably the most universal one. It gives you lots of control over its infrastructure, thus you can do anything you want within the extent of your knowledge.

And if you are just getting started with Google Cloud, then Compute Engine is the best entry point for you to learn how things work here. In this article, we will create a containerized application, deploy it to the Compute Engine and set firewall rules for it. So subscribe for more content and let's begin!

tl;dr:

Install docker on Mac: brew --cask install docker

Install docker on any other OS: https://docs.docker.com/engine/install/

server.js:
Dockerfile:
create.sh:
start.sh:
update.sh:

Create a containerized web server

The most basic and useful thing you can do on Google Cloud is to deploy a Containerized application.

First, create a very basic web server. I’ve created one using NodeJS.

const http = require('http')
const moment = require('moment')

const server = http.createServer((req, res) => {
  const headers = { 'Access-Control-Allow-Origin': '*' }
  res.writeHead(200, headers)
  res.end(moment().format('YYYY-MM-DD hh:mm a'))
})

server.listen(8080)

It's just a simple http server that returns current date and time. Now, let's wrap it into a Docker container.

Docker crash course

If you are not well familiar with Docker, here is a crash course. Docker lets you wrap your app into a container with its own tiny Operating System layer. This way, you can make sure that your app will behave identically on your local machine and in the cloud. Docker does more than this, but that’s a topic for another article.

image

Install docker

Now, go ahead and install it. If you are not working on a Mac, sorry, but you are going to have to suffer through this tutorial.

If you are on Mac, just brew it with a single terminal command.

brew --cask install docker 

Make sure the docker daemon is running. If not - start it.

Create a Dockerfile

A Dockerfile is a set of instructions on how to create a VM Image. And an Image is like a pre-built VM package. An Image becomes a Container when it runs on Docker Engine. Simple 🙂.

Use this template.

FROM node:12-alpine
WORKDIR /usr/app

COPY server.js package.json ./

RUN yarn --frozen-lockfile 

RUN apk add --no-cache tini
ENTRYPOINT ["tini", "--"]

CMD ["node", "server.js"]

FROM node:12-alpine - this is a base image. It usually contains the lightest possible version of OS you will need with some essential libraries installed. In my case, it’s alpine (al-py-ne) linux with NodeJS installed.

Here is how to choose the right one for you. Go to https://hub.docker.com/, search the official base image with the Engine you need, and read the docs. Most likely, you need alpine Linux because it’s the lightest of them all.

WORKDIR sets the working directory for any RUN, ENTRYPOINT, COPY and other instructions.

COPY server.js package.json ./ - this line copies your files to the container WORKDIR. You can replace it with a list of your files and folders.

Here you can specify all the dependency installation and build commands. Use this pattern:

RUN <a command you would use in your terminal>

tini (https://github.com/krallin/tini) is a tiny docker tweak that reaps zombie processes that can overflow your VM memory. tini also does some other useful stuff. Keep it because it’s certainly useful and, you know… tini 😄

Specify server start command at the end. In my case, it’s just node server.js.

And now let us build this image with docker build command

docker build . -t compute-engine-demo

The dot means lookup a Dockerfile in the current folder.

The -t option assigns a docker tag to this image, so we can refer to it later.

Now start it to make sure it works.

docker run -p 8080:8080 compute-engine-demo

The -p option exposes port 8080 and the compute-engine-demo is a reference to the docker image we just built.

Congratulations! You now know Docker! There are tons of ways to optimize this, but for now, this is enough.

Deploy container to Compute Engine

I can deploy it as-is, and Google Cloud would start it with the default docker run command. But, you’ll soon discover that you may need to run docker containers with additional flags, and this requires a different deployment approach. So, just use the more flexible approach right away.

Instead of writing the deployment commands one by one, I’ll write a Bash script, because in a real-life scenario, I'll need to deploy and re-deploy this application again and again after code changes.

Actually, I'll write two Bash scripts. The first one creates a new infrastructure, and the second one simply updates the VM.

The first script deletes the VM, if it already exists, then it creates a new one and sets firewall rules for it. Let’s go through it line by line.

set -e

PROJECT_ID=cloud-architect-demo
APP_ID=compute-engine-demo
GCR_ADDRESS="gcr.io/$PROJECT_ID/$APP_ID:latest"
ZONE=us-central1-a

gcloud auth activate-service-account \
	--key-file ./dev-key.json
gcloud config set project $PROJECT_ID
gcloud config set compute/zone $ZONE

gcloud services enable containerregistry.googleapis.com

docker build . -t $GCR_ADDRESS
gcloud auth configure-docker
docker push $GCR_ADDRESS

yes | gcloud compute instances delete $APP_ID || echo

gcloud compute firewall-rules create $APP_ID-firewall \
	--allow tcp:8080 \
	--target-tags $APP_ID-tag || echo
	
gcloud compute instances create $APP_ID \
	--image-project cos-cloud \
	--image-family cos-stable \
	--machine-type e2-micro \
	--zone $ZONE \
	--metadata google-logging-enabled=true,image-id=$APP_ID \
	--metadata-from-file startup-script=./start.sh \
	--tags $APP_ID-tag

At the start I configure this script to abort execution in case of any critical errors.

Then I declare variables for the sake of convenience and reusability.

PROJECT_ID=cloud-architect-demo
APP_ID=compute-engine-demo
GCR_ADDRESS="gcr.io/$PROJECT_ID/$APP_ID:latest"
ZONE=us-central1-a

PROJECT_ID is the id of my Google Cloud Project, do not confuse it with the project name.

APP_ID is whatever name I want to call my app and all the associated resources.

GCR_ADDRESS is the Google Container Registry address - the place where your container image is going to be stored. The Compute Engine VM will pull your containerized app from here.

The latest at the end is a docker tag. I’ll tell you more about it later.

gcloud auth activate-service-account \
	--key-file ./dev-key.json

Remember the service account I created earlier? Now I activate this service account to act using its permissions. I can also keep using my personal account, but the service account lets me run this script anywhere, in Continuous Integration pipelines, for instance. Also, its limited permissions ensure I’m not deploying stuff to the wrong project.

gcloud config set project $PROJECT_ID
gcloud config set compute/zone $ZONE

Set a project id and compute zone, as usual.

docker build . -t $GCR_ADDRESS

This line builds a docker image. The dot means to look for a Dockerfile in the current directory. The -t sets a tag for the docker image. The image is ready at this point.

gcloud services enable containerregistry.googleapis.com

If you are doing this for the first time on a new project, you’ll need to enable Container Registry API to push the container there. This command will enable Container Registry API for you. Or it’ll do nothing if this API is already enabled.

gcloud auth configure-docker

configure-docker command lets docker push your images to the Google Container Registry.

docker push $GCR_ADDRESS

And this line, well, pushes the image to the Container Registry.

yes | gcloud compute instances delete $APP_ID || echo

The delete command makes sure this instance doesn’t exist. It’s going to be useful when you need to completely replace your previous deployment.

If the demo server doesn’t exist, then delete command will throw an error. The echo part catches and prints it without stopping the deployment. If our VM does exist, this command will prompt you to confirm the deletion. The yes command automatically answers "yes" to any prompt.

Keep in mind that this kind of deployment induces downtime. It is possible to deploy a compute instance without downtime, but do not try to do it on a VM level. It’s not what VMs are for. There are different services for zero-downtime availability: Cloud Run, Kubernetes Engine, App Engine, or, at least, Compute Engine Instance Groups.

gcloud compute firewall-rules create demo-firewall \
	--allow tcp:8080 \
	--target-tags $APP_ID-tag || echo

Expose your server to the internet. This command creates a firewall rule to allow TCP traffic to port 8080 for all resources with $APP_ID-tag. And the echo caches the error in case you are re-deploying this app and this firewall rule already exists.

gcloud compute instances create $APP_ID \
	--image-project cos-cloud \
	--image-family cos-stable \
	--machine-type e2-micro \
	--zone $ZONE \
	--metadata google-logging-enabled true \
	--metadata google-logging-enabled=true,image-id=$APP_ID \
	--tags demo-tag

And finally, deploy the instance.

image-project and image-family define the Operating System this VM will run. In this case, I use the latest stable build of a Container-optimized OS.

machine-type defines the size of the VM. This is one of the smallest instances. It’ll cost $4 per month.

I specify the datacenter location in the zone argument.

I’ll need to add google-logging-enabled flag in metadata for Container-optimized OS to output startup logs correctly to the Cloud Logging.

I also set a custom metadata key image-id, because I'll need it later.

And the last argument is metadata-from-file. Earlier I’ve mentioned that you’ll need more granular access over your container. Mainly you’ll likely need to run the container with parameters. Thus, google Cloud doesn’t start the container on its own. We need to tell it how to start our app.

If at this point you feel like this deployment became out of hands complicated, remember, you’ll need to learn these concepts just once, and when you look at the whole picture, you’ll realize that it’s not complex. It’s just me going into all the intricacies to give you a full frame of reference on this topic.

Startup script

I need to tell Compute Engine how to start my containerized application. I'll create a bash script and point at it in the Compute Instance metadata. Compute Engine will spot it and execute upon instance boot up.

Let's create the start.sh file.

#!/bin/bash

export HOME=/home/app
mkdir $HOME || echo
cd $HOME
docker-credential-gcr configure-docker

PROJECT_ID=$(curl -X GET http://metadata.google.internal/computeMetadata/v1/project/project-id -H 'Metadata-Flavor: Google')
IMAGE_ID=$(curl -X GET http://metadata.google.internal/computeMetadata/v1/instance/attributes/image-id -H 'Metadata-Flavor: Google')

docker run --pull always -p 8080:8080 gcr.io/$PROJECT_ID/$IMAGE_ID:latest

A startup script file must begin with a shell path declaration: #!/bin/bash, so leave this part as is.

Root directory has limited permissions, so declare the HOME variable. Create a new directory with mkdir and navigate there with cd command. The echo catches and prints errors if I’m updating this VM and it already has the home directory created.

Give Docker daemon permissions to use the Container Registry.

Retrieve PROJECT_ID and IMAGE_ID from Google Cloud metadata. Do not hardcode these variables. This is going to be helpful when you have multiple environments like Production for end-users and Staging for internal testing.

And finally, write the docker run command. Remember I specified the latest tag in our deploy script? So, this is where I use it. If you specify anything but latest, this script will be pulling a specific image that you created with this tag. The latest tag is different. Google Container Registry may have only one latest tag per images group. So, each time you push an image with the latest tag, google unassigns this tag from all other images, and you end up having only one image with the latest tag.

That’s it. Now let’s deploy it. An ephemeral external IP will be assigned to my app. Let's verify that the server is online.

sh create.sh
curl <VM_INSTANCE_EXTERNAL_IP>:8080
> 2021-08-17 05:46 am

As you see, it responds with current date and time, as expected.

Now I’ll make a change to my source code and re-deploy it.

const http = require('http')
const moment = require('moment')

const server = http.createServer((req, res) => {
  const headers = { 'Access-Control-Allow-Origin': '*' }
  res.writeHead(200, headers)
  res.end(moment().format('YYYY-MM-DD hh:mm a'))
  res.end('Time is ' + moment().format('YYYY-MM-DD hh:mm a'))
})

server.listen(8080)

I’ll create a new script for this. It’s pretty much the same, except it resets the VM instance instead of deleting and creating it. It’ll still have downtime, though.

set -e

PROJECT_ID=cloud-architect-demo
APP_ID=compute-engine-demo
GCR_ADDRESS="gcr.io/$PROJECT_ID/$APP_ID:latest"
ZONE=us-central1-a

gcloud auth activate-service-account \
  --key-file ./dev-key.json
gcloud config set project $PROJECT_ID
gcloud config set compute/zone $ZONE

docker build . -t $GCR_ADDRESS
gcloud auth configure-docker
docker push $GCR_ADDRESS

gcloud compute instances reset $APP_ID

Let’s deploy our change. And verify it. As you can see, the server response changed.

sh update.sh
curl <VM_INSTANCE_EXTERNAL_IP>:8080
> Time is 2021-08-17 08:55 am

Alright, now you know how to deploy containerized applications to Google Compute Engine.

In the next articles, I’ll be showing how to deploy your app to the Compute Instance Group, so you can have Autoscaling and zero-downtime deploys without Kubernetes Engine complexity. I’ll also be showing how to cut 70% cost using a special kind of VMs, how to create self-destructing VMs, how to schedule VMs creation and more.

And this is just the Compute Engine.