How to automatically deploy new code to Kubernetes, triggered when you push to a special Git branch
In this article I’ll describe a simple way to implement continuous deployment from GitHub to a Kubernetes cluster. We’ll use CircleCI to automatically update production and staging deployments on Kubernetes whenever we push a new commit to specific branches in GitHub.
This workflow is a starting point for simple applications that don’t require advanced features like blue/green deployments, canary testing, or automated rollbacks.
For more advanced deployment scenarios, look into Spinnaker or CodeFresh. These systems are more flexible, but also far more complex than the one in this article.
Git Branch Setup
Assume we have three branches in our Git repository:
- dev for work-in-progress development
- staging for code that should run in the staging environment, and
- prod for code that should run in the production environment.
New code is written in feature branches forked from dev. From there, pull requests are submitted back to dev for review. Once a pull request is merged into the dev branch, it will run integration tests.
We’ll assume that we have already set up continuous integration testing using a CI platform like CircleCI. See the CircleCI documentation to get started.
When we deem the dev branch ready for live testing, we promote the code by merging it into the staging branch. This is when we want to trigger an automatic update of the Kubernetes staging deployment.
Similarly, when we are confident that the staging application is okay, we finally promote the code by merging it into the prod environment. We want this to trigger an automatic update of the Kubernetes production deployment.
Kubernetes Setup
Assume we’ve already created some Deployment resources in Kubernetes to host our application. (how to accomplish this is outside the scope of this article — see the Kubernetes documentation for details).
The CircleCI automation works by updating the Docker images referenced by these existing Deployments. When an image reference changes, Kubernetes will take care of spinning up new pods with the new image and spinning down the old ones.
myapp-deployment.yaml (see code below) is a sketch of what the Kubernetes Deployment spec could look like. One key piece is the “imagePullSecrets” section, which includes the credentials to pull images from the Docker registry. If you’re using a registry other than Google’s Container Registry, check its documentation for what to put here.
By the way, I recommend using Helm to maintain Kubernetes resources and Terraform to manage static resources outside of Kubernetes, like message queues and storage buckets.
Choosing a Continuous Integration Service
I prefer CircleCI for experimental projects because its free usage tier supports private GitHub repositories, and it comes with a substantial amount of free build time.
Google now offers its own Docker build service (Google Cloud Build), but you might prefer CircleCI for reasons such as better integration with GitHub, the ability to run build/test steps outside the Docker environment, or to take advantage of its free usage tier.
Choosing a Docker Container Registry
We’ll need somewhere to store Docker images after they are built, so that Kubernetes can deploy them. I prefer Google’s Container Registry for two reasons: first, its access permissions are integrated with other Google Cloud services, and second, we can pull images from the registry into Google’s Kubernetes Engine without external network transfers, which incur extra bandwidth costs.
If you prefer another registry, like Docker Hub, it’s trivial to adjust the code to push and deploy from it.
Choosing a Kubernetes Platform
Most cloud providers now offer some form of managed Kubernetes cluster. I find Google’s Kubernetes Engine the easiest to set up and maintain. If you’d like to try running Kubernetes on AWS, check out my introductory guide to Amazon’s EKS.
Note, if you’re on a different Kubernetes platform, you will need to replace the gcloud authentication steps in the CircleCI configuration to pass credentials using the appropriate method for your platform.
CircleCI Configuration
All of the “magic” happens in the single configuration file that drives CircleCI. I’ll include the full file here, then describe its individual parts below.
CircleCI Workflows
There are two independent workflows in this CircleCI configuration:
- build_and_test is a normal suite of continuous integration tests.
- deploy includes two consecutive job steps: docker_push to build and push Docker images, followed by kubernetes_deploy, which updates live Kubernetes cluster deployments with these images.
Environment Variables
The docker_push and kubernetes_deploy jobs require some configuration variables and credentials to access our Docker registry and Kubernetes deployments. The best way to handle these is by setting environment variables in CircleCI. You can simplify management of variables with a handy tool like EnvKey, or enter them manually in the CircleCI web interface.
Our CircleCI workflows require the following environment variables:
- GCR_DOCKER_REGISTRY_PASSWORD is a static credential granting access to push images to the Docker registry. In Google’s Cloud Platform, this takes the form of a JSON-formatted key belonging to a Service Account with write access to GCR. (in a later article, I’ll describe how to set up these Service Accounts).
- GKE_CD_SERVICE_ACCOUNT_KEY is another static Service Account credential. This one has permission to update the Kubernetes deployments.
- GCR_PROJECT and GKE_PROJECT are the Google Cloud projects used for the Docker registry and Kubernetes cluster.
- GKE_CLUSTER and GKE_ZONE tell the Google Cloud tools where to find our Kubernetes cluster.
In addition to these manually-configured environment variables, the scripts also reference CIRCLE_BRANCH and CIRCLE_SHA1, which are provided automatically inside the CircleCI environment, and refer to the Git branch name and commit hash of the code currently being processed. We use these variables to tag the Docker images for later reference, and to distinguish between dev, staging, and prod environments.
Security Note
Our CircleCI environment will have credentials with permission to change the Docker images used by our live production service. We must be extremely careful to prevent unauthorized access to these credentials. At a minimum, ensure that CircleCI only executes tests on closely-supervised Git branches, and ideally, also limit the scope and lifetime of CircleCI environment credentials (EnvKey is handy for this).
Happy Deployments
I hope this outline helps speed up your experiments with Kubernetes. Please drop me a line if you find the information useful!
