Automation and deployment of Go applications to staging and Production environments using, Docker, Terraform, AWS and Github Actions. (Part 1)

Building APIs are no easy tasks, ranging from complex algorithms to CORS (every developers nightmare), developers are faced with difficult tasks daily to get the best product out there and solve problems. Another pain point experienced is deployment.

Deployment of applications especially to various environment like staging and production adds another level of complexity to this tedious process where we deploy and simply hope for the best that nothing with fail.

Deployment these days shouldn’t be subject to guess works, inconsistencies and human error with great tools out there such as docker for containerising and configuring our application, Github Actions for our CI/CD pipelines that automates the software delivery process, and terraform as our infrastructure as code tool which can help avoid inconsistencies in our cloud deployments, increase productivity, lower cost and limit human error.

In this part one, we will be setting up our CI/CD pipelines for a Go application (Go is beautiful, you should give it a try) using GitHub actions. Even though we will be using a Go application, I think the practices that will be discussed in here, can be applied to any programming language out there.

Enough of long boring talks, let’s see the pre-requisites for this part (Don’t worry, I will link awesome resources to get you up and running with each).

• Some knowledge of Github actions (You can get up and running here and here).
• Experience with a programming language. (We will use Go in this article).

Let’s get started.

Let’s start by preparing our repository on GitHub with some good practice.

Open Github and go to the settings of the repository with your code and click on branches (We want to create the develop, master and production branches first). We should have something like this:

We want to start by adding rules to our branches. Let’s do that by clicking on the Add rule button and add the following rules as shown below. (I will explain them)

Okay, the first step is to set the Branch name pattern. It is important to know that this represents the branch this rules will be applied to, in our case we are applying this rule to the develop branch.

The next step is to set the Require pull request reviews before merging rule. A couple of things to see here.

What this rule does, is to make sure, there is at least one review for every pull request made, we can increase the number of reviews required with the dropdown with a maximum of 6 reviews. Also we can use something called a code owner here to ensure that the person set as the owner of this code must review it before it can be merged.

NB: If you are the only developer on the project, you don’t want to use this rule as it will prevent your merge until there is an external review.

We also want to Require status checks to pass before merging. This rule ensures that there are certain checks that must be completed before there is a merge. Currently, it is not showing any checks to include because we haven’t setup any workflow to go through hence we will come back to this rule later.

The next rule is Require branches to be up to date before merging . This rule ensures that the pull request branch is using the latest code. In simple terms, we want to test the branch with the latest code in the targeted branch to merge into. We also need to have our workflows checks enabled before this takes effect. This also protects any direct push to any of the branches and you have to checkout to another branch. This is great because it prevents a bad code that could the entire application from getting merged directly.

Finally, we want to Include administrators in our branch protection rule, hence the protection rules also apply to administrators as well.

Then we can simply create the rule. (You can also learn about the other rules, I don’t use them but the docs will provide some context about them).

Also we want to repeat this task for the master and production branch. Okay, let’s do some workflow!

I would show you a great way to get started with creating workflows using the starter-packs provided. Click on the actions tab.

You can see it is suggesting to us already a workflow because of the repository codebase is in Go, that way, we can use the rich editor it provides us to setup this workflow. Let’s do that with some changes to the workflow.

name: CI

on:
  push:
    branches: [ master, production, develop ]
  pull_request:
    branches: [ master, production, develop ]

jobs:

  build:
    name: Build
    runs-on: ubuntu-latest
    steps:

    - name: Set up Go 1.x
      uses: actions/setup-go@v2
      with:
        go-version: ^1.15

    - name: Check out code into the Go module directory
      uses: actions/checkout@v2

    - name: Get dependencies
      run: |
        go get -v -t -d ./...
        if [ -f Gopkg.toml ]; then
            curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
            dep ensure
        fi

    - name: Build
      run: go build -v ./...

    - name: Test
      run: go test -v ./...

We made a change to the workflow to run on not just master but production and develop branch as well as name it CI (you can give it any name you want) and change the go-version to 1.15 (this should be your version of Go). Let’s explain what this workflow does!

1. We are running in the steps an action to setup go using the actions/setup-go@v2 predefined actions.
2. We use the checkout action to checks-out the repository under $GITHUB_WORKSPACE, so your workflow can access it. I must say, THIS IS A VERY IMPORTANT STEP, since without it, the workflow can’t access our repository without it, our workflow will definitely fail.
3. We got the dependencies, an important step to run the next step.
4. We are building the project here, and this is crucial to check because we don’t want to merge any branch that will break our application.
5. We also make sure all our tests are passing!

That’s it for our first workflow! We should merge (checkout to a new branch, I believe it is a good practice). Then you will see that our workflow action ran after merging to master.

Now we have our first workflow!

Let’s setup one more, now we will do it manually in the code editor. We will be setting up the linting workflow.

One of the reasons why I love GitHub actions is I can split my workflows into separate files. Let’s see this in action.

Pull the latest master branch to your local repository.

You should have at the top a .github/workflows directory which contains our go.yml workflow. Let’s add another workflow for linting, for this, we want to look for a good action as well in the marketplace, we are looking for golangci-lint.

We will be using the Run golangci-lint, it has the highest stars and very good contributors. The default YAML file provided is good enough as well to use for the linting checks of our application.

Go ahead and create another file in the .github/workflows directory and add the example snippet!

We just need some modification as we might not need multiple OS support and also the branches need to be updated (we will do that a lot).

name: golangci-lint
on:
  push:
    tags:
      - v*
    branches: [ production, master, develop ]
  pull_request:
    branches: [ production, master, develop ]
jobs:
  golangci:
    name: lint
    runs-on: ubuntu-latest
    steps:
     - uses: actions/checkout@v2
     - name: golangci-lint
       uses: golangci/golangci-lint-action@v2
       with:
         version: v1.29

cool!

Now, if we push our code, we will see the Go workflow run and after merge, we will see the golint workflow run as well.

Now, we have some workflows, time to make sure our branches are fully protected!

On GitHub, open the repository and click on settings and then branches.

Edit each of the rules!

Now, these workflows must be completed before we can merge a pull request and also it will make sure our branch is tested with the targeted branch code.

That’s it for this article! Next, we will be Containerising our application using docker and also setup up IAM for AWS.

Thanks for reading!