Todd Rafferty's Blog

Self Compiling Go with Docker

Posted on
#docker #go #golang

Imagine a self contained development environment that could detect that there’s a file change on my file system, kill an existing go binary, rebuild the go binary, and then, launch a new process.

Introduction

I believe that setting up a docker container that self compiles my Go source upon changes within a local development environment will assist myself and colleagues iterate faster. I am a remote engineer with a mix of other disciplines on my team that are new to the language. The goal was to make a reproducible development environment extremely productive for either end of the spectrum. Enter docker. With docker and docker-compose, I can build, tear down, and recreate the entire development environment with single command, but could it be smarter? Could it be used to automate the tedious? Could it be reproduced across other developer machines?

After learning docker and trying out various approaches, I came across Reflex which wraps fsnotify. My approach was to get things working outside of the container first, to make sure that I understand how everything works, and then move the pieces into a container and get it working. Outside of the container, I could get reflex to listen to file changes on any go files within a directory. However, within a container I ran into limitations. I determined that it would be more performant to listen to the changes of 1 file, than entire sub-directories of potential matches.

So, what we’ll need as I guide you through this:

  • Some Linux knowledge. Familiarity with bash scripting.
  • Docker experience. Familiarity with the docker and docker-compose tools.
  • New docker beta. This is important because it’s using the native virtualization engine of the operating system instead of relying on virtualbox. Just trust me. It’s faster this way.
  • Understanding of Go. Familiarity with Go environment, compiling, and coding.
  • golang on dockerhub
  • Reflex which uses fsnotify internally.
  • Please note, I’ve tested this on OSX. I haven’t tested this on linux / windows, sorry!

Please note, everything you’re about to read is for local development environments. This isn’t meant to be a deployment strategy or for production usage.

Base Container Setup

First, we need a go environment within a docker container. Fortunately, there’s one already available to us on dockerhub. For this post, we’ll be using the alpine distro because it’s super small, but there is a debian based one available as well. There are no changes needed to switch between distros. Within the docker container, the $GOPATH is /go which means the go environment is right on the root path of the server.

We need more on this container though because while it has the Go environment on it, it doesn’t have everything we need to watch for file changes within our project. This is where reflex comes in. Reflex is a small program that is written in Go that will notice changes on our local file system and kick off a shell script within the docker container for us.

Base Dockerfile:

# Pull the golang version.
FROM golang:1.7-alpine
ENV GOBINARIES /go/bin

# Fix the DNS issue, this happens at raff's house.
RUN echo 'hosts: files [NOTFOUND=return] dns' >> /etc/nsswitch.conf
# Setup reflex env
ENV REFLEXURL http://s3.amazonaws.com/wbm-raff/bin/reflex
ENV REFLEXSHA dee8f77fac8c873c709117df6ebe4467fc9f57ed3339105d308f787e9b94059c
# Install reflex
WORKDIR $GOBINARIES
RUN wget -q "$REFLEXURL" -O reflex &&\
    echo "$REFLEXSHA  reflex" | sha256sum -c &&\
    chmod +x /go/bin/reflex

Brief explanation of what’s going on in this Dockerfile. We’re pulling golang:1.7-alpine, downloading a pre-built version of reflex. We’re avoiding building reflex on the container itself to make sure we have a reproducible environment and to avoid go get issues.

This is a pretty good base image. Each project we work on is probably going to be different in terms of path and configuration. My recommendation is to keep this base image lean so you can use this for different project configurations. The above has already been provided for you on dockerhub.

Building on top of the base image with further configuration

We did a lot of simplification above, and that base image has been built, tagged, and pushed to dockerhub. The above dockerfile is documented in case you want to make modifications to the base image and put it within your own docker hub environment.

FROM wbsl/go:1.7
# APP SPECIFIC ENV
ENV BUILDPATH /go/src/github.com/WreckingBallStudioLabs/SelfCompilingExample
ENV TOOLS /go/_tools

ENV PORT 8080
# DOCKER / APP PORT
EXPOSE $PORT

# Make directories and add files as needed
RUN mkdir -p $TOOLS
ADD build.sh $TOOLS
ADD reflex.conf $TOOLS
RUN chmod +x $TOOLS/build.sh
# Execute reflex.
WORKDIR $BUILDPATH
CMD ["reflex","-c","/go/_tools/reflex.conf"]

Breaking down the above dockerfile, we’re pulling the base image FROM wbsl/go:1.7rc6 that was created above. We’re setting environment variables and setting a port to 8080. Creating a /go/_tools directory and then adding our build.sh and reflex.conf to that directory. So, let’s pause here for a second. This entire environment depends on reflex kicking off a build script for us.

Here’s the content of build.sh:

#!/bin/sh

set -e
echo "[build.sh:building binary]"
cd $BUILDPATH && go build -o /servicebin && rm -rf /tmp/*
echo "[build.sh:launching binary]"
/servicebin

build.sh is removing a previous binary, if there is one. It’s going to do a change directory into the $BUILDPATH defined in the environment/dockerfile. It’s going to go build -o another binary into our $BUILDPATH, clean-up the /tmp directory afterwards to keep the container size down, and finally, it’s going to execute the binary.

Let’s take a quick look at reflex.conf file that reflex is going to use as a configuration.

-sr '\.build$' -- sh -c '/go/_tools/build.sh'

Reflex is going to run as a serivce, and if a file named .build has changed, run the build.sh script. We’re very close to starting this up. We’re just missing a sample Go file to modify.

Basic main.go example:

package main

import (
	"fmt"
	"log"
	"net/http"
	"os"
)

func handler(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Hello World!")
}

func main() {
	// Get the port from the OS ENV.
	port := ":" + os.Getenv("PORT")
	http.HandleFunc("/", handler)
	log.Printf("\nApplication now listening on %v\n", port)
	http.ListenAndServe(port, nil)
}

So, we’re done setting up the necessary scripts and configurations for the development environment within the container. There’s only one thing left to do and that’s create a post save hook outside of this container.

Post Save Hook

Our editor needs to have a post save hook and most editors have one. Sublime Editor has sublime-hooks, Vim users can use auto run upon save. I personally use Atom Editor with a plugin called on-save. The on-save requires me to have a file in the root of the project named .on-save.json with the following content:

[
  {
    "srcDir": ".",
    "destDir": ".",
    "files": "**/*.go",
    "command": "echo $(date) - ${srcFile} > .build"
  }
]

So, srcDir / destDir - I pretty much ignore and set to the current directory. files tells it to listen to listen to save changes made on *.go files. If a *.go file is changed, it kicks off a shell command:

echo $(date) - ${srcFile} > .build

Which is basically writing the current date and the file changed (e.g. Wed Aug 17 13:35:20 EDT 2016 - main.go) in a file named .build.

Something worth noting at this point. If you’d rather manually control the reloading of the go building / relaunching, there’s nothing in the process stopping you from deciding when the container is going to rebuild everything internally. Perhaps you’re ok with bringing down the environment and bringing it back up to rebuild. Find what works best for you.

Bringing up the environment

I have a sample docker-compose.yml in the project that will get you up and running pretty quickly. Again, I want to be as close to real world scenario as possible and that means that I may have an API server, a database server, memcache, etc. Docker Compose allows us to describe an environment and get it up pretty quickly.

With a terminal open, and the working directly in the root of the project, let’s launch the environment by typing docker-compose up. Here’s an animated gif showing what we should be seeing.

Self Compiling

The terminal is on the right hand side, the environment is coming up. Within the environment, it runs build.sh which removes the previous Go binary, rebuilds it, and relaunches it. On the left, a change is made within Atom. The post save hook kicks in when the file is saved which creates .build. Back on the right, the environment running reflex detects the change to .build and kicks off build.sh which removes the previous Go binary, rebuilds it, and relaunches it.

I’m ready to build some awesome stuff now. :)

Conclusion

I believe maintaining a reproducible local developer environment across team members is critical. However, if you can’t update the local environment as features and fixes are available, across the team with multiple disciplines, then having a docker container that self builds the environment could be an effective solution that saves your team time. In some cases, developers on the team might not have the expertise to update their local environment properly or may need to try a version of code and then roll it back. Finding a solution to solve this problem is important, especially when the diversity of disciplines on the team increases over time.

Project notes:

I’d like to thank: