Building smaller Golang images

Binary size

One of the frequently cited advantages of Golang is the simplicity and minimalism of the language, which leads to (according to the fans) an smaller and easier to understand codebase to maintain. On top of that, the lack of a required "runtime" to be able to run your application (yes, I'm looking to you, Java) is certainly a very welcome bonus.

This all should amount to relatively small container images across the board, particularly when comparing it to "chunkier" alternatives, right? Well... mostly yes. However, we can still do better with a little bit of fine-tuning when building our Docker images.

To make this point clear, let us create a new Go project:

$ mkdir myserver

$ cd myserver

$ go mod init github.com/ibraimgm/myserver
go: creating new go.mod: module github.com/ibraimgm/myserver

Next, create a main.go file with the following content:

package main

import (
  "log"
  "net/http"
)

func main() {
  http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    log.Println("Request received!")
    w.Write([]byte("<h1>It works!</h1>"))
  })

  log.Println(http.ListenAndServe(":8080", nil))
}

This is a just a simple web application, listening on port 8080 that returns a static It works! message. Nothing special to see here (even the logs are useless), but despite being a bit boring, it is enough for our purposes.

Try to compile this demo with a simple go build on the project root and check it's size... Whoa! 7.2MB? A bit heavy, even considering that Golang executables are "statically linked", but we can cut it a bit by removing the debug symbols. Try it by running go build -ldflags '-w -s' and check the size again.

On my machine, this shaved off 2MB of the final executable, and the nice part is that the stack trace for panics remains intact. Granted, debugging becomes harder, but we won't attach the debugger to a production build anyway.

If you're still not happy, you can try to reduce the size even more by using UPX. For the sake of comparison, running upx without additional parameters reduced the "already smaller" binary from 5.2MB to 1.9MB (about 36%). Depending on your project needs, it might be interesting to add such a tool to your pipeline.

In our case, using it won't matter that much because we're most interested in generating a smaller container image, and while the binary size directly affects it, the choice of the correct image will have a much greater impact. For this reason, I'm removing UPX from the equation for simplicity's sake.

Having said that, let's "dockerize" our app by creating a Dockerfile:

FROM golang:1.14 as stage

WORKDIR /app
COPY . .
RUN go build -ldflags '-w -s' -o myserver

FROM golang:1.14
COPY --from=stage /app/myserver .
CMD ["./myserver"]

Again, this is quite simple. In this example we do a multi-stage build process, using one container to compile the application and then putting only the final executable in a new, clean container. That's why the sloppy COPY . . from the first stage won't impact the final result, but keep in mind that in a large, real-world application, you should create a proper .dockerignore to avoid sending useless data to you build context.

Having said that, let's build the image and check the size:

$ docker build -t ibraimgm/myserver .

$ docker image ls
REPOSITORY                    TAG                 IMAGE ID            CREATED             SIZE
ibraimgm/myserver             latest              ce1e81b61e36        5 seconds ago       815MB

Even knowing that the image does not occupy exactly the size listed (because an image is built from layers, that are reused, remember?), the 815MB still does look a bit too much for me. Can we do better than this?

Choosing a more suitable image

If we take a look at the documentation on Docker Hub, we will notice that golang has various types of images available, including some of them on alpine. The logical step is here is to change our image on the Dockerfile (both the stage and the final one) to golang:1.14-alpine and check the results:

$ docker build -t ibraimgm/myserver .

$ docker image ls
REPOSITORY                    TAG                 IMAGE ID            CREATED             SIZE
ibraimgm/myserver             latest              0ad5bcc24990        21 seconds ago      375MB

Just using an alpine image cut our final size to less than half, and the final app works flawlessly. This is already a very big saving, but if we think a bit about it, we can optimize further. Since our executable does not need the Go runtime to run, we could, in theory, use a "plain" alpine image on the final build.

To do that, change the stage image to golang:1.14-alpine3.11 and the final image to alpine:3.11. Notice that we used a more specific version here to make sure we're compiling and running on basically the same alpine version. If we try to build the image again, the result is quite impressive:

$ docker build -t ibraimgm/myserver .

$ docker image ls
REPOSITORY                    TAG                 IMAGE ID            CREATED             SIZE
ibraimgm/myserver             latest              1391cd559a9f        16 seconds ago      11MB

And now we're down to just 11MB. This is crazy optimized and is just ~6MB more to our executable itself. We won, and our image is as small as we can get. Or is it?

Scratch that

If we follow the same logic that our binary does not depend on anything else to run, we could, in theory, further optimize the final result to use the scratch image. The scratch image is the one image that is used as a base to all other images, so it is the canonical "empty" image. Change the final image to scratch and try again:

$ docker build -t ibraimgm/myserver .

$ docker image ls
REPOSITORY                    TAG                 IMAGE ID            CREATED             SIZE
ibraimgm/myserver             latest              8e47a8a1321f        7 seconds ago       5.44MB

As expected, the final size is even smaller, just the executable plus ~0.24MB. As far as I know, we can't do better than that, since we're already using the empty image as a base. If you try to run this image, however, you will get a surprise:

$ docker run --rm -p 8080:8080 ibraimgm/myserver
standard_init_linux.go:211: exec user process caused "no such file or directory"

Wait, what? Why this sudden error? Well, it turns out that despite the final golang binary being statically linked, it still does some calls to C libraries for specific functionalities, like, for example, networking on the net package. This means that, depending on what core packages we use, we might still depend on a C runtime, like libc or musl. By running our app on an empty image, we end up with broken dependencies.

To try to remove these dependencies, we have some work to do on our build command. First, we need to define CGO_ENABLED to 0 to disable the use of cgo. This, by itself, may break your compilation, for example, when some of your dependencies make use of cgo. If this happens, there is simply no way around it (unless you somehow remove the offending package) and you're better off using alpine and moving on with your life.

The other big dependency is the net package, and since this is a know (and very common scenario), the go build command can automatically strip this dependency off if you pass the tag netgo when building your application. Last, we should also pass -static as an extra linker flag, to indicate that we don't want to dynamically link.

After making all these changes, the final Dockerfile should look like that:

FROM golang:1.14-alpine3.11 as stage

WORKDIR /app
COPY . .
RUN env CGO_ENABLED=0 go build -tags netgo -ldflags '-w -s -extldflags "-static"' -o myserver

FROM scratch
COPY --from=stage /app/myserver .
CMD ["./myserver"]

Rebuilding the image should give you similar results (minus the error when running, of course):

$ docker build -t ibraimgm/myserver .

$ docker image ls
REPOSITORY                    TAG                 IMAGE ID            CREATED             SIZE
ibraimgm/myserver             latest              a3b51059c9bc        18 seconds ago      5.39MB

$ docker run --rm -d -p 8080:8080 ibraimgm/myserver
09fb693910006629a19982b817c3abfcc0d2eea7fe17193329ae39dd123718bb

$ curl http://localhost:8080/
<h1>It works!</h1>

Nice! In the end, the final image is just about ~0.19MB bigger than the application. It is a little bit smaller than our first attempt at using the scratch image probably because the compiler could optimize a little bit more based on the extra flags.

Too much trouble

Using the scratch image put us into a position where there is nothing left to optimize (well... you could try UPX on top of all that), but the question here is: was it worth the trouble?

While interesting to do for learning purposes, striping everything into a bare minimum scratch image is not, In my opinion, worth the trouble. We gained only ~6MB, and this gain is only visible because our application is really small. Using scratch also means that we can't easily "enter" a running container to inspect it since there is no shell available. We also end up with a somewhat cryptic build command, where the intent of the used flags aren't exactly obvious for most programmers.

If you put this all together a mere ~6MB is not worth much in the real world, unless you're in a really constrained environment (if so, maybe you shouldn't use containers and just run the application directly). Personally, I think the alpine optimized image is good enough.