So, after my previous slightly ranty post, I’ve been trying out a few different tools and approaches to building containers, attempting to find something which is closer to my idea of what good looks like. One tool stands out from the rest: Habitus.

(Not to be confused with Habitat, an annoyingly similar project with an annoyingly similar name. I have no idea which came first, suffice to say, I had heard of Habitat before and discounted it as being irrelevant to my use cases – and therefore almost overlooked Habitus during my research)

Habitus provides just-enough-Make to bring some sanity to the Docker build process, with the following killer features:

  1. ability to order container builds by expressing a dependency from one to another
  2. first-class support for artefacts created during the container build process, which can be extracted and used as input for later builds
  3. management API to provide build-time secrets into the containers

It’s not all sunbeams and kittens, but functionally this is a truly excellent start. In order to separate out my build and runtime containers, I had previously resorted to using make, doing something like the following to extract a build artefact in a two-step process (this is – obviously – a node.js app):

node_modules.tgz: Dockerfile.build
    docker build -t myapp/build:latest -f Dockerfile.build  .
    docker run myapp/build tar -cz -C /srv/app node_modules > node_modules.tgz

app: Dockerfile node_modules.tgz
    docker build -t myapp/app -f Dockerfile  .

So I would run make app and this would kick off a build process that would create some output – like the node_modules installation – that would be captured as an artefact for use in the main build, by manually running tar out of the container itself. Remember, the point of this is that my npm install process is creating some native libraries and things, so my build container has all the build-essential toolchain and whatnot that I really don’t want in my production app container.

In Habitus-speak, the build.yml file – which looks a bit like compose.yml if you squint – looks like this instead:

build:
  version: 2016-03-14
  steps:
    builder:
      name: myapp/builder
      dockerfile: Dockerfile.build
      artifacts:
        - /srv/app/node_modules.tgz
    deployment:
      name: myapp/app
      dockerfile: Dockerfile
      depends_on:
        - builder
      cleanup:
        commands:
          - apt-get clean autoclean
          - apt-get autoremove -y
          - rm -rf /var/lib/{apt,dpkg,cache,log}/

At first glance it’s not totally obvious why this is better – it’s more verbose – but this is just a really simple example. The Habitus approach scales much better, it’s better documented, and you don’t have to play lots of different tricks (or teach fellow devs Make syntax…). You can have multiple artefacts, as mentioned you can inject secrets (this example doesn’t demonstrate that), and in fact Habitus is also going to do a couple other things for you:

  • image squashing kind of comes for free. That, combined with the cleaner build system, saved me a couple hundred Mb up-front in container size (admittedly, this was a trivial container with obvious problems – point is, I didn’t have to hand-optimize this)
  • you can add clean-up steps to containers. See my myapp/app container above – you can throw in a bunch of container-trimming commands, and with the image squashing in play this means you get actual size reduction. Much better than attempting to stuff as much logic into single RUN invocations as possible.

After all that good stuff, what are the downsides? Well, there are a few. The first is that this isn’t built into Docker – you need to go grab Habitus separately. Thankfully, it’s a stand-alone Go binary, so this is no hard task. Personally I think it’s crazy that this isn’t part of the base but there we go.

Building in Habitus is generally a lot slower. This is due to a couple of things: obviously, some of the things it is doing are more sophisticated, so that takes time, but it also encourages you to split things out into different container steps, and each one of those adds overhead. It’s not a lot slower, though, and (dependencies-allowing) it will build things in parallel.

Lastly, the documentation actually isn’t all that great. If you’re comfortable piecing things together it’s ok, but there’s not an awful lot of support. For example, running the thing in the first instance is surprisingly complicated – it assumes that you’re running docker-machine or similar, and that you have certain things in the environment. It will throw away “unused” containers by default too, and some of the other defaults are a bit suspect. Working on a basic development system, the correct invokation for me is:

sudo habitus --use-tls=false --host=unix:///var/run/docker.sock --binding=127.0.0.1 --noprune-rmi

This will vary from installation to installation, though, and the various messages that Habitus comes out with don’t point the finger in the right direction all the time.

Also, if you make mistakes in your build.yml, a lot of the messages can be quite cryptic there too. I accidentally removed some artefact files in a cleanup step, for example, and got a pretty bizarre message for my trouble:

2016/11/30 20:39:27 ▶ Starting container a4b9dd5eeb2ac86ce978040a6b7ec94e94ac69dc208 to fetch artifact permissions 
stat: cannot stat '/srv/app/node_modules': No such file or directory
2016/11/30 20:39:42 ▶ Failed to fetch artifact permissions for /srv/app/node_modules: strconv.ParseInt: parsing "": invalid syntax

So it fails with “invalid syntax”, but the real error is on the line above. Luckily, this stuff is not too difficult to grapple with if you’ve used tools like this before (let’s face it, the state of the art in this area is not particularly strong), but I would worry that some users will find the initial learning curve a bit steep until they get into a groove of building their containers.

I’m utterly convinced this is the right way to build containers at this point. It doesn’t solve everything – the Dockerfiles are still there rocking out like ’80s sh scripts – but the overall infrastructure Habitus provides is great.