Everyone wants faster Continuous Integration (CI). The quicker you can get from commit to test results, the better – but there’s a bit of tension here. A CI should start close to “first principles”, so that you know an entire build is good and repeatable. But the more work the CI is doing, the slower it will go.
I’ve mentioned before that I use a combination of stowage and container-based CI systems, and in professional life we’re very heavily invested in GitLab now (which is an excellent system in general). Containers are a good starting point for CI, because they’re very easy to set up, do some work, and then throw away again. Getting clean builds is very straightforward.
Today I released version 0.5.0 of stowage, which brings a few additional small features which really simplifies usage in CI.
Just to recap for those who haven’t used stowage before, it’s basically a really simple tool which can take container images and make them available as if they were local binaries. Here’s how I can take the official MS Azure container and turn it into a local command:
The point here is not to replace package management in the system at all, but the end effect in terms of the user experience is very similar: what’s happening underneath is that stowage is creating a little wrapper around docker run. It’s purely a convenience thing.
Why might this be useful for CI?
So, I find stowage useful in a couple of ways. Primarily, it means I can create “tools” (actually docker images) which I can distribute very easily. I can use these tools as a developer, or within the CI – but ideally I use them in both. Then I know that I have exactly the same tool chain across my team.
This is great for new developers; setting up a new environment is very simple. If I have docker and stowage installed, that’s all I need to start pulling down everything else.
A while ago, I had an insight that CI systems look an awful lot like new developers. A new developer doesn’t have an environment set up or the code available; in a similar sense, neither does the CI. You’re starting from a clean base.
I believe there’s a lot of correlation between the quality of onboarding experience for new developers, and the quality of a team’s CI process. When I’ve found teams with great onboarding processes, their CI process is also good: equally, when they struggle to integrate new developers, I’ve found the CI to be highly complex and fragile (if present at all). The quality of one really does predict the quality of the other.
So, it made sense to me that a tool I had created for the purpose of improving the developer experience would also be applicable to CI.
Some patterns, then!
So, first, having the main build tools be available as containers that you use as tools with stowage is a good plan. Ensuring that developers and CI uses the same tools is valuable. For example, rather than developers installing node and npm locally, why not use the official NodeJS container image?
