by Gary Ormsby, Senior Software Engineer
Bamboo, Atlassian‘s Continuous Integration (CI) product, can be an attractive option for those utilizing Atlassian’s other products, as we do. We started using its cloud version recently for a client’s turn-key web app that we are developing. Historically, we’ve used self-hosted servers for our infrastructure but here we wanted to explore an online solution. The initial setup to integrate these systems was substantial and we have found that their maintenance has required spikes in time as well.
We are using Bamboo, along with Docker, to package our code and tests for deployment and execution on an image hosted on Amazon‘s Elastic Compute Cloud (EC2). Test runs on Bamboo are triggered by code check-ins to Atlassian’s Bitbucket, our Revision Control System. Once a check-in is detected, Bamboo creates or re-uses an existing EC2 elastic instance from an earlier test run. Then Docker is installed on it and then the defined tasks in the Bamboo build plan are executed. The tasks check out the code, tests, and the Docker configuration files. Next, the product is set up (via Docker), tests run, and the test results parsed. Once the tests are run, the results are published.
The Docker website says that it “allows you to compose your application from microservices, without worrying about inconsistencies between development and production environments.” This is why we chose to use Docker. Initially, we didn’t know our production or CI environments, so it seemed advantageous to use Docker. Unfortunately, this flexibility has a price; Docker can have a high disk footprint, and this caused problems on the disk constrained EC2 Elastic Instance used by Bamboo. After a few months of successful CI runs, they began to fail intermittently when the /var/lib partition was being exhausted. After a while all CI runs began to fail. We never fully understood what changed to cause this. But after researching solutions we decided to fix it by relocating where Docker keeps its temporary files to a larger partition on the EC2 Elastic Instance. Fortunately, the partition has been large enough and this problem has not recurred.
We also had a few cases where the Docker service would fail to start after it was installed on the EC2 image. Since Bamboo attempts to re-use AWS EC2 images if they’re available, we wrote Bamboo set-up scripts that would clean up a previously existing Docker configuration prior to downloading and running a new test build. Occasionally, Docker failed to start for this cleanup process which prevented our tests from running. We were seeing the error “Error log: Failed to restart docker.service: Unit docker.service is masked.” It appeared that the problem was with Systemd (a Linux init system and system manager tool), which is used to control the starting and stopping of Docker. From searching the web we found that by simply unmasking a few of the Docker services we were able to get it to work. The exact commands we used were: “sudo systemctl unmask docker.service” and “sudo systemctl unmask docker.socket.”
Aside from the problems with Docker, we’ve had some other, smaller problems with Bamboo, like false positive test results being reported and the results of our jasmine tests not consistently being reported. However, despite a few challenges, Bamboo and Docker have enabled us to get a CI environment running pretty quickly that integrates well with our other Atlassian and AWS services. Since we have a small, new project, with no legacy environment we have found its adoption easy. However, if you have a preference for using a cloud hosting service other than AWS, or you’re using an unsupported repo (git, svn, cvs, mercurial, perforce is currently supported) you may wish to consider other options.