Best way to start a docker container during a task

In many of our projects that we continuously integrate with Concourse, we use TestContainers library to create various disposable containers in integration tests. Some examples include Postgres, Redis, Localstack. There containers are destroyed (by the library) once tests are finished.

Till now we’ve been really running “Docker in Docker”:

  • Alpine based on github concourse/docker-image-resource/blob/master/assets/common.sh
  • Ubuntu based on github jpetazzo/dind

After update to Concourse 5.0 / “overlay” worker baggage claim driver, Docker daemon is no longer able to start.
After I applied https://github.com/concourse/docker-image-resource/commit/fd492346246e9e9f61040bf31c740e80fccc9365 for our Alpine solution it worked, but for Ubuntu I still have to find a solution.

Next to it, running “Docker in Docker” is not recommended (e.g., here: https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/).

I am wondering what would be a recommended approach to spin up docker containers during builds on CI that would be forward compatible?

A workaround is to still use btrfs on worker by add --baggageclaim-driver=btrfs. But with 5.0.0, concourse binary seems no longer embed btrfs and mkfs.btrfs binaries, if your worker doesn’t contain those two binary files, you may have to manually install them, then use --baggageclaim-btrfs-bin= and --baggageclaim-mkfs-bin= to specify paths of them.

@evanchaoli, thanks for the workaround.
For now, I can probably follow commits on https://github.com/concourse/docker-image-resource to have a working solution for Alpine. However, I think the idea is to have it deprecated eventually and get/build images without Docker actually running.
For Ubuntu setup, I’ll probably still have to fix it myself.

But I was wondering more about the long term approach, so that I don’t need to apply fixes now and then when new version of Concourse comes out.
I think there should be quite some teams out there that use Docker for spinning up resources during (integration) tests as a part of build scripts (maven, Gradle, …) to ensure quicker feedback when running both on developers machines and in CI.
Ideally, I would like to have a task configuration similar to Travis CI:
- task: build image: jdk file: git/ci/tasks/build.yml timeout: 30m services: [docker] ...
(with DOCKER_HOST passed to the task, e.g.).

1 Like

this is an ongoing gap in concourse today: https://github.com/concourse/concourse/issues/324 for more info

Hi @artamonovkirill,
Apologies for replying to your old post. We’ve been exploring using Localstack in test tasks also, and thinking along similar lines to you. Having now read the long-debated Issue #324, it seems there aren’t very many other options.
How did you get on with a LocalStack container within Concourse? Did you experience any issues there?
Thanks

@Gskal,

When we need Docker daemon in our builds, we use an image that has Docker installed and startable.
We based our images on https://github.com/meAmidos/dcind (Alpine) or https://github.com/jpetazzo/dind (Ubuntu).

With parameterized parent image, like:

ARG BASE
FROM $BASE

we can easily install Docker on almost any base image.

To start Docker in a Concourse task, enable the “privileged” mode and then just execute:

source /docker-lib.sh
start_docker

I don’t think we experienced any problems so far.

One limitation / possible improvement of our current solution is that we don’t cache Docker images - so, they are downloaded each time a job is executed.

Thanks for the quick reply @artamonovkirill.
Yeah, that’s very useful to know. Thanks.
I guess we’re a bit hesitant of using Privileged mode and something relatively unknown to us like LocalStack. If that could have the potential to impact the worker / host perhaps?

@Gksal, we haven’t noticed any impact on workers, except perhaps a bit more space being used (due to downloaded Docker images) and a bit more CPU/RAM due to running the containers.

OK, thanks @artamonovkirill - that’s good to know your experiences with this.
Thanks for sharing. Much appreciated. :+1: