Need to cache an immovable path inside a docker container


#1

Hello fellow operators,

I’m hoping to get some advice on how to solve an issue we’re facing with concourse at the moment.

We’re using nix for building our projects inside a docker image, and we’re running into an issue with caching. The problem is that for nix to work, it’s store (thing something like node_modules) has to be in the absolute path /nix/store, no symlinks or hardlinks allowed.

Furthermore, we’d like to share this folder between jobs in the same pipeline, which while I know is a can of worms, would be safe in our case since nix is content addressed anyway.

We’ve tried copying the files in and out of the cache, but for various reasons that hasn’t been feasible (namely, it’s slow and the cache is per task anyway) and uploading to S3 wouldn’t make sense as nix has a global cache that’s in s3 anyway.

Although I think this is generally the wrong direction for concourse, is there a way to bind mount a folder from the host?

The other potential option is to get fancy using bind mounts or union mounts inside docker, but from what I recall (and read actually) that would require us to make all jobs privileged.