How to avoid re-downloading pushed images

Hey guys!

We’re using Harbor as our container registry, and we want to enable automatic image vulnerability scanning on push.

When doing a put using the image resource, Concourse successfully pushes the image, but then immediately tries to pull the image again. Because Harbor (Trivy, in our case) is still scanning the image, the pull fails, and consequently the entire task fails.

I’ve tried using attempts to force a retry, but this re-tries the entire build-and-push process, which just perpetuates the problem.

Is there a way I can avoid the image resource attempting a pull after a push, or somehow introduce a delay/retry into this process?

Thanks!
David

The implicit get after a put is the thing doing the pull on the image. You can configure parameters for this get step using the get_params option. Handily, the docker-image & registry-image resource both have a skip_download flag to their get steps! So you should just need to add this to your image put step and hopefully it should work ("true" needs to be a string for some reason):

get_params:
  skip_download: "true"

It will still pull some metadata about the image such as the digest but it won’t try and pull any image layers.

So like this?


    # Build image to use for unit testing
    - put: image-test
      params:
        build: git-source
        dockerfile: git-source/ci/Dockerfile.test
        build_args_file: build-args/build-args.json
      get_params:
        skip_download: "true"

This configuration still seems to trigger a get… and then fail :frowning:

Hmm, this sounds like a more fundamental problem - if the get after the put fails, a get anywhere else in the pipeline would, too.

What error is returned from the registry? Would it make sense to retry on it? How long does scanning typically take?

The error message reported by the registry is:

Error response from daemon: unknown: current image with "Running" status of vulnerability 
scanning cannot be pulled due to configured policy in 'Prevent images with vulnerability severity of 
"High" or higher from running.' To continue with pull, please contact your project administrator for 
help.

It doesn’t take long to scan an image, but presumably longer than concourse allows between the push and the pull :wink: