Error: resource script '/opt/resource/in [/tmp/build/get]' failed: exit status 1

Very new to Concourse and trying to understand and implement a private resource-type to understand. I currently have Concourse 5.6.0, using the stable/concourse helm chart in minikube.

When I try to run my pipeline I get resource script '/opt/resource/in [/tmp/build/get]' failed: exit status 1. The resource_type in question is an exact copy of the time-resource, with the JSON parsing commented out and instead of writing static strings.

The pipeline is as follows:

resource_types:
  - name: debug-type
    type: registry-image
    source:
      repository: quay.io/jkamenik/dev-test
      tag: pipeline-debug-7
      username: ((quay.username))
      password: ((quay.password))

resources:
  - name: debug
    type: debug-type
    icon: bug
    source: {}

jobs:
  - name: main
    serial: true # i.e., max_in_flight: 1
    plan:
      # Used to check the resource as an check and input
      - get: debug
        trigger: true
      - task: check-output
        config:
          platform: linux
          inputs:
            - name: debug
          image_resource:
            type: registry-image
            source:
              repository: busybox
          run:
            path: /bin/sh
            args:
              - "-c"
              - "ls -laR"

I don’t see anything in the logs for any of the pods that clearly indicate any kind of issue, but it doesn’t appear like my container is actually being executed, unless concourse is supressing stderr for some reason.

I am not sure how to modify my code to improve the debuggability of this issue, or what settings to enable to increase the logging.

Also, not sure if it is related, but I am not able to hijack this resource.

I think your pipeline is failing at this part in the concourse code: https://github.com/concourse/concourse/blob/eb4c434bba9ba2e256241e9fe4797ff46226f2e5/atc/resource/run_script.go#L118-L126

Which would mean that your resource container is being created (not sure why hijack is not working; are you not on the main team?). Therefore, the issue should be with the /opt/resource/in script.

You could try locally running your custom time-resource container and seeing if you get the same error. Make the entry point /opt/resource/in and pass in one arg to reproduce what Concourse is doing when running a resource.

$ docker run -it quay.io/jkamenik/dev-test:pipeline-debug-7 /opt/resource/in /tmp

If the script doesn’t fail right away then pass in an empty json object {} and see if it emits a version. This is what a successful run looks like:

$ docker run -it concourse/time-resource /opt/resource/in /tmp
{}
{"version":{"time":"2019-11-17T00:29:52.1787606Z"},"metadata":null}

Hope that helps!

I am on the main team, so I am also not sure why hijack isn’t working. But I created a little shell harness to test what I think harness is doing and it works as I expect.

The harness:

#!/bin/bash

# set -x

function harness() {
  local image=$1
  local function=$2
  echo "==== Running $2 harness against $1 ===="

  local id=$(docker run -t -d --rm -v $(pwd):/tmp/data --entrypoint /bin/sh "$image")

  $function $id $@
  echo "Status: $?"

  docker kill $id
}

function check() {
  local conatiner=$1

  docker exec -t -w /tmp/data "$1" /bin/sh -c '/opt/resource/check < test-data/check.json'
}

function blank_in() {
  local container=$2

  mkdir -p output

  docker exec -t -w /tmp/data "$1" /bin/sh -c '/opt/resource/in /tmp/data/output < test-data/blank_in.json'
}


harness "$1" check
harness "$1" blank_in

This tries to get around the fact that I cannot pipe directy to a container (e.g., cat check.json | docker run -ti ... doesn’t work).

The test-data/check.json is

{
  "source": {},
  "version": {}
}

And test-data/blank_in.json is

{}

The output is:

$ test quay.io/jkamenik/dev-test:pipeline-debug-7
==== Running check harness against quay.io/jkamenik/dev-test:pipeline-debug-7 ====
2019/11/26 20:29:29 main.go:11: Check main
2019/11/26 20:29:29 main.go:15: {
[]
Status: 0
68b2e134d02ab177f6c48917cab736d517e7d2f40caf42b243cb52a069626376
==== Running blank_in harness against quay.io/jkamenik/dev-test:pipeline-debug-7 ====
2019/11/26 20:29:31 main.go:17: In main
2019/11/26 20:29:31 main.go:26: creating /tmp/data/output
{"version":{},"metadata":null}
Status: 0
f3f1ea429a627dcc0767e707baeedd13fa4a301cdf4efa09fd42d4f76c141fd9

To me this seems like a expected output so not sure why it would exiting with a 0 for me but a 1 for concourse.

Oh, not stated before but lines like “2019/11/26…” are on STDERR. Anything not in that format (before the “Status” line) is on STDOUT.

Also, I have hacked down the in code to the following:

package main

import (
	"encoding/json"
	"fmt"
	"log"
	"os"
	"path/filepath"

	// This need to match the path that
	"bitbucket.com/opaqnetworks/pipeline-debug/models"
)

var logger = log.New(os.Stderr, "", log.Ldate|log.Ltime|log.Lshortfile)

func main() {
	logger.Print("In main")
	if len(os.Args) < 2 {
		println("usage: " + os.Args[0] + " <destination>")
		os.Exit(1)
	}

	destination := os.Args[1]

	// Create the provided directory
	logger.Print("creating " + destination)
	err := os.MkdirAll(destination, 0755)
	if err != nil {
		fatal("creating destination", err)
	}

	// It appears you MUST put something in the output, so just write
	// the input
	file, err := os.Create(filepath.Join(destination, "input"))
	if err != nil {
		fatal("writing input file", err)
	}
	defer file.Close()

	var input string
	fmt.Scan(&input)
	file.WriteString(input)

	// Emit a version
	json.NewEncoder(os.Stdout).Encode(models.InResponse{
		Version: models.Version{},
	})

	os.Exit(0)
}

func fatal(doing string, err error) {
	logger.Println("Error " + doing + ":" + err.Error())
	os.Exit(1)
}

Just occurred to me; you probably can’t hijack into your container because it doesn’t have /bin/bash. When you do fly hijack it tries to hijack in by running /bin/bash. You can override this by specifying a different executable like you would for docker:

fly -t target hijack -u <url> /bin/sh

If I’m reading the output correctly, it appears your resource eventually outputs:

{"version":{},"metadata":null}

Maybe this is why there’s an exit 1? Maybe it’s coming from concourse failing to read the output of the resource? Hope that helps!
Concourse isn’t expecting an object in this format. It’s expecting an array of json objects:

[
  { key: value, key: value...},
  { key: value, key: value...}
]

The docker container “should” have bash. At least it does when I run it locally. Here is the docker file.

FROM golang:alpine as builder
COPY . /go/src/bitbucket.com/jkamenik/pipeline-debug
ENV CGO_ENABLED 0
RUN go get -d ./...
RUN go build -o /assets/check bitbucket.com/jkamenik/pipeline-debug/check
RUN go build -o /assets/in bitbucket.com/jkamenik/pipeline-debug/in
RUN go build -o /assets/out bitbucket.com/jkamenik/pipeline-debug/out

FROM alpine
LABEL revision 7
RUN apk add --no-cache bash ca-certificates
COPY --from=builder assets/ /opt/resource
RUN chmod +x /opt/resource/*

So went and tried /bin/sh but I think the search isn’t finding a container vs not being able to connect to one.

$ fly -t main --verbose hijack -c debug-test/debug /bin/sh                                                                                                                                                                                *[master]
2019/11/27 07:09:26 GET /api/v1/info HTTP/1.1
Host: 192.168.64.4:30080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


2019/11/27 07:09:26 HTTP/1.1 200 OK
Content-Length: 86
Content-Type: application/json
Date: Wed, 27 Nov 2019 12:09:26 GMT
X-Concourse-Version: 5.6.0
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: deny
X-Xss-Protection: 1; mode=block

{"version":"5.6.0","worker_version":"2.2","external_url":"http://192.168.64.4:30080"}

2019/11/27 07:09:26 GET /api/v1/teams/main/containers?pipeline_name=debug-test&resource_name=debug&type=check HTTP/1.1
Host: 192.168.64.4:30080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


2019/11/27 07:09:26 HTTP/1.1 200 OK
Content-Length: 3
Content-Type: application/json
Date: Wed, 27 Nov 2019 12:09:26 GMT
X-Concourse-Version: 5.6.0
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: deny
X-Xss-Protection: 1; mode=block

[]

no containers matched your search parameters!

they may have expired if your build hasn't recently finished.

I get the same output using fly -t main --verbose hijack -u http://192.168.64.4:30080/teams/main/pipelines/debug-test/resources/debug /bin/sh so it seems fly parses the url as a check which is what I expect.

Checking the logs on the worker pod (k8s) I found that it is in fact exiting with a clean exit code. But it for some reason calls in before check. It also cleans up the container immediately at exit.

The full output is below, but specific exit code is

{"timestamp":"2019-11-27T21:16:18.883876279Z","level":"info","source":"guardian","message":"guardian.run.exec.exec-with-bndl.execrunner.runc-exit-status","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","id":"resource","path":"/opt/resource/in","session":"33094.1.2.2","status":0}}

So I think my container is working as expected, but Concourse is erroring in a later stage.

{"timestamp":"2019-11-27T21:16:18.090400778Z","level":"info","source":"guardian","message":"guardian.create.start","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"33092"}}
{"timestamp":"2019-11-27T21:16:18.149673131Z","level":"info","source":"guardian","message":"guardian.create.network-depot-setup-bindmounts.start","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"33092.2"}}
{"timestamp":"2019-11-27T21:16:18.170914925Z","level":"info","source":"guardian","message":"guardian.create.network-depot-setup-bindmounts.finished","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"33092.2"}}
{"timestamp":"2019-11-27T21:16:18.171034947Z","level":"info","source":"guardian","message":"guardian.create.containerizer-create.start","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"33092.3"}}
{"timestamp":"2019-11-27T21:16:18.182187151Z","level":"info","source":"guardian","message":"guardian.create.containerizer-create.depot-create.started","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"33092.3.1"}}
{"timestamp":"2019-11-27T21:16:18.184204450Z","level":"info","source":"guardian","message":"guardian.create.containerizer-create.depot-create.finished","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"33092.3.1"}}
{"timestamp":"2019-11-27T21:16:18.184600794Z","level":"info","source":"guardian","message":"guardian.create.containerizer-create.create.creating","data":{"bundle":"/concourse-work-dir/depot/dc8d8893-0cf5-4e2d-7462-d21d62a34b92","bundlePath":"/concourse-work-dir/depot/dc8d8893-0cf5-4e2d-7462-d21d62a34b92","handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","id":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","logPath":"/concourse-work-dir/depot/dc8d8893-0cf5-4e2d-7462-d21d62a34b92/create.log","pidFilePath":"/concourse-work-dir/depot/dc8d8893-0cf5-4e2d-7462-d21d62a34b92/pidfile","runc":"goci.RuncBinary{Path:\"/var/gdn/assets/linux/bin/runc\", Root:\"/run/runc\"}","session":"33092.3.2"}}
{"timestamp":"2019-11-27T21:16:18.488516789Z","level":"info","source":"guardian","message":"guardian.create.containerizer-create.create.completing","data":{"bundle":"/concourse-work-dir/depot/dc8d8893-0cf5-4e2d-7462-d21d62a34b92","handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"33092.3.2"}}
{"timestamp":"2019-11-27T21:16:18.488800006Z","level":"info","source":"guardian","message":"guardian.create.containerizer-create.create.finished","data":{"bundle":"/concourse-work-dir/depot/dc8d8893-0cf5-4e2d-7462-d21d62a34b92","handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"33092.3.2"}}
{"timestamp":"2019-11-27T21:16:18.488879755Z","level":"info","source":"guardian","message":"guardian.create.containerizer-create.finished","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"33092.3"}}
{"timestamp":"2019-11-27T21:16:18.489782772Z","level":"info","source":"guardian","message":"guardian.create.containerizer-create.create.watch.watching","data":{"bundle":"/concourse-work-dir/depot/dc8d8893-0cf5-4e2d-7462-d21d62a34b92","handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"33092.3.2.1"}}
{"timestamp":"2019-11-27T21:16:18.504396043Z","level":"info","source":"guardian","message":"guardian.create.network.started","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"33092.7","spec":""}}
{"timestamp":"2019-11-27T21:16:18.504589817Z","level":"info","source":"guardian","message":"guardian.create.network.config-create","data":{"config":{"ContainerHandle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","HostIntf":"wm4lidf3eh9a-0","ContainerIntf":"wm4lidf3eh9a-1","IPTablePrefix":"w--","IPTableInstance":"m4lidf3eh9a","BridgeName":"wbrdg-0afe0000","BridgeIP":"10.254.0.1","ContainerIP":"10.254.0.2","ExternalIP":"172.17.0.6","Subnet":{"IP":"10.254.0.0","Mask":"/////A=="},"Mtu":1500,"PluginNameservers":null,"OperatorNameservers":["172.17.0.6"],"AdditionalNameservers":[],"AdditionalHostEntries":null,"PluginSearchDomains":null},"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"33092.7","spec":""}}
{"timestamp":"2019-11-27T21:16:18.562136585Z","level":"info","source":"guardian","message":"guardian.create.network.finished","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"33092.7","spec":""}}
{"timestamp":"2019-11-27T21:16:18.567846800Z","level":"info","source":"guardian","message":"guardian.create.created","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"33092"}}
{"timestamp":"2019-11-27T21:16:18.568036578Z","level":"info","source":"guardian","message":"guardian.api.garden-server.create.created","data":{"request":{"Handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","GraceTime":0,"RootFSPath":"raw:///concourse-work-dir/volumes/live/6f0f7536-1b38-4c81-6d75-3288b0738e46/volume","BindMounts":[{"src_path":"/concourse-work-dir/volumes/live/2304e761-4a5c-4ae7-751a-300079ee7da3/volume","dst_path":"/scratch","mode":1},{"src_path":"/concourse-work-dir/volumes/live/de7cf65a-e401-487c-7f5b-25f9dcde6198/volume","dst_path":"/tmp/build/get","mode":1}],"Network":"","Privileged":false,"Limits":{"bandwidth_limits":{},"cpu_limits":{},"disk_limits":{},"memory_limits":{},"pid_limits":{}}},"session":"3.1.58103"}}
{"timestamp":"2019-11-27T21:16:18.601168399Z","level":"info","source":"guardian","message":"guardian.api.garden-server.get-properties.got-properties","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"3.1.58104"}}
{"timestamp":"2019-11-27T21:16:18.609489030Z","level":"info","source":"guardian","message":"guardian.api.garden-server.get-properties.got-properties","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","session":"3.1.58105"}}
{"timestamp":"2019-11-27T21:16:18.622228810Z","level":"info","source":"guardian","message":"guardian.attach.started","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","process-id":"resource","session":"33093"}}
{"timestamp":"2019-11-27T21:16:18.622476150Z","level":"info","source":"guardian","message":"guardian.attach.finished","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","process-id":"resource","session":"33093"}}
{"timestamp":"2019-11-27T21:16:18.631276172Z","level":"info","source":"guardian","message":"guardian.run.started","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","path":"/opt/resource/in","session":"33094"}}
{"timestamp":"2019-11-27T21:16:18.631356020Z","level":"info","source":"guardian","message":"guardian.run.exec.start","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","path":"/opt/resource/in","session":"33094.1"}}
{"timestamp":"2019-11-27T21:16:18.633376017Z","level":"info","source":"guardian","message":"guardian.run.exec.exec-with-bndl.start","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","path":"/opt/resource/in","session":"33094.1.2"}}
{"timestamp":"2019-11-27T21:16:18.651472240Z","level":"info","source":"guardian","message":"guardian.run.exec.exec-with-bndl.execrunner.start","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","id":"resource","path":"/opt/resource/in","session":"33094.1.2.2"}}
{"timestamp":"2019-11-27T21:16:18.670869324Z","level":"info","source":"guardian","message":"guardian.run.exec.exec-with-bndl.execrunner.read-exit-fd","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","id":"resource","path":"/opt/resource/in","session":"33094.1.2.2"}}
{"timestamp":"2019-11-27T21:16:18.883876279Z","level":"info","source":"guardian","message":"guardian.run.exec.exec-with-bndl.execrunner.runc-exit-status","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","id":"resource","path":"/opt/resource/in","session":"33094.1.2.2","status":0}}
{"timestamp":"2019-11-27T21:16:18.884036275Z","level":"info","source":"guardian","message":"guardian.run.exec.exec-with-bndl.execrunner.done","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","id":"resource","path":"/opt/resource/in","session":"33094.1.2.2"}}
{"timestamp":"2019-11-27T21:16:18.884568932Z","level":"info","source":"guardian","message":"guardian.run.exec.exec-with-bndl.finished","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","path":"/opt/resource/in","session":"33094.1.2"}}
{"timestamp":"2019-11-27T21:16:18.890484115Z","level":"info","source":"guardian","message":"guardian.run.exec.finished","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","path":"/opt/resource/in","session":"33094.1"}}
{"timestamp":"2019-11-27T21:16:18.890547974Z","level":"info","source":"guardian","message":"guardian.run.finished","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","path":"/opt/resource/in","session":"33094"}}
{"timestamp":"2019-11-27T21:16:18.890598241Z","level":"info","source":"guardian","message":"guardian.api.garden-server.run.spawned","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","id":"resource","session":"3.1.58107","spec":{"Path":"/opt/resource/in","Dir":"","User":"root","Limits":{},"TTY":null}}}
{"timestamp":"2019-11-27T21:16:19.273664600Z","level":"info","source":"guardian","message":"guardian.api.garden-server.run.exited","data":{"handle":"dc8d8893-0cf5-4e2d-7462-d21d62a34b92","id":"resource","session":"3.1.58107","status":1}}
{"timestamp":"2019-11-27T21:16:20.678966359Z","level":"info","source":"guardian","message":"guardian.list-containers.starting","data":{"session":"33095"}}
{"timestamp":"2019-11-27T21:16:20.679076989Z","level":"info","source":"guardian","message":"guardian.list-containers.finished","data":{"session":"33095"}}
...
{"timestamp":"2019-11-27T21:16:21.674568685Z","level":"info","source":"guardian","message":"guardian.api.garden-server.get-properties.got-properties","data":{"handle":"eb12047e-e835-496b-5248-4bcf5d5f574d","session":"3.1.58110"}}
{"timestamp":"2019-11-27T21:16:21.682073642Z","level":"info","source":"guardian","message":"guardian.run.started","data":{"handle":"eb12047e-e835-496b-5248-4bcf5d5f574d","path":"/opt/resource/check","session":"33097"}}
{"timestamp":"2019-11-27T21:16:21.682133004Z","level":"info","source":"guardian","message":"guardian.run.exec.start","data":{"handle":"eb12047e-e835-496b-5248-4bcf5d5f574d","path":"/opt/resource/check","session":"33097.1"}}
{"timestamp":"2019-11-27T21:16:21.686590087Z","level":"info","source":"guardian","message":"guardian.run.exec.exec-with-bndl.start","data":{"handle":"eb12047e-e835-496b-5248-4bcf5d5f574d","path":"/opt/resource/check","session":"33097.1.2"}}
{"timestamp":"2019-11-27T21:16:21.714101198Z","level":"info","source":"guardian","message":"guardian.run.exec.exec-with-bndl.execrunner.start","data":{"handle":"eb12047e-e835-496b-5248-4bcf5d5f574d","id":"be9e25a9-52b3-4613-5ce4-3faa36c58ab4","path":"/opt/resource/check","session":"33097.1.2.2"}}
{"timestamp":"2019-11-27T21:16:21.728470415Z","level":"info","source":"guardian","message":"guardian.run.exec.exec-with-bndl.execrunner.read-exit-fd","data":{"handle":"eb12047e-e835-496b-5248-4bcf5d5f574d","id":"be9e25a9-52b3-4613-5ce4-3faa36c58ab4","path":"/opt/resource/check","session":"33097.1.2.2"}}
{"timestamp":"2019-11-27T21:16:21.930457842Z","level":"info","source":"guardian","message":"guardian.run.exec.exec-with-bndl.execrunner.runc-exit-status","data":{"handle":"eb12047e-e835-496b-5248-4bcf5d5f574d","id":"be9e25a9-52b3-4613-5ce4-3faa36c58ab4","path":"/opt/resource/check","session":"33097.1.2.2","status":0}}
{"timestamp":"2019-11-27T21:16:21.930576167Z","level":"info","source":"guardian","message":"guardian.run.exec.exec-with-bndl.execrunner.done","data":{"handle":"eb12047e-e835-496b-5248-4bcf5d5f574d","id":"be9e25a9-52b3-4613-5ce4-3faa36c58ab4","path":"/opt/resource/check","session":"33097.1.2.2"}}
{"timestamp":"2019-11-27T21:16:21.933731972Z","level":"info","source":"guardian","message":"guardian.run.exec.exec-with-bndl.finished","data":{"handle":"eb12047e-e835-496b-5248-4bcf5d5f574d","path":"/opt/resource/check","session":"33097.1.2"}}
{"timestamp":"2019-11-27T21:16:21.936110869Z","level":"info","source":"guardian","message":"guardian.run.exec.finished","data":{"handle":"eb12047e-e835-496b-5248-4bcf5d5f574d","path":"/opt/resource/check","session":"33097.1"}}
{"timestamp":"2019-11-27T21:16:21.937442527Z","level":"info","source":"guardian","message":"guardian.run.finished","data":{"handle":"eb12047e-e835-496b-5248-4bcf5d5f574d","path":"/opt/resource/check","session":"33097"}}
{"timestamp":"2019-11-27T21:16:21.937761925Z","level":"info","source":"guardian","message":"guardian.api.garden-server.run.spawned","data":{"handle":"eb12047e-e835-496b-5248-4bcf5d5f574d","id":"be9e25a9-52b3-4613-5ce4-3faa36c58ab4","session":"3.1.58111","spec":{"Path":"/opt/resource/check","Dir":"","User":"root","Limits":{},"TTY":null}}}
{"timestamp":"2019-11-27T21:16:22.337840827Z","level":"info","source":"guardian","message":"guardian.api.garden-server.run.exited","data":{"handle":"eb12047e-e835-496b-5248-4bcf5d5f574d","id":"be9e25a9-52b3-4613-5ce4-3faa36c58ab4","session":"3.1.58111","status":0}}

So I decided to tear down the cluster and rebuild with the newest Concourse version imageTag: 5.7.1. Now I am getting

resource script '/opt/resource/check []' failed: exit status 1

stderr:
2019/11/28 12:38:56 [DEBUG] GET https://quay.io/v2/
2019/11/28 12:38:57 [DEBUG] GET https://quay.io/v2/auth?scope=repository%3Aopaqnetworks%2Fdev-test%3Apull&service=quay.io
2019/11/28 12:38:57 [DEBUG] GET https://quay.io/v2/opaqnetworks/dev-test/manifests/pipeline-debug-9
e[31mERROe[0m[0000] failed to get remote image: unsupported MediaType: "application/vnd.docker.distribution.manifest.v1+prettyjws", see https://github.com/google/go-containerregistry/issues/377 

After reading the comments it appears that quay.io only support v2 API on new repos. The repo I am using is a pretty old one even though the image is new. I switched the resource-type from repository-image to docker-image and now everything works as expected.