Set-pipeline: base resource type not found

#1

Hello, Concourse community,

First of all, lots of love for concource-ci thanks for such an awesome tool.

I am trying to build a spring application(using gradle build).
What I want from concourse-ci is:
Whenever there is a new commit to master: Trigger a new build

I am trying to achieve this on my local concourse set-up. Once that is successfull I will do it on remote server.

Now the struggle:
I am not able to set a pipeline for my local concourse instance. Whenever I set a pipeline and unpause it, I see an error saying base resource type not found

concourse logs

{"timestamp":"1523347987.763197660","source":"atc","message":"atc.syncer.buildnews:radar.scan-resource.interval-runner.tick.failed-to-find-or-create-resource-config-check-session","log_level":2,"data":{"error":"base resource type not found","pipeline-scoped-name":"buildnews:resource:my-news","session":"44.133.20.1.1"}}
{"timestamp":"1523347987.764551640","source":"atc","message":"atc.syncer.buildnews:radar.scanresources-runner-error","log_level":1,"data":{"error":{},"session":"44.133"}}
{"timestamp":"1523348001.464837790","source":"atc","message":"atc.destroying-pipeline.start","log_level":1,"data":{"name":"buildnews","session":"10065"}}
{"timestamp":"1523348001.476152182","source":"atc","message":"atc.destroying-pipeline.done","log_level":1,"data":{"name":"buildnews","session":"10065"}}
{"timestamp":"1523348006.976919413","source":"atc","message":"atc.set-config.saving","log_level":1,"data":{"session":"10078"}}
{"timestamp":"1523348007.004070044","source":"atc","message":"atc.set-config.saved","log_level":1,"data":{"session":"10078"}}
{"timestamp":"1523348007.756317854","source":"atc","message":"atc.syncer.buildnews:scheduler.done","log_level":1,"data":{"session":"44.134"}}
{"timestamp":"1523348007.756320715","source":"atc","message":"atc.syncer.buildnews:radar.done","log_level":1,"data":{"session":"44.133"}}
{"timestamp":"1523348027.759681940","source":"atc","message":"atc.syncer.buildnews:radar.start","log_level":1,"data":{"session":"44.135"}}
{"timestamp":"1523348027.759690523","source":"atc","message":"atc.syncer.buildnews:scheduler.start","log_level":1,"data":{"interval":"10s","session":"44.136"}}
{"timestamp":"1523348027.763847589","source":"atc","message":"atc.syncer.buildnews:radar.scan-resource.interval-runner.tick.failed-to-find-or-create-resource-config-check-session","log_level":2,"data":{"error":"base resource type not found","pipeline-scoped-name":"buildnews:resource:my-news","session":"44.135.1.1.1"}}
{"timestamp":"1523348027.764539719","source":"atc","message":"atc.syncer.buildnews:radar.scanresources-runner-error","log_level":1,"data":{"error":{},"session":"44.135"}}
{"timestamp":"1523348052.201259375","source":"atc","message":"atc.destroying-pipeline.start","log_level":1,"data":{"name":"buildnews","session":"10219"}}
{"timestamp":"1523348052.205008030","source":"atc","message":"atc.destroying-pipeline.done","log_level":1,"data":{"name":"buildnews","session":"10219"}}
{"timestamp":"1523348056.776679993","source":"atc","message":"atc.set-config.saving","log_level":1,"data":{"session":"10228"}}
{"timestamp":"1523348056.803318739","source":"atc","message":"atc.set-config.saved","log_level":1,"data":{"session":"10228"}}
{"timestamp":"1523348057.763397694","source":"atc","message":"atc.syncer.buildnews:scheduler.done","log_level":1,"data":{"session":"44.136"}}
{"timestamp":"1523348057.763398647","source":"atc","message":"atc.syncer.buildnews:radar.done","log_level":1,"data":{"session":"44.135"}}
{"timestamp":"1523348067.766259909","source":"atc","message":"atc.syncer.buildnews:scheduler.start","log_level":1,"data":{"interval":"10s","session":"44.138"}}
{"timestamp":"1523348067.766248941","source":"atc","message":"atc.syncer.buildnews:radar.start","log_level":1,"data":{"session":"44.137"}}
{"timestamp":"1523348067.771704435","source":"atc","message":"atc.syncer.buildnews:radar.scan-resource.interval-runner.tick.failed-to-find-or-create-resource-config-check-session","log_level":2,"data":{"error":"base resource type not found","pipeline-scoped-name":"buildnews:resource:my-news","session":"44.137.1.1.1"}}
{"timestamp":"1523348067.772624969","source":"atc","message":"atc.syncer.buildnews:radar.scanresources-runner-error","log_level":1,"data":{"error":{},"session":"44.137"}}
{"timestamp":"1523348127.781753063","source":"atc","message":"atc.syncer.buildnews:radar.scan-resource.interval-runner.tick.failed-to-find-or-create-resource-config-check-session","log_level":2,"data":{"error":"base resource type not found","pipeline-scoped-name":"buildnews:resource:my-news","session":"44.137.2.1.1"}}
{"timestamp":"1523348127.783041716","source":"atc","message":"atc.syncer.buildnews:radar.scanresources-runner-error","log_level":1,"data":{"error":{},"session":"44.137"}}
{"timestamp":"1523348187.798425436","source":"atc","message":"atc.syncer.buildnews:radar.scan-resource.interval-runner.tick.failed-to-find-or-create-resource-config-check-session","log_level":2,"data":{"error":"base resource type not found","pipeline-scoped-name":"buildnews:resource:my-news","session":"44.137.3.1.1"}}
{"timestamp":"1523348187.799827814","source":"atc","message":"atc.syncer.buildnews:radar.scanresources-runner-error","log_level":1,"data":{"error":{},"session":"44.137"}}
{"timestamp":"1523348247.801526070","source":"atc","message":"atc.syncer.buildnews:radar.scan-resource.interval-runner.tick.failed-to-find-or-create-resource-config-check-session","log_level":2,"data":{"error":"base resource type not found","pipeline-scoped-name":"buildnews:resource:my-news","session":"44.137.4.1.1"}}
{"timestamp":"1523348247.802726030","source":"atc","message":"atc.syncer.buildnews:radar.scanresources-runner-error","log_level":1,"data":{"error":{},"session":"44.137"}}
{"timestamp":"1523348307.806749582","source":"atc","message":"atc.syncer.buildnews:radar.scan-resource.interval-runner.tick.failed-to-find-or-create-resource-config-check-session","log_level":2,"data":{"error":"base resource type not found","pipeline-scoped-name":"buildnews:resource:my-news","session":"44.137.5.1.1"}}
{"timestamp":"1523348307.808030605","source":"atc","message":"atc.syncer.buildnews:radar.scanresources-runner-error","log_level":1,"data":{"error":{},"session":"44.137"}}
{"timestamp":"1523348367.810498953","source":"atc","message":"atc.syncer.buildnews:radar.scan-resource.interval-runner.tick.failed-to-find-or-create-resource-config-check-session","log_level":2,"data":{"error":"base resource type not found","pipeline-scoped-name":"buildnews:resource:my-news","session":"44.137.6.1.1"}}
{"timestamp":"1523348367.811683416","source":"atc","message":"atc.syncer.buildnews:radar.scanresources-runner-error","log_level":1,"data":{"error":{},"session":"44.137"}}

  1. start concourse:
    concourse web \                                                      
      --basic-auth-username concourse \
      --basic-auth-password concourse \
      --session-signing-key session_signing_key \
      --tsa-host-key host_key \
      --tsa-authorized-keys authorized_worker_keys
  1. Login

fly -t news-service login -c http://127.0.0.1:8080

  1. set-pipeline:

fly set-pipeline -t news-service -c ci/pipeline.yml -p buildnews

  1. Unpause pipeline

fly -t news-service unpause-pipeline -p buildnews
unpaused 'buildnews'

Error: base resource type not found

my%20news%20%20%20Concourse

My pipeline.yml

---
resources:
  - name: my-news
    type: git
    source:
      uri: https://github.com/swapab/my-news.git
      branch: master

jobs:
  - name: job-build-news
    plan:
      - get: my-news
        trigger: true
      - task: build-news
        config:
          platform: linux
          image_resource:
            type: docker-image
            source: {repository: openjdk, tag: 8-jdk}
          run:
            user: root
            path: sh
            dir: my-news
            args:
            - -exc
            - |
              gradle --stacktrace build

How do I make the pipeline work and run the build command?

Any help would be highly appreciated.

#2

Weird, what version of Concourse are you using? Folks have reported a similar issue in our GitHub issues https://github.com/concourse/concourse/issues/2081

#3

@swapab I see that you start the concourse web component, but did you also run a concourse worker start command as well (scroll down to the 3.3.3 Running worker Nodes section on this page)

#4

I started a worker and looks like the issue is still there.

sudo concourse worker --baggageclaim-log-level=debug \
  --work-dir /opt/concourse/worker \
  --tsa-host 127.0.0.1:2222 \
  --tsa-public-key host_key.pub \
  --tsa-worker-private-key worker_key
#5

But, the quick-start concourse setup works fine!

$ wget https://concourse-ci.org/docker-compose.yml
$ docker-compose up

This will work for me :smile:

#6

We’re seeing the same log entries and UI notification with a fresh installation of 3.11 (yes, released today) via Bosh - 1 web, 1 db, 2 workers, all linux.

bosh upload-release https://bosh.io/d/github.com/cloudfoundry/garden-runc-release?v=1.13.0 --sha1 74814d3ee8c17b1e79ac6f124ae0fe50af7a5164
bosh upload-release https://bosh.io/d/github.com/concourse/concourse?v=3.11.0 --sha1 57a64d9a95922ba73fe2deb4ddcb225a8db5015b
bosh upload-release https://bosh.io/d/github.com/cloudfoundry/postgres-release?v=28 --sha1 c1fcec62cb9d2e95e3b191e3c91d238e2b9d23fa

Recent entry in ATC log from Web (retrieved with bosh -d [deployment-name] logs):

{"timestamp":"1523661130.765697241","source":"atc","message":"atc.syncer.integration:radar.scan-resource.interval-runner.tick.failed-to-find-or-create-resource-config-check-session","log_level":2,"data":{"error":"base resource type not found","pipeline-scoped-name":"integration:resource:git-restapi","session":"46.127.3.1.1"}}

When we run fly check-resource we get this:

root@c09908d81923:/tools# fly -t [target-name] cr -r integration/git-restapi
error: Unexpected Response
Status: 500 Internal Server Error
Body:

root@c09908d81923:/tools# 

We’ve also noted open GitHub issue 2081, among other reports of this behavior.

Appreciate any suggestions of where to look for more information.

#7

Hello - same issue here after upgrade to concourse 5.1… worked perfectly before. I’m behind a proxy is that problematic after introduction of 5.1? Previous version 4.2.3 no problems with this issue. Using helm chart for 5.1

{"timestamp":"2019-04-26T13:19:10.683300142Z","level":"error","source":"atc","message":"atc.pipelines.radar.scan-resource.interval-runner.tick.failed-to-set-resource-config-id-on-resource","data":{"error":"base resource type not found: s3
","pipeline":"concourse-management","resource":"pipelines-backup","session":"18.5.31.1.1","team":"infra"}}                                                                                                                                    
{"timestamp":"2019-04-26T13:19:10.690258999Z","level":"error","source":"atc","message":"atc.pipelines.radar.failed-to-run-scan-resource","data":{"error":"base resource type not found: s3","pipeline":"concourse-management","session":"18.5"
,"team":"infra"}}                                                                                                                                                                                                                             
{"timestamp":"2019-04-26T13:20:10.670478908Z","level":"error","source":"atc","message":"atc.pipelines.radar.scan-resource.interval-runner.tick.failed-to-set-resource-config-id-on-resource","data":{"error":"base resource type not found: gi
t","pipeline":"concourse-management","resource":"common-scripts","session":"18.5.33.1.1","team":"infra"}}                                                                                                                                     
{"timestamp":"2019-04-26T13:20:10.676262507Z","level":"error","source":"atc","message":"atc.pipelines.radar.scan-resource.interval-runner.tick.failed-to-set-resource-config-id-on-resource","data":{"error":"base resource type not found: s3
","pipeline":"concourse-management","resource":"pipelines-backup","session":"18.5.34.1.1","team":"infra"}}                                                                                                                                    
{"timestamp":"2019-04-26T13:20:10.677505468Z","level":"error","source":"atc","message":"atc.pipelines.radar.scan-resource-type.interval-runner.tick.failed-to-set-resource-config-id-on-resource-type","data":{"error":"base resource type not
 found: docker-image","pipeline":"concourse-management","resource-type":"slack-notification","session":"18.5.32.1.1","team":"infra"}}                                                                                                         
{"timestamp":"2019-04-26T13:20:10.688463161Z","level":"error","source":"atc","message":"atc.pipelines.radar.failed-to-run-scan-resource","data":{"error":"base resource type not found: git","pipeline":"concourse-management","session":"18.5
","team":"infra"}}                                                                                                                                                                                                                            
{"timestamp":"2019-04-26T13:20:10.702880012Z","level":"error","source":"atc","message":"atc.pipelines.radar.failed-to-run-scan-resource","data":{"error":"base resource type not found: s3","pipeline":"concourse-management","session":"18.5"
,"team":"infra"}}                                                                                                                                                                                                                             
{"timestamp":"2019-04-26T13:20:10.703223713Z","level":"error","source":"atc","message":"atc.pipelines.radar.failed-to-run-scan-resource-type","data":{"error":"base resource type not found: docker-image","pipeline":"concourse-management","session":"18.5","team":"infra"}}