Concourse pipeline resource could not find configuration file


#1

Hi,

While trying to deploy with concourse pipeline resource, it can not find the configuration file.
Although I followed the documentation, I could not figure out why it searches for under “/tmp/build/put/pipelineSchedular/ci/templates/pipeline.yml” instead of “/tmp/build/5f27c6a5/pipelineSchedular/ci/templates/pipeline.yml”?

resource_types:
- name: concourse-pipeline
  type: docker-image
  source:
    repository: concourse/concourse-pipeline-resource

resources:
- name: pipelineScheduler
  type: git
  source:
    uri: https://github.com/myorg/myrepo.git
    branch: master
    username: myuser
    password: ((accessToken))

- name: my-pipelines
  type: concourse-pipeline
  source:
    target: http://10.23.21.68:8080
    insecure: "true"
    teams:
    - name: main
      username: test
      password: test

jobs:
- name: set-my-pipelines
  plan:
  - get: pipelineScheduler
    trigger: false
  - task: debugContent
    config:
      platform: linux
      image_resource:
        type: docker-image
        source: {repository: mydockeruser/centos}
      inputs:
      - name: pipelineScheduler
      run:
        path: sh
        args:
          - -exc
          - |
            ls
            pwd
            ls -alR ./pipelineScheduler/ci/
  - put: my-pipelines
    params:
      pipelines:
      - name: lbg-k8s-applications-hello_hapi_master
        team: main
        config_file: pipelineSchedular/ci/templates/pipeline.yml 

After triggering it, I am getting the failure below:

$> fly -t tutorial tj -j test1/set-my-pipelines -w
started test1/set-my-pipelines #10

initializing
running sh -exc ls
pwd
ls -alR ./pipelineScheduler/ci/

+ ls
pipelineScheduler
+ pwd
/tmp/build/5f27c6a5
+ ls -alR ./pipelineScheduler/ci/
./pipelineScheduler/ci/:
total 16
drwxr-xr-x 3 root root 4096 Dec 3 17:09 .
drwxr-xr-x 1 root root 4096 Dec 3 17:48 ..
-rw-r--r-- 1 root root 122 Dec 3 17:09 teamsPipelines.yml
drwxr-xr-x 2 root root 4096 Dec 3 17:09 templates

./pipelineScheduler/ci/templates:
total 20
drwxr-xr-x 2 root root 4096 Dec 3 17:09 .
drwxr-xr-x 3 root root 4096 Dec 3 17:09 ..
-rw-r--r-- 1 root root 5763 Dec 3 17:09 pipeline.yml
-rw-r--r-- 1 root root 432 Dec 3 17:09 seed.yml
Logging to /tmp/concourse-pipeline-resource-out.log114862619
2018/12/03 17:48:45 exit status 1 - error: invalid argument for flag `-c, --config' (expected atc.PathFlag): path '/tmp/build/put/pipelineSchedular/ci/templates/pipeline.yml' does not exist

failed

#2

I guess you need to add inputs: to put.


#3

Hi matakan,

can you try to replace “./pipelineScheduler/ci/” with “pipelineScheduler/ci”?
I think that could have caused the issue.

best,
D


#4

AFAIK, there is no way to define “input” for put.
Could you please send me the link about how to do that?


#5

Hi gdenn,

I have changed it but the result is the same. Actually the problem is not in this task (task:debugContent) where I had used “.pipelineScheduler”. It is in the put task that could not access the “pipelineScheduler” git directory.

Below is the output of what you have recommended:

   $> fly -t tutorial tj -j test1/set-my-pipelines -w
    started test1/set-my-pipelines #18

    initializing
    running sh -exc ls
    pwd
    ls -alR pipelineScheduler/ci/

    + ls
    pipelineScheduler
    + pwd
    /tmp/build/5f27c6a5
    + ls -alR pipelineScheduler/ci/
    pipelineScheduler/ci/:
    total 16
    drwxr-xr-x 3 root root 4096 Dec 4 10:18 .
    drwxr-xr-x 1 root root 4096 Dec 4 10:28 ..
    -rw-r--r-- 1 root root 181 Dec 4 10:18 teamsPipelines.yml
    drwxr-xr-x 2 root root 4096 Dec 4 10:18 templates

    pipelineScheduler/ci/templates:
    total 20
    drwxr-xr-x 2 root root 4096 Dec 4 10:18 .
    drwxr-xr-x 3 root root 4096 Dec 4 10:18 ..
    -rw-r--r-- 1 root root 5763 Dec 4 10:18 pipeline.yml
    -rw-r--r-- 1 root root 432 Dec 4 10:18 seed.yml
    Logging to /tmp/concourse-pipeline-resource-out.log425136443
    2018/12/04 10:28:27 exit status 1 - error: invalid argument for flag `-c, --config' (expected atc.PathFlag): path '/tmp/build/put/pipelineSchedular/ci/templates/pipeline.yml' does not exist

    failed

#6

This is my docker-compose.yml file to install concourse. I have only changed CONCOURSE_EXTERNAL_URL in order to reach concourse within the concourse containers. I am not sure but may it be related with the issue?

version: '3'

services:
  concourse-db:
    image: postgres
    environment:
    - POSTGRES_DB=concourse
    - POSTGRES_PASSWORD=concourse_pass
    - POSTGRES_USER=concourse_user
    - PGDATA=/database

  concourse:
    image: concourse/concourse
    command: quickstart
    privileged: true
    depends_on: [concourse-db]
    ports: ["8080:8080"]
    environment:
    - CONCOURSE_POSTGRES_HOST=concourse-db
    - CONCOURSE_POSTGRES_USER=concourse_user
    - CONCOURSE_POSTGRES_PASSWORD=concourse_pass
    - CONCOURSE_POSTGRES_DATABASE=concourse
#    - CONCOURSE_EXTERNAL_URL
    - CONCOURSE_EXTERNAL_URL=http://10.23.21.68:8080
    - CONCOURSE_ADD_LOCAL_USER=test:$$2a$$10$$0W9/ilCpYXY/yCPpaOD.6eCrGda/fnH3D4lhsw1Mze0WTID5BuiTW
    - CONCOURSE_MAIN_TEAM_ALLOW_ALL_USERS=true
    - CONCOURSE_WORKER_GARDEN_NETWORK

#7

Mmh one thing thats comes into my mind is that you missed to define an output in the task which comes before the put: my-pipelines.

Add an output e.g. outpus: [tested-pipeline] to debugContenct and move your pipeline.yml at the end of the task to tested-pipeline/pipeline.yml.

In your put step refer to config_file: tested-pipeline/pipeline.yml.

best,
D


#8

I have done it but it did not help.
However, by doing so I have realized that I made a typo in resource name (pipelineScheduler vs. pipelineSchedular). Now, it is working as expected.

Thank you for your time and efforts to help indeed.


#9

no worries, glad to help :slight_smile: