Concourse Pipeline Resource - set-pipeline from local worker fs

I need to build a deployment pipeline, that allows me to fetch a local file from the workers filesystem.

The concourse-pipeline-resource offers the config_file option, but this looks like it’ll only work if you’ve created the directory in a previous task.

Can Concourse be used to write a local file to a directory inside the pipeline container?

resource_types:
- name: concourse-pipeline
  type: docker-image
  source:
    privileged: true
    repository: concourse/concourse-pipeline-resource
    tag: 2.1.1

resources:
- name: my-pipelines
  type: concourse-pipeline
  source:
    teams:
    - name: main
      username: example
      password: example

jobs:
- name: set-my-pipelines
  plan:
  - put: my-pipelines
    params:
      pipelines:
      - name: pipeline1
        team: main
        config_file: pipelines/pipeline1.yaml <--- How do I get this local dir into the pipeline container

Hi @DandyDeveloper welcome to the forum.

Each step in a Concourse job runs in a different container. For security reasons I don’t believe there is any way to break out of a container into the filesystem of the worker that it is running on.

In Concourse you have two choices for getting that config_file is either:

  1. Generating the file in a task earlier on in the job and outputting it so it can be used in the put to the pipeline resource. This is the case you identified.

  2. Storing the file somewhere outside of Concourse (such as git or S3) and using a resource to get it.

@crsimmons I’m sorry for not replying to this. I didn’t get notified on your response :frowning: I found a better way around this. My understanding of the jobs was just fundamentally wrong.

Sorry!