Passing build_name between jobs

Hi.

Apologies if asked many times before, I’ve searched and searched

I want to pass the build number between the jobs as part of my experiment with ArgoCD

I’m failing misreable, any help is appreciated

resource_types:

  • name: metadata
    source:
    repository: olhtbr/metadata-resource
    tag: 2.0.1
    type: docker-image

  • name: keyval
    type: docker-image
    source:
    repository: moredhel/keyval-resource

resources:

  • name: meta
    type: metadata
  • name: keyval
    type: keyval

jobs:

  • name: build
    plan:

    • get: meta
    • task: simple-task
      config:
      platform: linux
      image_resource:
      type: docker-image
      source: { repository: alpine }
      run:
      path: sh
      args:
      - -exc
      - |
      echo “branch_name=1234” >> keyval/keyval.properties
      inputs:
      - name: meta
    • put: keyval
      params:
      file: keyval/keyval.properties
  • name: prod-deploy
    plan:

    • get: keyval
      passed:
      • build
        trigger: true
    • task: spit-it-out
      config:
      platform: linux
      image_resource:
      type: docker-image
      source: { repository: alpine }
      run:
      path: sh
      args:
      - -exc
      - |
      ls -larth
      env

You need to define an output in the simple-task task in the build job and write the build number to a file in that output directory. Then you also need to define keyval as an input to the spit-it-out task in the prod-deploy job.

If you don’t declare an output in the task that produces the build number then Concourse won’t mount a volume to the task container. This means that echo “branch_name=1234” >> keyval/keyval.properties is writing to the local disk on the container which gets reaped as soon as the task finishes thus making it unavailable to the put.

Similarly, resources that you get in a job are not implicitly mounted to task containers unless you specify them as inputs. So even though you are getting keyval in the second job it isn’t available to your task.

So something like (I haven’t tested this):

jobs:
- name: build
  plan:
  - get: meta
  - task: simple-task
    config:
      platform: linux
      image_resource:
        type: docker-image
        source: { repository: alpine }
      inputs:
      - name: meta
      outputs:
      - name: build_number
      run:
        path: sh
        args:
        - -exc
        - |
          echo “branch_name=1234” >> build_number/keyval.properties
  - put: keyval
    params:
      file: build_number/keyval.properties

- name: prod-deploy
  plan:
  - get: keyval
    passed:
    - build
    trigger: true
  - task: spit-it-out
    config:
      platform: linux
      image_resource:
        type: docker-image
        source: { repository: alpine }
      inputs:
      - keyval
      run:
        path: sh
        args:
        - -exc
        - |
          build_number=$(cat keyval/keyval.properties)

Thank you for the reply.

I’ve tried the above, however there is no such file in the next job with the build number, here is the second job output

  • ls -larth

total 12K

drwxr-xr-x 2 root root 4.0K Feb 1 09:09 build_number

drwxr-xr-x 3 root root 4.0K Feb 1 09:09 …

drwxr-xr-x 3 root root 4.0K Feb 1 09:09 .

  • cd build_number

  • ls -larth

total 20K

-rw-r–r-- 1 root root 30 Feb 1 09:09 file

-rw-r–r-- 1 root root 36 Feb 1 09:09 UUID

-rw-r–r-- 1 root root 32 Feb 1 09:09 UPDATED

drwxr-xr-x 2 root root 4.0K Feb 1 09:09 .

drwxr-xr-x 3 root root 4.0K Feb 1 09:09 …

  • cat file

build_number/keyval.properties

As if the file path has become the variable itself

Here is a working pipeline

resource_types:

  • name: job-passer
    type: docker-image
    source:
    repository: moredhel/keyval-resource

resources:

  • name: promote
    type: job-passer
    icon: arrow-right-drop-circle
    public: true

jobs:

  • name: testing
    plan:
    • task: testing
      config:
      platform: linux
      image_resource:
      type: docker-image
      source: {repository: node, tag: 10.15.3-alpine}
      run:
      path: sh
      args:
      - -exc
      - |
      echo “hello”
    • put: promote
      params:
      current: “testing”
  • name: acceptance
    public: true
    plan:
    • in_parallel:
      • get: promote
        passed: [testing]
        trigger: true
    • task: kubectl-apply
      config:
      inputs:
      • name: promote
        platform: linux
        image_resource:
        type: docker-image
        source: {repository: node, tag: 10.15.3-alpine}
        run:
        path: sh
        args:
        • -exc
        • |
          ls -l promote/

It looks like this is to do with the keyval resource that you’re using. I’ve never encountered it before. The example in my previous reply would work with a resource like S3 instead.

I don’t really understand what the keyval resource is doing. IMHO the concept of passing data between jobs without putting to an external resource goes against some of the core principles of Concourse. By keeping state external to Concourse and running everything in ephemeral containers the idea is that any run of a job is repeatable and predictable.

1 Like