How to tag the docker image with release tag name dynamically


#1

Hi Team,

I am running the below task script to get the latest release tag name dynamically, but not able to store output in s3/git using semver resource

eg:

  • name: version
    type: semver
    source:
    driver: git
    #initial_version: 0.0.0
    uri: https
    branch: master
    file: version
    • task: update-git-tag
      config:
      platform: linux
      image_resource:
      type: docker-image
      source:
      repository: alpine/git
      inputs:
  • name: nginx-deployment
    outputs:
  • name: tags
    run:
    dir: nginx-deployment
    path: sh
    args:
  • -excu
  • |
    git clone https://example.git
    cd bv-demo
    mkdir tags
    git describe --tags (git rev-list --tags --max-count=1) > tags/tag.txt echo (cat tags/tag.txt)>tags/tag.txt need to pass above output value in above semver resource
  • get: version
  • put: version
    params:
    file: version/version
    tag: tags/tag.txt ###not able to pass the output value here

Thanks,
Sanjay


#2

Hi sanjay,

i think this is not a valid use case for the semver resource. I suggest you use the Concourse s3 Resource with file versioning (specify the versioned_file: source parameter).

The semver resource is there to, once you initialised the semver value, to increment this semver using the put step with the bump: major/minor/patch parameter.

best,
D


#3

can we pass versioned_file: from task output to s3 resource job

task script:

  • task: update-git-tag
    config:
    platform: linux
    image_resource:
    type: docker-image
    source:
    repository: alpine/git
    inputs:
    - name: nginx-deployment
    outputs:
    - name: tags
    run:
    dir: nginx-deployment
    path: sh
    args:
    - -excu
    - |
    git clone https://@bitbucket.org/bv-demo.git
    cd bv-demo
    mkdir tags
    git describe --tags (git rev-list --tags --max-count=1) > tags/tag.txt echo (cat tags/tag.txt)>tags/tag.txt
    export VERSION=cat tags/tag.txt >tags/tag.txt

job:

  • put: s3
    params:
    versioned_file: tags/tag.txt

#4

Close but versioned_file is a parameter set in the resource configuration. In the put you would just use file: tags/tags.txt.


#5

I have deployed above pipeline but it giving the below error

" error running command: no matches found for pattern: tags/tag.txt"

run:

        path: sh
        args:
        - -excu
        - |
          apk add --no-cache bash git openssh
          git clone https://bv-demo.git
          cd bv-demo
          mkdir tags
          git describe --tags $(git rev-list --tags --max-count=1) > tags/tag.txt
          echo $(cat tags/tag.txt)>tags/tag.txt
          export VERSION=`cat tags/tag.txt` >tags/tag.txt
       

  #- get: s3-resource
  - put: s3-resource
    params:
      file: tags/tag.txt

#6

hijack the container, execute the script manually and take a look if tags/tag.txt was created.
fly -t <my_env> hijack -j <my_pipeline>/<my_job>
Always take the last container which matches you desired task name in list.

If that is not solving your problem, you should provide the whole pipeline e.g. as Gist… maybe the resource is misconfigured or something else is wrong.

best,
D


#8

Idk if this is fixing it but seems to be part of the problem

:wink:


#9

I have tried with commenting but initially tried with versioned_file

It given same error in both ways


#10

here a snippet of one of my s3 versioned_file resources:

- name: last-release
  type: s3
  source:
    bucket: ((bucket_name))
    access_key_id: ((access_key_id))
    secret_access_key: ((secret_access_key))
    region_name: ((region))
    versioned_file: release.yml

Is your identation broken because you pasted it in here? If not i would look into that or hijack the container (always a good debugging tool).

I would also encourage you to use a secrets file or Credhub instead of having secrets in the pipeline.

fly -t <target> sp -p fancy-pipelines -c pipeline.yml -l secrets-file.yml

Most likely you will have your pipeline code saved to version control and having the secrets in Github is not a good idea :slight_smile:

best,
D


#11

fly -t test hijack -j dynamic-versions/build-number-update
1: build #17, step: s3-resource, type: put
2: build #17, step: update-git-tag, type: task
choose a container: 1
bash-4.4# ls
nginx-deployment tags
bash-4.4# cd tags
bash-4.4# ls
bash-4.4# ls
bash-4.4#

there is no tag.txt file in directory


#12

Then there is something wrong with your script.

After hijacking you know:

  1. The ouptut folder tags was created -> thats already something :smile:
  2. The script was executed and

probably haven’t received the list of tags.

So you should execute these commands and see what is going on

To make your live more convenient, i suggest you reference real bash scripts in your task files.
This way you can hijack the container and execute the script directly instead of trying out all commands manually.

It is even possible to have a separate definition of your task in a task.yml and reference only the task within your pipeline.

That makes you pipeline much more readable and maintainable.

Here an example task:


---
platform: linux

image_resource:
  type: docker-image
  source:
    repository: gmbh/a9s-automation

inputs:
- name: scripts

params:
  BOSH_CLIENT:
  BOSH_CLIENT_SECRET:
  BOSH_CA_CERT:
  BOSH_ENVIRONMENT:
  BOSH_ENVIRONMENT_NAME:
  BOSH: "bosh -n"

run: 
  path: bash
  args:
    - scripts/pipelines/deployment-update/scripts/bosh-cleanup-all.sh

The pipeline requires now a git resource scripts in which you can find the script scripts/pipelines/deployment-update/scripts/bosh-cleanup-all.sh.

If you hijack the container you can simply do bash scripts/pipelines/deployment-update/scripts/bosh-cleanup-all.sh.

Thats a snippet of the pipeline that uses this task:

- task: bosh-cleanup-all-underbosh
      file: scripts/pipelines/deployment-update/tasks/bosh-cleanup-all.yml
      params:
        BOSH_CLEANUP_ALL: true

best,
D


#13

This i found from UI

  • cd bv-demo
  • mkdir tags
  • git rev-list --tags ‘–max-count=1’
  • git describe --tags e7586662d3b4ae947a1d63231a846c6516c0b921
  • cat tags/tag.txt
  • echo REL-20180805
  • cat tags/tag.txt
  • export ‘VERSION=REL-20180805
    REL-20180805’

#14

Hi

i am trying in below way but getting below error:

" sh: can’t open nginx-deployment/k8s/tag.yml: no such file"

  • task: update-git-tag
    config:
    platform: linux
    image_resource:
    type: docker-image
    source:
    repository: alpine
    run:
    path: sh
    args:
    - -excu
    - |
    apk add --no-cache bash git openssh
    git clone https:///bv-demo.git
    cd bv-demo
    export VERSION1=(git describe --tags (git rev-list --tags --max-count=1))
    apk add --no-progress gettext
    envsubst < nginx-deployment/k8s/tag.yml > output/tag.yml
    inputs:
    • name: nginx-deployment
    • name: s3-resource
      outputs:
    • name: output
  • put: s3-resource
    params:
    file: output/tag.yml

#15

First off, can you wrap your pipeline samples in ‘```’ in your comments so that spacing is conserved. Otherwise its hard read.

In

run:
      path: sh
      args:
      - -excu
      - |
        apk add --no-cache bash git openssh
        git clone https:///bv-demo.git
        cd bv-demo
        export VERSION1=(git describe --tags (git rev-list --tags --max-count=1))
        apk add --no-progress gettext
        envsubst < nginx-deployment/k8s/tag.yml > output/tag.yml

When this block of shell executes you are in a directory with nginx-deployment, s3-resource, and output in it. Then you clone bv-demo and change into that directory. The reason it can’t find nginx-deployment/k8s is because the path bv-demo/nginx-deployment doesn’t exist. It would probably work if you changed it to envsubst < ../nginx-deployment/k8s/tag.yml > ../output/tag.yml

Why are you git cloning in the script instead of pulling in that repo using the git resource?


#16

its working now and able to upload file to s3


#17

@ crsimmons gdenn

Thanks a lot :slight_smile:


#18

Hi,

now concourse deployment automatically triggering whenever changes done in master branch

  • name: nginx-deployment
    type: git
    source:
    username: ((git username))
    password: ((git password))
    uri: ((giturl))
    branch: master

but automatic build not triggering whenever changes done in tags and I have used tag_filter: “REL-201809*” but it not worked