Manual gate for deploy job

Hi There!

I am a kind of new to concourse but have CICD background with other tools like GoCD and Jenkins. I am trying to configure a manual gate infront of QA deploy job once after dev deployment occurs successfully.

Is there any reference available to me to look into advanced manual gate configurations for picking the particular version while there are multiple versions of code waiting in queue for deployment and any time out setting on the queue?

FYI - I was referring to the below ship-it documentation but that’s not exactly what I need to achieve. Appreciate your help!

Thanks
Sri

Hi @srihari_bobilla
If you’re looking for a way to specify a version of a Concourse resource a Concourse job should consume, then you could pin a resource.

This is a doc reference for pinning a resource using fly - https://concourse-ci.org/managing-resources.html

You can also pin a resource directly in the UI by visiting the resources page and clicking the pin icon

Make “qa_deploy” only get versions that have passed “dev_deploy”. Then set trigger to false on “qa_deploy”, so it will only run if triggered manually.

Another way I have done this is to have a QA deploy job trigger automatically based on some manifest file containing a version to deploy that lives in git. When you want to deploy QA, update the manifest file to the last good version in dev and send a PR. Merging the PR is the manual gate.

1 Like

Thanks @xtreme-sameer-vohra and @rjosephwright.

I have realized that the following manual-gate with shipit configuration may serve the need I am looking for.

While I have configured it and running the pipeline I am running into the error below. Any idea or did you guys ever tried that?

+ mv ./fly-release/fly_linux_amd64 ./fly

10:32:16

mv: can’t rename ‘./fly-release/fly_linux_amd64’: No such file or directory

Sounds like you aren’t getting the fly-release properly.

@crsimmons that’s right. fly-release resource configuration is not as straightforward as it seems. I needed to tweak around to get to that point. I manually wrote simple shell commands to untar the fly package and so on.

Glad you figured it out @srihari_bobilla. Some resources like the S3 one can do the unpacking for you when you get them. It’s too bad the github release resource doesn’t.

@rjosephwright our team is interested to move forward in the similar direction. Can you elaborate little more on how you configured your git resource pointing to the yaml file and if you can provide some example? Thanks!

I don’t have an example as it was done at another company. A simplified version of what I did was to create a git repo that had a directory for each environment. The directory contained a manifest file which contained a list of each application and the version, and other variables that were passed along to the applications.

Each environment had its own pipeline, and the concourse git resource was configured using “paths” to watch only that environment’s directory in the git repo for changes. It was also configured with “trigger: true”.

To update the version of an application, someone would send a PR to update the manifest file. When it was approved, merging the PR would trigger the pipeline.

When the pipeline deploy job triggered, it did a “get” on the git resource, then ran a helm resource step which read the manifest file to get the version and other variables, then ran helm to deploy the application, using the version and variables from the manifest.

Thanks @rjosephwright. Looks like helm resource is specifically meant for Kube stuff. In my case I can just use general git-resource using the path to the deploy yaml file location in git ripo

Sure @srihari_bobilla, in my case we were using Kubernetes and helm, but of course you can pass the manifest along to a task running a deploy script or whatever you want.

@here In the process of manual deploy gate implementation I have a requirement where the build job should automatically update a file with the new image tag ID in git repository. I wonder if anyone has experimented that part and how comfortable using git CLI commands within concourse

Also to put it in other words -

I am looking for something like the build job (after it successfully completed) should automatically update a deploy.yaml file in git repository with the new image tag ID. I wonder if anyone has experimented that part and how comfortable using git CLI within concourse jobs/tasks to execute clone, commits and stuff like that

Is there any reason why you can’t use the git resource to do your git operations? You can get the repo in the job then add it as an input and and define an output like -out to your task. Then in the task you can git clone <input> <output> and . modify the file in the repo and make a commit. Then just put the resource as the next step.

Something like:

#!/bin/bash

set -exu

git config --global user.email "somebody@example.com"
git config --global user.name "CI"
git clone ./git-repo ./git-repo-out

%change stuff in git-repo-out%

cd git-repo-out || exit
status="$(git status --porcelain)"
if [ -n "$status" ]; then
  git add -A
  git commit -m "changed stuff"
fi

@crsimmons That looks interesting. Thanks for throwing me the idea with some example. I am trying to implement in that direction. Let’s see.

@crsimmons I am doing something like this and hoping that will work the way I want but implementation still in progress. Just let me know your thoughts. Thanks!

task: update-deploy-trigger-file
file: code-repo/ci/tasks/deploy-trigger.yaml
params:
DIRECTORY: code-repo

========== my deploy-trigger.yaml =========
platform: linux
inputs:
- name: code-repo
- name: build_output
outputs:
- name: code-repo-updated
run:
path: code-repo/ci/scripts/deploy-trigger-file-update.sh

=========== my deploy-trigger-file-update.sh =======
#!/bin/bash
set -e
git clone ./$DIRECTORY ./code-repo-updated
cd ./code-repo-upddated/ci/tasks
cp build_output/version.txt .
git status
git add version.txt
git commit -m “updated with latest image tag”

So my job is failing with some wiard error though “no image plugin configured” :grinning:

You aren’t specifying an image for the task to run in. Follow the docs on tasks and add an image_resource pointing to a docker image with whatever tools you need in your script.

Also, you don’t need to specify the code-repo as a parameter on the task. By defining it as an input it will already be mounted in the container for you.

@crsimmons Thanks for the response. Understand what you are saying while I look at the tasks docs but what resource image I have to configure for this task ? should I use some generic git image resource?

You can choose any docker image that contains the tools you need. You can even write your own Dockerfile and build something custom. Ubuntu would have git in it (https://hub.docker.com/_/ubuntu/) but I’m sure there’s some lighter weight images out there.

Thanks @crsimmons @rjosephwright @xtreme-sameer-vohra. I have successfully completed the implementation.
Goal: Auto-trigger deploy jobs after successful build in GitOps way . Build job updates a version file in git which needs to be deployed to dev. As soon as new version gets auto-committed into the file, it kicks-off deploy job.

I have created a resource with “paths” - looking up to the version file and then

Here is my task config that looks like -

  • task: update-deploy-trigger-file
    file: my-repo/ci/tasks/update-deploy-trigger-file.yml
    • put: my-repo
      params:
      repository: my-repo-updated

Here is my update-deploy-trigger-file.yml


platform: linux

image_resource:
type: docker-image
source:
repository: node
tag: latest

inputs:

  • name: my-repo
  • name: build_output

outputs:

  • name: my-repo-updated

run:
path: my-repo/ci/scripts/update-deploy-trigger-file.sh

update-deploy-trigger-file.sh

#!/bin/bash

set -ex

git clone my-repo my-repo-updated
git config --global user.name “xxxxxxxxxxxx”
git config --global user.email “XXXXXXXXXXX”
cp build_output/version.txt my-repo-updated/ci/deploy-triggers/dev
cd my-repo-updated
git add .
git commit -m “commit latest version.txt file”
echo “done commiting latest version.txt file”