Dynamic Job Names


#1

Hi all, I posted this on discord but thought there may be a different audience here - apologies if that’s bad form. I’ve been trying to get Concourse into my current project for the past 14 months and it looks like we’re getting somewhere - we’ve got a particular requirement though and I wondered if anyone could give me a confirmation/any tips. We want the name of a Job as displayed in the UI to be updated with the version that is ready to deploy (eg. the block says something like “Deploy-ServiceA-v.1.1.0”). I’ve seen that done in the past this way: https://github.com/pivotalservices/concourse-pipeline-samples/blob/master/concourse-pipeline-patterns/gated-pipelines/03-shipit-enhanced/gated-pipeline-03-shipit-enhanced.yml. Is that still the best/only way to do that or is there another method (or if there’s something coming in 5 that solves the visibility of a waiting/deployable version in a different way)? Thanks.


#2

Hello, I don’t think that changing the name of a job at runtime is a good idea, it will loose the build history each time it changes name.

In Concourse 5 I don’t think there will be anything that would help; on the other hand there is an existing discussion here Improved UI for manually triggered jobs? and a POC PR with a video that shows how it works: https://github.com/concourse/concourse/pull/3163

If the goal is to know that something new is deployable, then I see 2 possibilities:

  1. you could use a notification to an IM channel (search for the Concourse resources enabling IM or email notifications)
  2. the current Concourse UI already shows the “the visibility of a waiting/deployable version”, the trick is to look at the version history. See below for an example.

Consider this simple pipeline

deploy1

with the following characteristics:

  • the deploy step is manual, as can be seen by the non-triggering line
  • For the sake of keeping the example simple, I am using a git resource as the thing you want to deploy. Substitute that with the real artifact (eg, S3 resource).

Now, iff we look at the history of the versions:

deploy2

we see that version 515e has been deployed, while version 3bab has never been used: that version is waiting to be deployed.

The other gotcha is the following: there is a race between you manually triggering the “deploy” job and the git/S3 resource: it is possible that a new resource becomes available and so it will be the one deployed.

To avoid this, with Concourse < 5 you have to watch the whole pipeline before clicking “deploy” and be sure that nothing is running.

To avoid this with Concourse 5.x it will be easier, since a given version can be “pinned”.

With Concourse < 5, there is no pinning; there is something similar that is enable/disable (see the “switch on” symbol on the left of “ref”). The enable/disable doesn’t solve the race, but allows to go back in the past. For example, if you disable 3bab, what will be deployed will be the previous version.


#3

Hi marco-m, thanks for the response. So the reason that the team would like something like that is for easy visibility for non technical people - I know that for technical users it’s reasonably easy to see what’s going on (and the pinned resources look like a nice method). We’ve got quite a large non-technical set of people in our projects however (Project/Deliver/Release Manager type peeps - there’s quite a bit of long-standing old-school governance that we’re working with) and from a “big screen to see the state of the pipeline” point of view, just being able to see a box at the end of the pipeline with the waiting version in would make it easier for them. To be honest even for the technical peeps it would make it easier to glance at the screen and see that “OK, 1.2.1 of component X is ready to be deployed”.

Perhaps the ideal solution is not so much having dynamic job names as much as having an optional and dynamically configurable label that could be displayed under the job (or resource) name itself.


#4

You could create a dummy build, at the end of the Pipeline, which simply reflects the latest Build version. There, each time its changed (using FLY CLI for instance) its not such a big deal that the history is lost. That is however a hack … so have a careful think about that, your situation is not unique :wink:

The IM feed would be more useful IMO, or deploying to an Artifact Repository.


#5

So we’d be sticking things like Slack notifications in anyway, but I can understand them wanting a quick visual way of telling without scrolling through Slack history - and if that was at the same time as seeing the pipeline state it would be quite nice. There’s ways around that (even at a simple level we could effectively just frame the part of our test dashboard - that shows passed versions - alongside Concourse). I think the dummy jobs would work fine though - although I have to admit an optional label for UI elements could be nice - I could see that being particularly useful for pinned versions. Whether that was a generic label or something only available on resources to show the latest version I could see either being handy. I’m not sure how easy that would be to add - or if I’m alone in seeing it as useful (although to be fair a chunk of that usefulness in my case is convincing others in the business who view it as surprisingly important!).


#6

I think the original method you referenced is OK, or a Dummy Job to achieve the effect.

Do you expect to have lots of versions … because in that case I would suggest a Slack channel specifically for each components Version Notification. Then its very focused and useful information.

And then I would automate the deployment or subsequent processes so that you do not need the notification anymore!!! For me, when I build a component, it gets pushed to a Repository … and then I have other Pipelines which are watching that Resource Repository … they get triggered and do their thing - its automatic - we just look at the screen to see if anything has gone wrong.

I worked in an organisation which spent 3 or 4 years turning Jenkins CI into a Project Management Tool. One could argue that was not really the best thing to do! What you have is a common problem.

Cheers, Tim.


#7

So we’re approaching Continuous Delivery but not Continuous Deployment yet for historical and governance reasons. We actually build our artefacts and wrap them in Helm charts way to the left, most of our pipeline is then the various integration tests (we’ve got over 10 delivery teams delivering components so there’s a fair few artefacts kicking around in various stages of tests).

Again for the same historical/governance reasons there’s a process that is gone through on the release front - so that’s where having that immediate view of what versions are where in the pipeline (most specifically ready to release) would be useful as the team can match the releases to the notes/test reports that we generate.