Multiple paths for a task

#1

Hi,

I have a pipeline with a handful of tasks that are expensive (they take a lot of time, about 20 minutes to complete). Each task results in a docker image being pushed to my repository.

It’s not always necessary to run every task every time my pipeline runs - sometimes the previous docker image is up-to-date.

I’d like to be able to do something like this (pseudo-code):

check tagged_version_exists_in_docker:
    - on_success:
        do_nothing
    - on_failure:
        run_unit_tests
        build_docker_image
        push_to_docker_repo

In Concourse, when I have a failed step, I can’t change the status back to success at the end of the remediation steps. That is, I’d like to be able to have this compound step return a successful state at the end.

Is there a way to achieve what I’d like?
I’d rather not have each docker image generation step as its own pipeline, because having these steps in one overall pipeline is visually informative and convenient.

Thanks for any advice,
Brendon

0 Likes

#2

Hello,
I see your reasoning, but you cannot (and you shouldn’t) use on-success and friends to emulate conditional logic, they are there to handle failure mode.

The Concoursey way of doing this is to represent each check tagged_version_exists_in_docker as a Concourse resource. This resource should return a new version when it detects that something has changed, so that you can use it as a trigger, exactly as the git resource does.

To make an example, the https://github.com/concourse/github-release-resource will return a new version each time a new release is published on github of a given project.

Now I understand that your logic above is almost the opposite: this imaginary resource we are talking about should return a new version when the expected docker image is not there :wink:

What I suggest is to rethink the approach: can you change the pipeline to obtain the same result, infused by the Concoursey principle above ? To give more detailed suggestions I need more details :slight_smile:

0 Likes

#3

Try combining your workflow into a single task and use caching. (In other words, make your tasks idempotent if the work is cached)

0 Likes

#4

Thanks for the suggestion! At the moment I use the default docker image resource, and I can’t see how to use that and make a task idempotent in the way that I want to.

So I think to do this, I will have to make my task contain docker functionality and not rely on the docker-image-resource. My pseudo-code above will have logic within the task.

I’ll think about this some more… :slight_smile:

0 Likes