Telling git resource to recheck a tag that's been force-pushed

#1

I’ve got a git resource defined like so:

- name: versions
  type: git
  source:
    uri: git@bitbucket.org:whatever/whatever.git
    branch: 'master'
    private_key: {{deploy_key}}
    tag_filter: "v*"

which works great and tags get properly built whenever I push a new one.

The problem is a tag has already been built, but I tagged the wrong commit. So I force-pushed the same tag name to a new commit hash. Now I can see that concourse is looking at the tag’s old commit hash, so it won’t rebuild it.

How can I tell Concourse to forget the old tag location, check again, and build the tag at its new commit?

Solution?

I did the following, which seems to have “worked”. I’m not sure if I inadvertently broke something else in the process:

  1. I opened a psql prompt into the Concourse database
  2. I found the version that was wrong (in my case, v3.7.5) via
select * from versioned_resources where version like '%v3.7.5%';
  1. Once I confirmed that returned one row and was pointing at the wrong commit hash, I did
update versioned_resources set version='{"ref":"v3.7.5-bad"}' where version like '%v3.7.5%';
  1. I started a new build from the Concourse web frontend, and that seemed to pick up v3.7.5 again, this time pointing at the right commit hash.
0 Likes

#2

Generally Concourse resource versions are treated as being immutable so there isn’t any built in feature that does what you want afaik. The preferred workflow would be to just make another tag on the correct commit.

Recreating the worker(s) forces Concourse to download the resource again on the get but I’m not sure if it would force it to pick up the new commit SHA.

0 Likes

#3

I am quite stubborn about trying to do what I want. Software should be obedient. I think I managed to do it? I understand if there isn’t a built-in way, but I think messing with its db might have worked.

0 Likes

#4

Fair enough. Concourse is very opinionated on things like this. Relevant issue.

0 Likes

#5

I am not sure what you are asking then. You are breaking the “warranty seals” of Concourse, and not only of Concourse, of any sound engineering workflow. There are certain things of git that should be treated as immutable once published, and for sure git tags are one of these.

0 Likes

#6

I would rather not argue out of abstract principles. Git allows force-pushing tags to undo mistakes. I can make Concourse do the same. Making software do what I want is all the “engineering” I need.

I can see ways to make Concourse do the same. For example, could just delete the tag and any outputs it has, forcing Concourse to refetch it and try again. I didn’t just delete the database row because I was thinking that would probably disturb the schema in some way, but it seems that renaming seemed to have approximately the desired effect.

0 Likes

#7

This should be fixed in v1.1 of the git resource, which is included in Concourse 5.0+.

The fix was this commit, which makes it so the resource versions versions also include the commit that their tag points to. So if the tag points to a new commit, it’ll result in a new version, and then result in a new cache.

FWIW I’d be happy to see a PR for fly clear-resource-cache, analogous to fly clear-task-cache. We initially pushed back on it and hoped to force resources (and humans) to be responsible with their versioning, but it’d still be nice to have that command around when you’re in a pinch. It should be pretty straightforward to implement something like this:

fly clear-resource-cache -r pipeline-name/resource-name

The API endpoint would just need to run this query:

DELETE FROM worker_resource_caches wrc
USING resource_caches rc, resources r
WHERE wrc.id = rc.id
AND rc.resource_config_id = r.resource_config_id
AND r.id = $1

(where $1 is the id of the resource identified by the -r flag)

This will delete the cache entries from the worker and result in their cache volumes being GC’d once they’re no longer in use.

2 Likes