Regex does not match provided version

Am running into an issue with a new pipeline I’ve just set up. I’ve set up similar before so I’m not quite sure what I’m doing differently.

pipeline.yml

resources:
- name: config
  type: s3
  source:
    access_key_id: ((minio.username))
    bucket: ((minio-bucket))
    endpoint: ((minio-url))
    regexp: fe-configs/configs-*.tar
    secret_access_key: ((minio.passowrd)

(other git resources)

jobs:
- name: clean-and-push
  serial: true
  plan:
  - aggregate:
    - get: build-scripts (git resource)
    - get: git-configs (git resource)
      trigger: true
    - task: clean-configs
      file: build-scripts/tasks/clean-configs.yml
      params:
      input_mapping:
        concourse-repo: build-scripts
        repo: git-configs
    - put: config
      params:
        file: configs/configs-*.tar

clean-configs.yml just launches a script that zips up configs and puts them in a folder. The * in the regex is for the git hash I put in the name.

My pipeline puts the tar in minio but then spits out an error at the last second:
error running command: regex does not match provided version: s3resource.Version(Path: “fe-configs/configs-d371bveg1224g3vhg4v2hg3v.tar”, VersionID:"")
Which is a bit confusing since I’m not even using versions, nor does Minio support them.

Concourse 5.0.0

From a quick look, you have

regexp: fe-configs/configs-*.tar
...
file: configs/configs-*.tar

this might be the reason

I believe the former represents where to store the file in minio (s3), and the latter represents where the file to be uploaded is located.

Figured it out. I needed to use capture groups.
regexp: fe-configs/configs-*.tar
needed to become
regexp: fe-configs/configs-(.*).tar

So I’ve spent a little more time on this problem, and noticed this is still a frequently viewed topic. My previous solution of wrapping the regex in brackets does work to get rid of the error, but it is not suitable for “versioning” in minio. Instead, I used time/date then the shortened hash at the end such as:

fe-configs/configs-(\d+\.\d+\.\d+)(\.\w{8}).tar

The crux of this is that Concourse requires clearly defined versions for resources. With a versioned bucket the storage provider does the versioning for you so you can just reference a file name. Without versioned buckets you need to do the versioning yourself. This is why the S3 resource regexp requires at least one capture group. The capture group should contain something that can be versioned against (like a timestamp).

In theory -(.*).tar would work for you provided you append a timestamp to the filename when you create it before putting to the resource.