Resource With No Versions


#1

How might one go about implementing a custom resource for something that has no explicit versions? For example, I have an endpoint that I’d like to treat as a resource, however it does not have any versions, it just returns whatever may be returned, which may be the same some of the time, and may be different other times.

My first thought was to just create some sort of unique hash of the return information and set that as its version, but then come the in portion for the resource, I have no way to use this hash in any way to pull down the exact same set of information as was found during the check phase. This is a super edge case, but if there is significant delay between the check and the in phases, then the data returned from the endpoint may differ slightly from the “version” found during the check phase.

Is there a way around this or suggestions on how this could be handled better and/or differently?

Thanks!


#2

In that case I guess you’ll have to fetch down all the info in the check phase and put it in the returned version json object. You can still put a hash in front and then base64 encode the blob into the version object. For small objects that should work, though I have no idea what Concourse’s limit is on the version json blob (which will end up in postgres at some point)


#3

Hello @jgoodhouse, when you find a solution, could you please post a summary in this thread ? I am sure it will help other people :slight_smile:


#4

That was something I had thought of. If I went that route I would basically be only reaching out to the API endpoint during the check phase and then loading the actual data during the in phase from the base64 encoded data in the version. Does that sound correct?


#5

Yes, that’s exactly what I meant


#6

I just did something similar with the cfssl resource I made: https://github.com/Snapkitchen/concourse-cfssl-resources

I store checksums of the files as s3 metadata, and then simply pull down the metadata and serve that as the version.

If the in phase requests a version that no longer exists, it will fail, as it checks the checksum on download, as well (e.g.: https://github.com/Snapkitchen/concourse-cfssl-resources/blob/master/lib/concourse.py#L540).