Expoxt teams and pipelines


#1

For DR, we employ a number of fallbacks. 2 include data from the DB. Worst case is keeping a backup of the teams table, so if we have to zap the DB, we can repopulate with the configured teams. We would also like to include pipelines. But this table seems to depend on other tables. Are there known ways to export and import pipelines between concouse DB’s using SQL

Bosh, 3.9.2, 500 pipelines


#2

I don’t have a direct answer but a potential alternative, although given the size you mention (500 pipelines) I don’t know how well it could work.

We use the Concourse pipeline resource https://github.com/concourse/concourse-pipeline-resource. All the “main” pipelines (the pipelines we want to survive a disaster) are listed in the pipeline handled by the Concourse pipeline resource.

When I started using the pipeline resource, I found its usage quite confusing, so I wrote a blog post that might be useful https://www.orsolabs.com/post/bootstrapping-concourse-pipelines/

Note: clearly this approach would NOT maintain the build history in case of loss of the Concourse database. On the other hand, it would allow to respawn automatically all the “main” pipelines, which is what is more important for us.

In addition to all the “main” pipelines we also have per-feature branch pipelines. We consider these disposable, we don’t track them.