I think you’re now a far from Jenkins … Jenkins pipeline support is pretty good with balance between declarative and scripted (it can be mixed). The big thing here, is that it uses a pseudo-Groovy intepreter. Writing in pure Groovy style makes you often fight with CPS. But it has advantage to be close to code. Sometime it’s better sometime not. But by experience is more often better being close to code than external. As long as you have good design and makes external, external things that can change (i.e. infrastructure, URLs, credentials, etc.)
Jenkins main problem is the master. It delivers Web UI, manages slaves, dispatch works, store job data and metadata, … It scales very badly. Changes are in progress such as pluggable storage. There’s also some “work” (design ?) about a microservice-like architecture … As always with Jenkins things are continuously changing …
Other problem I have in Jenkins (and many other tools, including Gitlab CI), it lacks for real “pipeline” support. I mean you can generally cut your “pipeline” into sequential stages and only doing parallel inside them. While Concourse offers fan in/fan out : you can consume produced resources as they are available. It can really speed up your pipeline as you can start long checks very soon and join at latest moment.
My final problem regarding Jenkins is plugin/extension. In facts, there’s many problems there because Jenkins have many ways to offer integration of new tool/capabilities.
So, let’s start focusing on plugins. Plugins are install once for all. Meaning it can sometimes break something that some people don’t want (rare but it happens), or different teams may need absolutely different versions. Without speaking some incompatilities between plugins version … And only Jenkins admins can manage plugins (install / update / remove) generating a lot of work, regression testing, etc. Plugin management is really painful.
Let’s speak about tools (e.g. JDK, Node, …). Jenkins add many strategies over the years. First, were snowflakes workers : with preinstalled tool that you need to manually manage or use additionnal tools such as NVM or RVM. Then come Tools (and Custom Tools Plugin), that let you install new tool (or new version). But you have to manually manage decomissionning (i.e. prune) or your worker disk will be filled by tools over time. And more “recently” Docker integration with at least 3 different plugin and way to integrate.
On the opposite, Concourse rely natively on container both for jobs and resources. More, image and their versions are managed by each team. Making each one choose the best one for their needs. And Concourse clean-up containers and images.
Now storage management under Concourse is not satisfactory. Handling NPM/Yarn/Maven/etc. cache/local repository is hard and painful. it’s because you have few access to storage management. I know only three wayt to “manage” storage: resource, job and cache feature. None meet the needs. Jenkins let you access slave local storage. Gitlab CI have a “dynamic key”-base cache that let you better define how such data is shared.
Another problem I have with Concourse is to share piece of code/pipeline. I have successfully PoC-ed an auto-updating shared pipeline : https://github.com/loganmzz/concourse-presentation-introduction/tree/master/src/pipeline/05-Indus/02-autoupdate/indus-common. But don’t know how to provide some customization. May be some templating is required to manage it. But even if that, pipeline is basically just a workflow, and sometimes flow changes at some conditions. Scripted-nature of Jenkins pipeline shine a lot in this domain. I assumed some dynamic behavior can be introduced into Concourse pipeline through resources but it’s still very limited.
To complete this pseudo-review, I need to speak about secret management. Jenkins has credentials that can be scoped and store many different type of secrets: user/password, text, file, SSH key, certificates, … And Concourse integrates with external dedicated tool : Vault, Credhub, Amazon SSM and Amazon Secrets Manager. I never looked if Jenkins can also integrates with them … But I’m curious about it.
As far as I know Gitlab CI has only variables to support it. Not the best way to handle such data I think.
I have said no word about community/manpower but it seems obvious. Jenkins is there since many years with thousands users finding help/support and guys to work with you is very easy. On the opposite Concourse is very limited. I think Gitlab CI will also be better at this point.
A final note, I forgot to speak about documentation. I never used Gitlab CI (and so never read the documentation). For Concourse, there’s some behavior that’s not written in documentation but nothing terrible ; main part of DSL is documented enough and resource/task are just using images. For Jenkins, the main problem is the documentation is not versionned (only latest one is available). Most plugins are not well (or not at all) documented … Even the step reference doesn’t always describe well all parameters or how to format them.