Kubernetes & knative


As the Concourse team continues our sporadic work on Kubernetes and its Helm chart, we’re also starting to expand our thinking to cover the runtime aspects of Concourse + Kubernetes. We’ve already prioritized the need to have Kubernetes as a supporting backend in addition to Garden, but what about the spiffy new developments in the Kubernetes world? We’re hearing a lot about knative and knative services like build and eventing. Are there any kubernetes users who’d like to weigh in on the topic?


For some background, I’ve been pitching the idea of folding an evolved Concourse into Knative as the default onramp. A lot of the features in the near and medium term Build roadmap are essentially features that Concourse already has.

The main technical blocker is, of course, that Concourse is not using Kubernetes as its container scheduler. So while it can be run inside Kubernetes, it’s not really using Kubernetes yet.

It would definitely be useful to hear more about how you’re using Concourse in your Kubernetes-related workflows today. The more the merrier.


To expand a little, here’s an example Knative Build:

apiVersion: build.knative.dev/v1alpha1
kind: Build
  name: example-build
  serviceAccountName: build-auth-example
      url: https://github.com/example/build-example.git
      revision: master
  - name: ubuntu-example
    image: ubuntu
    args: ["ubuntu-build-example", "SECRETS-example.md"]
  - image: gcr.io/example-builders/build-example
    args: ['echo', 'hello-example', 'build']

If you squint a bit, the source section is really describing a git resource for both check and get and steps is a job with some tasks in it, one of which might behave like a put.

Currently under discussion is an idea of some kind of upload step so that artifact metadata can be captured (you will need to join this google group to access the file). From the Concourse perspective this is a put.

During my time working on the topic Topher & I wrote a façade service for Kubernetes that takes a Pipeline CRD from kubectl and forwards it to Concourse. The resulting YAML should look pretty familiar:

apiVersion: concourse.concourse-ci.org/v5alpha1
kind: Pipeline
  name: pipeline.example.com
  - name: git-repo
    type: git
      uri: https://example.com/repo.git
  - name: successful-job
    public: true
    - get: git-repo
    - task: job-task
      file: git-repo/tasks/successful/task.yml

Then it becomes possible to check the build:

$ kubectl describe pipeline pipeline.example.com
Name:         pipeline.example.com
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"concourse.concourse-ci.org/v5alpha1","kind":"Pipeline","metadata":{"annotations":{},"name":"pipeline.example.com","namespace":"default"}...
API Version:  concourse.concourse-ci.org/v5alpha1
Kind:         Pipeline
  Cluster Name:
  Creation Timestamp:  2018-08-14T17:49:11Z
  Generation:          1
  Resource Version:    2298601
  Self Link:           /apis/concourse.concourse-ci.org/v5alpha1/namespaces/default/pipelines/pipeline.example.com
  UID:                 59a83bc4-9fea-11e8-a301-42010a800163
    Name:  successful-job
      Get:   git-repo
      File:  git-repo/tasks/successful/task.yml
      Task:  job-task
    Name:    failed-job
    Name:  git-repo
      Uri:  https://example.com/repo.git
    Type:   git
    Duration:                52.705731518s
    End Time:                <nil>
    Job Name:                successful-job
    Start Time:              2018-08-14T17:58:59Z
    Status:                  started
    URL:                     https://ci.example.com/teams/main/pipelines/pipeline.example.com/jobs/successful-job/builds/1
    Duration:                20s
    End Time:                2018-08-14T17:59:20Z
    Job Name:                failed-job
    Start Time:              2018-08-14T17:59:00Z
    Status:                  failed
    URL:                     https://ci.example.com/teams/main/pipelines/pipeline.example.com/jobs/failed-job/builds/1
  Concourse Version:         4.0.0
  Concourse Worker Version:  2.1
  Paused:                    true
  Pipeline Set:              true
  Pipeline URL:              https://ci.example.com/teams/main/pipelines/pipeline.example.com
  Public:                    false
Events:                      <none>

The point of this work was to show that, at a minimum, there was a pathway to converging the two worlds.

So. Thoughts?