`error:forbidden` when using LDAP auth. [fixed]


#1

TL;DR: It looks like my LDAP is working but any instructions fail with error: forbidden.

(Concourse v 4.2.1)

I have mostly followed the instructions to set up Concourse on Ubuntu 16.04 from here, but with these settings in the web_environment file:

CONCOURSE_SESSION_SIGNING_KEY=/etc/concourse/session_signing_key
CONCOURSE_TSA_HOST_KEY=/etc/concourse/tsa_host_key
CONCOURSE_TSA_AUTHORIZED_KEYS=/etc/concourse/authorized_worker_keys
CONCOURSE_POSTGRES_SOCKET=/var/run/postgresql
CONCOURSE_LDAP_DISPLAY_NAME=Ident
CONCOURSE_LDAP_HOST=<ldap host>
CONCOURSE_LDAP_BIND_DN=<service account>
CONCOURSE_LDAP_BIND_PW=<service account pw>
CONCOURSE_LDAP_USER_SEARCH_BASE_DN=dc=<workspace>,dc=<company>,dc=<com>
CONCOURSE_LDAP_USER_SEARCH_ID_ATTR=uid
CONCOURSE_LDAP_USER_SEARCH_USERNAME=uid
CONCOURSE_LDAP_USER_SEARCH_EMAIL_ATTR=mail
CONCOURSE_LDAP_USER_SEARCH_NAME_ATTR=name
CONCOURSE_LDAP_INSECURE_SKIP_VERIFY=1
CONCOURSE_LDAP_INSECURE_NO_SSL=1

CONCOURSE_EXTERNAL_URL=http://<machine>:8080

I can log in using my LDAP credentials into the web frontend.
When trying to use fly:

$ ~/concourse/fly -t tutorial login -c http://<machine>:8080
logging in to team 'main'

navigate to the following URL in your browser:

  http://<machine>:8080/sky/login?redirect_uri=http://127.0.0.1:49603/auth/callback

or enter token manually:
target saved
$ ~/concourse/fly -t tutorial workers
name           containers  platform  tags  team  state    version
<machine>  0           linux     none  none  running  2.1
$ ~/concourse/fly -t tutorial execute -c task_hello_world.yml
error: forbidden

I’ve run the web and worker from the command-line to check the logs but I can’t see anything useful in those logs. (I can post the logs if that’s helpful, might do that in this thread)

Any help or hints greatly appreciated!


#2

Web logs:

$ concourse web
{"timestamp":"1537831813.692242622","source":"tsa","message":"tsa.listening","log_level":1,"data":{}}
{"timestamp":"1537831813.696560621","source":"atc","message":"atc.listening","log_level":1,"data":{"debug":"127.0.0.1:8079","http":"0.0.0.0:8080"}}
{"timestamp":"1537831817.546485186","source":"tsa","message":"tsa.connection.keepalive","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1","type":"keepalive"}}
{"timestamp":"1537831817.546772718","source":"tsa","message":"tsa.connection.channel.channel-request","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1.1","type":"exec"}}
{"timestamp":"1537831817.547101736","source":"tsa","message":"tsa.connection.tcpip-forward.forwarding-tcpip","log_level":1,"data":{"remote":"127.0.0.1:34490","requested-bind-addr":"0.0.0.0:7777","session":"1.2"}}
{"timestamp":"1537831817.547282934","source":"tsa","message":"tsa.connection.channel.forward-worker.forwarded-tcpip","log_level":1,"data":{"bindAddr":"0.0.0.0:7777","bound-port":40964,"remote":"127.0.0.1:34490","session":"1.1.1"}}
{"timestamp":"1537831817.547442198","source":"tsa","message":"tsa.connection.tcpip-forward.forwarding-tcpip","log_level":1,"data":{"remote":"127.0.0.1:34490","requested-bind-addr":"0.0.0.0:7788","session":"1.3"}}
{"timestamp":"1537831817.547595024","source":"tsa","message":"tsa.connection.channel.forward-worker.forwarded-tcpip","log_level":1,"data":{"bindAddr":"0.0.0.0:7788","bound-port":36497,"remote":"127.0.0.1:34490","session":"1.1.1"}}
{"timestamp":"1537831817.547681808","source":"tsa","message":"tsa.connection.channel.forward-worker.start","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1.1.1"}}
{"timestamp":"1537831817.547988415","source":"tsa","message":"tsa.connection.channel.forward-worker.done","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1.1.1"}}
{"timestamp":"1537831817.548100710","source":"tsa","message":"tsa.connection.channel.forward-worker.register.start","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1.1.1.3","worker-address":"127.0.0.1:40964","worker-platform":"linux","worker-tags":""}}
{"timestamp":"1537831817.567053556","source":"tsa","message":"tsa.connection.channel.forward-worker.register.done","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1.1.1.3","worker-address":"127.0.0.1:40964","worker-platform":"linux","worker-tags":""}}
{"timestamp":"1537831822.546749592","source":"tsa","message":"tsa.connection.keepalive","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1","type":"keepalive"}}
{"timestamp":"1537831827.546694040","source":"tsa","message":"tsa.connection.keepalive","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1","type":"keepalive"}}
{"timestamp":"1537831832.546716452","source":"tsa","message":"tsa.connection.keepalive","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1","type":"keepalive"}}
{"timestamp":"1537831837.546665192","source":"tsa","message":"tsa.connection.keepalive","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1","type":"keepalive"}}
{"timestamp":"1537831842.546528578","source":"tsa","message":"tsa.connection.keepalive","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1","type":"keepalive"}}
{"timestamp":"1537831847.545067787","source":"tsa","message":"tsa.connection.channel.channel-request","log_level":1,"data":{"remote":"127.0.0.1:34504","session":"2.1","type":"exec"}}
{"timestamp":"1537831847.553966522","source":"tsa","message":"tsa.connection.keepalive","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1","type":"keepalive"}}
{"timestamp":"1537831847.554172993","source":"tsa","message":"tsa.connection.channel.report-containers-worker.successfully-report-containers","log_level":1,"data":{"num-handles":0,"remote":"127.0.0.1:34504","session":"2.1.1"}}
{"timestamp":"1537831847.554264784","source":"tsa","message":"tsa.connection.channel.report-containers-worker.finished-reporting-containers","log_level":1,"data":{"num-handles":0,"remote":"127.0.0.1:34504","session":"2.1.1"}}
{"timestamp":"1537831847.560590267","source":"tsa","message":"tsa.connection.channel.channel-request","log_level":1,"data":{"remote":"127.0.0.1:34508","session":"3.1","type":"exec"}}
{"timestamp":"1537831847.564323187","source":"tsa","message":"tsa.connection.channel.report-volumes-worker.successfully-report-volumes","log_level":1,"data":{"num-handles":0,"remote":"127.0.0.1:34508","session":"3.1.1"}}
{"timestamp":"1537831847.564433098","source":"tsa","message":"tsa.connection.channel.report-volumes-worker.finished-reporting-volumes","log_level":1,"data":{"num-handles":0,"remote":"127.0.0.1:34508","session":"3.1.1"}}
{"timestamp":"1537831847.567775011","source":"tsa","message":"tsa.connection.channel.forward-worker.heartbeat.start","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1.1.1.4","worker-address":"127.0.0.1:40964","worker-platform":"linux","worker-tags":""}}
{"timestamp":"1537831847.571655989","source":"tsa","message":"tsa.connection.channel.channel-request","log_level":1,"data":{"remote":"127.0.0.1:34510","session":"4.1","type":"exec"}}
{"timestamp":"1537831847.578685522","source":"tsa","message":"tsa.connection.channel.sweep-containers-worker.successfully-sweep-containers","log_level":1,"data":{"remote":"127.0.0.1:34510","session":"4.1.1"}}
{"timestamp":"1537831847.578784943","source":"tsa","message":"tsa.connection.channel.sweep-containers-worker.finished-getting-sweep-containers","log_level":1,"data":{"handles":"[]\n","remote":"127.0.0.1:34510","session":"4.1.1"}}
{"timestamp":"1537831847.578951120","source":"tsa","message":"tsa.connection.channel.sweep-containers-worker.finished-writing-sweeper-containers","log_level":1,"data":{"bytes-written":3,"err":null,"remote":"127.0.0.1:34510","session":"4.1.1"}}
{"timestamp":"1537831847.584286213","source":"tsa","message":"tsa.connection.channel.forward-worker.heartbeat.done","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1.1.1.4","worker-address":"127.0.0.1:40964","worker-platform":"linux","worker-tags":""}}
{"timestamp":"1537831847.587137938","source":"tsa","message":"tsa.connection.channel.channel-request","log_level":1,"data":{"remote":"127.0.0.1:34522","session":"5.1","type":"exec"}}
{"timestamp":"1537831847.590368271","source":"tsa","message":"tsa.connection.channel.sweep-volume-worker.successfully-sweep-volumes","log_level":1,"data":{"remote":"127.0.0.1:34522","session":"5.1.1"}}
{"timestamp":"1537831847.590474606","source":"tsa","message":"tsa.connection.channel.sweep-volume-worker.finished-getting-sweep-volumes","log_level":1,"data":{"handles":"[]\n","remote":"127.0.0.1:34522","session":"5.1.1"}}
{"timestamp":"1537831847.590647221","source":"tsa","message":"tsa.connection.channel.sweep-volume-worker.finished-writing-sweeper-volumes","log_level":1,"data":{"bytes-written":3,"err":null,"remote":"127.0.0.1:34522","session":"5.1.1"}}
{"timestamp":"1537831852.546741724","source":"tsa","message":"tsa.connection.keepalive","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1","type":"keepalive"}}
{"timestamp":"1537831855.770945311","source":"tsa","message":"tsa.connection.tcpip-forward.interrupted","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1.2"}}
{"timestamp":"1537831855.772168398","source":"tsa","message":"tsa.connection.tcpip-forward.interrupted","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1.3"}}
{"timestamp":"1537831855.772257805","source":"tsa","message":"tsa.connection.channel.forward-worker.wait-for-process.failed-to-close-channel","log_level":2,"data":{"error":"EOF","remote":"127.0.0.1:34490","session":"1.1.1.2"}}
^C{"timestamp":"1537831894.935544252","source":"atc","message":"atc.drain.releasing-tracker","log_level":1,"data":{"session":"16"}}
{"timestamp":"1537831894.936171055","source":"atc","message":"atc.build-tracker.release.calling-release-on-builds","log_level":1,"data":{"session":"17.1"}}
{"timestamp":"1537831894.936485529","source":"atc","message":"atc.build-tracker.release.waiting-on-builds","log_level":1,"data":{"session":"17.1"}}
{"timestamp":"1537831894.936761856","source":"atc","message":"atc.build-tracker.release.calling-release-in-exec-engine","log_level":1,"data":{"session":"17.1"}}
{"timestamp":"1537831894.937038422","source":"atc","message":"atc.build-tracker.release.finished-waiting-on-builds","log_level":1,"data":{"session":"17.1"}}
{"timestamp":"1537831894.937224150","source":"atc","message":"atc.drain.released-tracker","log_level":1,"data":{"session":"16"}}
{"timestamp":"1537831894.937359571","source":"atc","message":"atc.drain.sending-atc-shutdown-message","log_level":1,"data":{"session":"16"}}

#3

Worker logs:

$ sudo ./start.sh
{"timestamp":"1537831817.526277542","source":"worker","message":"worker.setup.already-done","log_level":1,"data":{"session":"1"}}
{"timestamp":"1537831817.526623964","source":"worker","message":"worker.garden.extract-resources.extract.already-extracted","log_level":1,"data":{"resource-type":"bosh-io-release","session":"2.1.1"}}
{"timestamp":"1537831817.526799679","source":"worker","message":"worker.garden.extract-resources.extract.already-extracted","log_level":1,"data":{"resource-type":"bosh-io-stemcell","session":"2.1.2"}}
{"timestamp":"1537831817.526970625","source":"worker","message":"worker.garden.extract-resources.extract.already-extracted","log_level":1,"data":{"resource-type":"cf","session":"2.1.3"}}
{"timestamp":"1537831817.527136087","source":"worker","message":"worker.garden.extract-resources.extract.already-extracted","log_level":1,"data":{"resource-type":"docker-image","session":"2.1.4"}}
{"timestamp":"1537831817.527304888","source":"worker","message":"worker.garden.extract-resources.extract.already-extracted","log_level":1,"data":{"resource-type":"git","session":"2.1.5"}}
{"timestamp":"1537831817.527457952","source":"worker","message":"worker.garden.extract-resources.extract.already-extracted","log_level":1,"data":{"resource-type":"github-release","session":"2.1.6"}}
{"timestamp":"1537831817.527602434","source":"worker","message":"worker.garden.extract-resources.extract.already-extracted","log_level":1,"data":{"resource-type":"hg","session":"2.1.7"}}
{"timestamp":"1537831817.527748346","source":"worker","message":"worker.garden.extract-resources.extract.already-extracted","log_level":1,"data":{"resource-type":"pool","session":"2.1.8"}}
{"timestamp":"1537831817.527896166","source":"worker","message":"worker.garden.extract-resources.extract.already-extracted","log_level":1,"data":{"resource-type":"registry-image","session":"2.1.9"}}
{"timestamp":"1537831817.528046846","source":"worker","message":"worker.garden.extract-resources.extract.already-extracted","log_level":1,"data":{"resource-type":"s3","session":"2.1.10"}}
{"timestamp":"1537831817.528200150","source":"worker","message":"worker.garden.extract-resources.extract.already-extracted","log_level":1,"data":{"resource-type":"semver","session":"2.1.11"}}
{"timestamp":"1537831817.528347254","source":"worker","message":"worker.garden.extract-resources.extract.already-extracted","log_level":1,"data":{"resource-type":"time","session":"2.1.12"}}
{"timestamp":"1537831817.528491020","source":"worker","message":"worker.garden.extract-resources.extract.already-extracted","log_level":1,"data":{"resource-type":"tracker","session":"2.1.13"}}
{"timestamp":"1537831817.529824972","source":"baggageclaim","message":"baggageclaim.using-driver","log_level":1,"data":{"driver":"btrfs"}}
{"timestamp":"1537831817.529956818","source":"worker","message":"worker.sweep-starting","log_level":1,"data":{}}
{"timestamp":"1537831817.530330420","source":"guardian","message":"guardian.no-port-pool-state-to-recover-starting-clean","log_level":1,"data":{}}
{"timestamp":"1537831817.530868053","source":"guardian","message":"guardian.metrics-notifier.starting","log_level":1,"data":{"interval":"1m0s","session":"5"}}
{"timestamp":"1537831817.530941010","source":"guardian","message":"guardian.start.starting","log_level":1,"data":{"session":"6"}}
{"timestamp":"1537831817.532526731","source":"guardian","message":"guardian.cgroups-tmpfs-already-mounted","log_level":1,"data":{"path":"/sys/fs/cgroup"}}
{"timestamp":"1537831817.532686949","source":"guardian","message":"guardian.mount-cgroup.started","log_level":1,"data":{"path":"/sys/fs/cgroup/cpuset","session":"7","subsystem":"cpuset"}}
{"timestamp":"1537831817.532792807","source":"guardian","message":"guardian.mount-cgroup.subsystem-already-mounted","log_level":1,"data":{"path":"/sys/fs/cgroup/cpuset","session":"7","subsystem":"cpuset"}}
{"timestamp":"1537831817.532862902","source":"guardian","message":"guardian.mount-cgroup.finished","log_level":1,"data":{"path":"/sys/fs/cgroup/cpuset","session":"7","subsystem":"cpuset"}}
{"timestamp":"1537831817.532922983","source":"guardian","message":"guardian.creating-garden-cgroup.started","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/cpuset/garden","session":"8"}}
{"timestamp":"1537831817.532995224","source":"guardian","message":"guardian.creating-garden-cgroup.finished","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/cpuset/garden","session":"8"}}
{"timestamp":"1537831817.533041239","source":"guardian","message":"guardian.mount-cgroup.started","log_level":1,"data":{"path":"/sys/fs/cgroup/cpu","session":"9","subsystem":"cpu,cpuacct"}}
{"timestamp":"1537831817.533109665","source":"guardian","message":"guardian.mount-cgroup.subsystem-already-mounted","log_level":1,"data":{"path":"/sys/fs/cgroup/cpu","session":"9","subsystem":"cpu,cpuacct"}}
{"timestamp":"1537831817.533166647","source":"guardian","message":"guardian.mount-cgroup.finished","log_level":1,"data":{"path":"/sys/fs/cgroup/cpu","session":"9","subsystem":"cpu,cpuacct"}}
{"timestamp":"1537831817.533213854","source":"guardian","message":"guardian.creating-garden-cgroup.started","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/cpu/system.slice/ssh.service/garden","session":"10"}}
{"timestamp":"1537831817.533241749","source":"guardian","message":"guardian.creating-garden-cgroup.finished","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/cpu/system.slice/ssh.service/garden","session":"10"}}
{"timestamp":"1537831817.533288479","source":"guardian","message":"guardian.mount-cgroup.started","log_level":1,"data":{"path":"/sys/fs/cgroup/cpuacct","session":"11","subsystem":"cpu,cpuacct"}}
{"timestamp":"1537831817.533349991","source":"guardian","message":"guardian.mount-cgroup.subsystem-already-mounted","log_level":1,"data":{"path":"/sys/fs/cgroup/cpuacct","session":"11","subsystem":"cpu,cpuacct"}}
{"timestamp":"1537831817.533387423","source":"guardian","message":"guardian.mount-cgroup.finished","log_level":1,"data":{"path":"/sys/fs/cgroup/cpuacct","session":"11","subsystem":"cpu,cpuacct"}}
{"timestamp":"1537831817.533424139","source":"guardian","message":"guardian.creating-garden-cgroup.started","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/cpuacct/system.slice/ssh.service/garden","session":"12"}}
{"timestamp":"1537831817.533477545","source":"guardian","message":"guardian.creating-garden-cgroup.finished","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/cpuacct/system.slice/ssh.service/garden","session":"12"}}
{"timestamp":"1537831817.533520699","source":"guardian","message":"guardian.mount-cgroup.started","log_level":1,"data":{"path":"/sys/fs/cgroup/blkio","session":"13","subsystem":"blkio"}}
{"timestamp":"1537831817.533581734","source":"guardian","message":"guardian.mount-cgroup.subsystem-already-mounted","log_level":1,"data":{"path":"/sys/fs/cgroup/blkio","session":"13","subsystem":"blkio"}}
{"timestamp":"1537831817.533619642","source":"guardian","message":"guardian.mount-cgroup.finished","log_level":1,"data":{"path":"/sys/fs/cgroup/blkio","session":"13","subsystem":"blkio"}}
{"timestamp":"1537831817.533656359","source":"guardian","message":"guardian.creating-garden-cgroup.started","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/blkio/system.slice/ssh.service/garden","session":"14"}}
{"timestamp":"1537831817.533709049","source":"guardian","message":"guardian.creating-garden-cgroup.finished","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/blkio/system.slice/ssh.service/garden","session":"14"}}
{"timestamp":"1537831817.533751965","source":"guardian","message":"guardian.mount-cgroup.started","log_level":1,"data":{"path":"/sys/fs/cgroup/memory","session":"15","subsystem":"memory"}}
{"timestamp":"1537831817.533810854","source":"guardian","message":"guardian.mount-cgroup.subsystem-already-mounted","log_level":1,"data":{"path":"/sys/fs/cgroup/memory","session":"15","subsystem":"memory"}}
{"timestamp":"1537831817.533850670","source":"guardian","message":"guardian.mount-cgroup.finished","log_level":1,"data":{"path":"/sys/fs/cgroup/memory","session":"15","subsystem":"memory"}}
{"timestamp":"1537831817.533887863","source":"guardian","message":"guardian.creating-garden-cgroup.started","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/memory/system.slice/ssh.service/garden","session":"16"}}
{"timestamp":"1537831817.533941984","source":"guardian","message":"guardian.creating-garden-cgroup.finished","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/memory/system.slice/ssh.service/garden","session":"16"}}
{"timestamp":"1537831817.533964634","source":"guardian","message":"guardian.mount-cgroup.started","log_level":1,"data":{"path":"/sys/fs/cgroup/devices","session":"17","subsystem":"devices"}}
{"timestamp":"1537831817.534037352","source":"guardian","message":"guardian.mount-cgroup.subsystem-already-mounted","log_level":1,"data":{"path":"/sys/fs/cgroup/devices","session":"17","subsystem":"devices"}}
{"timestamp":"1537831817.534094572","source":"guardian","message":"guardian.mount-cgroup.finished","log_level":1,"data":{"path":"/sys/fs/cgroup/devices","session":"17","subsystem":"devices"}}
{"timestamp":"1537831817.534133434","source":"guardian","message":"guardian.creating-garden-cgroup.started","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/devices/system.slice/ssh.service/garden","session":"18"}}
{"timestamp":"1537831817.534186125","source":"guardian","message":"guardian.creating-garden-cgroup.finished","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/devices/system.slice/ssh.service/garden","session":"18"}}
{"timestamp":"1537831817.534559011","source":"guardian","message":"guardian.mount-cgroup.started","log_level":1,"data":{"path":"/sys/fs/cgroup/freezer","session":"19","subsystem":"freezer"}}
{"timestamp":"1537831817.534652710","source":"guardian","message":"guardian.mount-cgroup.subsystem-already-mounted","log_level":1,"data":{"path":"/sys/fs/cgroup/freezer","session":"19","subsystem":"freezer"}}
{"timestamp":"1537831817.534709215","source":"guardian","message":"guardian.mount-cgroup.finished","log_level":1,"data":{"path":"/sys/fs/cgroup/freezer","session":"19","subsystem":"freezer"}}
{"timestamp":"1537831817.534767628","source":"guardian","message":"guardian.creating-garden-cgroup.started","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/freezer/garden","session":"20"}}
{"timestamp":"1537831817.534827471","source":"guardian","message":"guardian.creating-garden-cgroup.finished","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/freezer/garden","session":"20"}}
{"timestamp":"1537831817.534891367","source":"guardian","message":"guardian.mount-cgroup.started","log_level":1,"data":{"path":"/sys/fs/cgroup/net_cls","session":"21","subsystem":"net_cls,net_prio"}}
{"timestamp":"1537831817.534961700","source":"guardian","message":"guardian.mount-cgroup.subsystem-already-mounted","log_level":1,"data":{"path":"/sys/fs/cgroup/net_cls","session":"21","subsystem":"net_cls,net_prio"}}
{"timestamp":"1537831817.535020828","source":"guardian","message":"guardian.mount-cgroup.finished","log_level":1,"data":{"path":"/sys/fs/cgroup/net_cls","session":"21","subsystem":"net_cls,net_prio"}}
{"timestamp":"1537831817.535078764","source":"guardian","message":"guardian.creating-garden-cgroup.started","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/net_cls/garden","session":"22"}}
{"timestamp":"1537831817.535139322","source":"guardian","message":"guardian.creating-garden-cgroup.finished","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/net_cls/garden","session":"22"}}
{"timestamp":"1537831817.535200834","source":"guardian","message":"guardian.mount-cgroup.started","log_level":1,"data":{"path":"/sys/fs/cgroup/perf_event","session":"23","subsystem":"perf_event"}}
{"timestamp":"1537831817.535268545","source":"guardian","message":"guardian.mount-cgroup.subsystem-already-mounted","log_level":1,"data":{"path":"/sys/fs/cgroup/perf_event","session":"23","subsystem":"perf_event"}}
{"timestamp":"1537831817.535323858","source":"guardian","message":"guardian.mount-cgroup.finished","log_level":1,"data":{"path":"/sys/fs/cgroup/perf_event","session":"23","subsystem":"perf_event"}}
{"timestamp":"1537831817.535379410","source":"guardian","message":"guardian.creating-garden-cgroup.started","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/perf_event/garden","session":"24"}}
{"timestamp":"1537831817.535440445","source":"guardian","message":"guardian.creating-garden-cgroup.finished","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/perf_event/garden","session":"24"}}
{"timestamp":"1537831817.535503387","source":"guardian","message":"guardian.mount-cgroup.started","log_level":1,"data":{"path":"/sys/fs/cgroup/net_prio","session":"25","subsystem":"net_cls,net_prio"}}
{"timestamp":"1537831817.535574675","source":"guardian","message":"guardian.mount-cgroup.subsystem-already-mounted","log_level":1,"data":{"path":"/sys/fs/cgroup/net_prio","session":"25","subsystem":"net_cls,net_prio"}}
{"timestamp":"1537831817.535631657","source":"guardian","message":"guardian.mount-cgroup.finished","log_level":1,"data":{"path":"/sys/fs/cgroup/net_prio","session":"25","subsystem":"net_cls,net_prio"}}
{"timestamp":"1537831817.535686970","source":"guardian","message":"guardian.creating-garden-cgroup.started","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/net_prio/garden","session":"26"}}
{"timestamp":"1537831817.535746098","source":"guardian","message":"guardian.creating-garden-cgroup.finished","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/net_prio/garden","session":"26"}}
{"timestamp":"1537831817.535788059","source":"guardian","message":"guardian.mount-cgroup.started","log_level":1,"data":{"path":"/sys/fs/cgroup/hugetlb","session":"27","subsystem":"hugetlb"}}
{"timestamp":"1537831817.535837889","source":"guardian","message":"guardian.mount-cgroup.subsystem-already-mounted","log_level":1,"data":{"path":"/sys/fs/cgroup/hugetlb","session":"27","subsystem":"hugetlb"}}
{"timestamp":"1537831817.535874367","source":"guardian","message":"guardian.mount-cgroup.finished","log_level":1,"data":{"path":"/sys/fs/cgroup/hugetlb","session":"27","subsystem":"hugetlb"}}
{"timestamp":"1537831817.535912514","source":"guardian","message":"guardian.creating-garden-cgroup.started","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/hugetlb/garden","session":"28"}}
{"timestamp":"1537831817.535952806","source":"guardian","message":"guardian.creating-garden-cgroup.finished","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/hugetlb/garden","session":"28"}}
{"timestamp":"1537831817.535994291","source":"guardian","message":"guardian.mount-cgroup.started","log_level":1,"data":{"path":"/sys/fs/cgroup/pids","session":"29","subsystem":"pids"}}
{"timestamp":"1537831817.536042929","source":"guardian","message":"guardian.mount-cgroup.subsystem-already-mounted","log_level":1,"data":{"path":"/sys/fs/cgroup/pids","session":"29","subsystem":"pids"}}
{"timestamp":"1537831817.536079168","source":"guardian","message":"guardian.mount-cgroup.finished","log_level":1,"data":{"path":"/sys/fs/cgroup/pids","session":"29","subsystem":"pids"}}
{"timestamp":"1537831817.536130190","source":"guardian","message":"guardian.creating-garden-cgroup.started","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/pids/system.slice/ssh.service/garden","session":"30"}}
{"timestamp":"1537831817.536158323","source":"guardian","message":"guardian.creating-garden-cgroup.finished","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/pids/system.slice/ssh.service/garden","session":"30"}}
{"timestamp":"1537831817.536204576","source":"guardian","message":"guardian.mount-cgroup.started","log_level":1,"data":{"path":"/sys/fs/cgroup/systemd","session":"31","subsystem":"name=systemd"}}
{"timestamp":"1537831817.536305904","source":"guardian","message":"guardian.mount-cgroup.subsystem-already-mounted","log_level":1,"data":{"path":"/sys/fs/cgroup/systemd","session":"31","subsystem":"name=systemd"}}
{"timestamp":"1537831817.536362410","source":"guardian","message":"guardian.mount-cgroup.finished","log_level":1,"data":{"path":"/sys/fs/cgroup/systemd","session":"31","subsystem":"name=systemd"}}
{"timestamp":"1537831817.536399364","source":"guardian","message":"guardian.creating-garden-cgroup.started","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/systemd/system.slice/ssh.service/garden","session":"32"}}
{"timestamp":"1537831817.536445141","source":"guardian","message":"guardian.creating-garden-cgroup.finished","log_level":1,"data":{"gardenCgroup":"/sys/fs/cgroup/systemd/system.slice/ssh.service/garden","session":"32"}}
{"timestamp":"1537831817.536491632","source":"guardian","message":"guardian.create-global-iptables-chains.started","log_level":1,"data":{"session":"2"}}
{"timestamp":"1537831817.537398338","source":"guardian","message":"guardian.create-global-iptables-chains.create-skipped","log_level":1,"data":{"session":"2"}}
{"timestamp":"1537831817.539342642","source":"guardian","message":"guardian.create-global-iptables-chains.finished","log_level":1,"data":{"session":"2"}}
{"timestamp":"1537831817.539448977","source":"guardian","message":"guardian.start.clean-all-peas.start","log_level":1,"data":{"session":"6.1"}}
{"timestamp":"1537831817.539521217","source":"guardian","message":"guardian.start.clean-all-peas.end","log_level":1,"data":{"session":"6.1"}}
{"timestamp":"1537831817.539585829","source":"guardian","message":"guardian.start.completed","log_level":1,"data":{"session":"6"}}
{"timestamp":"1537831817.539635181","source":"guardian","message":"guardian.list-containers.starting","log_level":1,"data":{"session":"33"}}
{"timestamp":"1537831817.539696217","source":"guardian","message":"guardian.list-containers.finished","log_level":1,"data":{"session":"33"}}
{"timestamp":"1537831817.539753914","source":"guardian","message":"guardian.started","log_level":1,"data":{"addr":"127.0.0.1:7777","network":"tcp"}}
{"timestamp":"1537831817.540021181","source":"guardian","message":"guardian.metrics-notifier.started","log_level":1,"data":{"interval":"1m0s","session":"5","time":"2018-09-25T09:30:17.540019878+10:00"}}
{"timestamp":"1537831817.540471077","source":"baggageclaim","message":"baggageclaim.listening","log_level":1,"data":{"addr":"127.0.0.1:7788"}}
{"timestamp":"1537831817.548100710","source":"tsa","message":"tsa.connection.channel.forward-worker.register.start","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1.1.1.3","worker-address":"127.0.0.1:40964","worker-platform":"linux","worker-tags":""}}
{"timestamp":"1537831817.549074173","source":"guardian","message":"guardian.list-containers.starting","log_level":1,"data":{"session":"34"}}
{"timestamp":"1537831817.549221039","source":"guardian","message":"guardian.list-containers.finished","log_level":1,"data":{"session":"34"}}
{"timestamp":"1537831817.567053556","source":"tsa","message":"tsa.connection.channel.forward-worker.register.done","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1.1.1.3","worker-address":"127.0.0.1:40964","worker-platform":"linux","worker-tags":""}}
{"timestamp":"1537831847.530361891","source":"worker","message":"worker.reporting-containers","log_level":1,"data":{}}
{"timestamp":"1537831847.531468391","source":"guardian","message":"guardian.list-containers.starting","log_level":1,"data":{"session":"35"}}
{"timestamp":"1537831847.531943321","source":"guardian","message":"guardian.list-containers.finished","log_level":1,"data":{"session":"35"}}
{"timestamp":"1537831847.564619303","source":"worker","message":"worker.sweep","log_level":1,"data":{"cmd":"sweep-containers"}}
{"timestamp":"1537831847.567775011","source":"tsa","message":"tsa.connection.channel.forward-worker.heartbeat.start","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1.1.1.4","worker-address":"127.0.0.1:40964","worker-platform":"linux","worker-tags":""}}
{"timestamp":"1537831847.571243525","source":"guardian","message":"guardian.list-containers.starting","log_level":1,"data":{"session":"36"}}
{"timestamp":"1537831847.571365356","source":"guardian","message":"guardian.list-containers.finished","log_level":1,"data":{"session":"36"}}
{"timestamp":"1537831847.579114199","source":"worker","message":"worker.sweep","log_level":1,"data":{"cmd":"sweep-volumes"}}
{"timestamp":"1537831847.584286213","source":"tsa","message":"tsa.connection.channel.forward-worker.heartbeat.done","log_level":1,"data":{"remote":"127.0.0.1:34490","session":"1.1.1.4","worker-address":"127.0.0.1:40964","worker-platform":"linux","worker-tags":""}}
{"timestamp":"1537831847.591017008","source":"baggageclaim","message":"baggageclaim.api.volume-server.destroy-volumes.waiting-for-volumes-to-be-destroyed","log_level":1,"data":{"session":"5.1.4"}}
{"timestamp":"1537831847.591123104","source":"baggageclaim","message":"baggageclaim.api.volume-server.destroy-volumes.destroyed","log_level":1,"data":{"session":"5.1.4"}}
^C{"timestamp":"1537831855.770570755","source":"worker","message":"worker.sweeper.exiting-from-mark-and-sweep","log_level":1,"data":{"session":"6"}}
{"timestamp":"1537831855.771287918","source":"guardian","message":"guardian.api.garden-server.waiting-for-connections-to-close","log_level":1,"data":{"session":"3.1"}}
{"timestamp":"1537831855.773329258","source":"guardian","message":"guardian.api.garden-server.stopping-backend","log_level":1,"data":{"session":"3.1"}}
{"timestamp":"1537831855.773700953","source":"guardian","message":"guardian.api.garden-server.stopped","log_level":1,"data":{"session":"3.1"}}


#4

Found the problem, had to add this line to my web_environment file:
CONCOURSE_MAIN_TEAM_ALLOW_ALL_USERS=1