Terraform for running concourse in AWS

#1

Hey all. This isn’t an advertisement–we expect nothing from this nor do we want anything–but we’ve been maintaining a terraform repo for a while for running concourse dockerized behind a classic ELB. We’ve managed to work a lot of the issues out of it over the past 6 months and I think it’s ready to share.

PRs are most definitely welcome and hope this helps anyone out there struggling with an AWS deploy.

1 Like
#2

@jduv thanks for sharing.

#3

Why are you running it in a docker container when you’re spinning up the host yourself?

I could see using docker if you’re limited to k8s or something, but concourse itself already orchestrates containers, so why not run the binaries native on the host?

#4

@eedwards-sk Honestly for convenience (i.e. docker automagically restarting on failure, etc). I’ve not seen any need to run it on metal (i.e. optimizations) but if it were so required simply modify the user-data template to install concourse binaries. The command to run concourse would only need to be slightly modified.

Perhaps I should test that to see if it is, indeed, faster. We run our web nodes on t3.micro and our workers on t3.medium. No issues.

The major upgrade for us was swapping to ubuntu from AWS linux 2. For some reason it’s just absolutely better for concourse to run on. We had mysterious crashes on AWS linux.

1 Like
#5

As an update, our new version of this terraform installs on bare metal via systemd!

We’ve also ported to 5.1.0, so there will very likely be breaking changes to anyone using this module.

#6

Nice iteration!

Here’s an opportunity: bake an AMI first, so you have concourse at a known version already installed in the system, then configure it (set env vars / config files) on boot. You’ll get the benefits of version locking so you can promote them throughout environments, and your startup time will be faster.

#7

Not a bad idea. We’ll look into it.