How to get real-time stdout?

If I run a long-running process in a Concourse job, such as a terraform apply that builds a bunch of cloud stuff, it might take an hour to run and build all the stuff I need it to build, which is fine. And when I run the same thing outside of the context of Concourse, I can see what my process is doing because it outputs to stdout as it does what it needs to do.

But when I run it in Concourse, I don’t get any stdout (in the web UI of the pipeline job, or via fly watch). Then when the process finishes, it dumps all of the stdout from the process to the web UI all at once.

Is there any recommended way to get real-time stdout in the web UI while the process is running?

Concourse does print everything from stdout and stderr to the UI/fly watch in real-time by default. I haven’t seen what you’re describing before. It sounds like output is getting buffered for some reason.

Do you see this buffering on every task or only on really long running or verbose ones?

Thanks for your reply.

I’m now thinking that this is probably just a side-effect of how I’m launching my process (ruby’s Open3.popen3). I’ll get that worked out and report back I have any evidence that Concourse is doing this, not my scripts. But until then, it’s probably just my scripts.