Windows Worker Is Very Slow to Transfer Volumes and Errors


#1

We’ve been using concourse for a couple years now with Linux workers with almost no problems. We’re just starting to spin up some Windows workers for .Net Framework projects we have, and are running into issues with incredibly slow volume transfers.

In our first windows test pipeline, we have a single project being tested using the Github PR resource. In the web interface, the actual test job just spins without starting for about 15 minutes before it times out with the error message:

Put /volumes/0201dbc8-5aea-475f-58b4-19513111269b/stream-in?path=.: net/http: timeout awaiting response headers

If you look on the windows worker, you can see the volume get created and files appear slowly over time

PS C:\Users\<me>> ls C:\hab\svc\concourse-windows-worker\var\volumes\live\9c220f5d-bdb6-423e-5b73-a6fb199718
b3\volume\


    Directory: C:\hab\svc\concourse-windows-worker\var\volumes\live\9c220f5d-bdb6-423e-5b73-a6fb199718b3\volume


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----         7/2/2018   5:29 PM                .git
d-----         7/2/2018   5:23 PM                ac
d-----         7/2/2018   5:21 PM                delivery
d-----         7/2/2018   5:21 PM                dev_env
d-----         7/2/2018   5:22 PM                mta
d-----         7/2/2018   5:21 PM                templates
-a----        6/30/2018   2:23 AM             69 .dockerignore
-a----        6/30/2018   2:23 AM           2519 .gitattributes
-a----        6/30/2018   2:23 AM           1039 .gitignore

The Windows host is on the exact same subnet as our Linux workers that operate without issue. Using Concourse 3.13.0. Windows worker is Server 2016 Core.


#2

As an experiment, I broke the project specific files into a stand alone repository, and concourse was able to transfer them to the windows host in about 30 seconds, so it seems this is related to the size of the repository.

For reference, we have tagged remote workers for jobs that need to run behind specific firewalls, and Concourse is able to move the same git resource/repository between linux hosts across a WAN in about ~30-60 seconds, depending on congestion.


#3

To gather an additional data point I would try to copy (with a command-line utility) the same repo (same size) from the ATC to the Windows worker, to discriminate if this is something specific to Concourse or if for any reason the Windows worker has network problems. If the copy is fast, then it looks like a Concourse bug.


#4

Whoa, been a while. Got distracted with other projects. This is no longer an issue with 4.2.1. I’m wondering if it was related to tar no longer being used for volume streaming on windows.