"no route to host" for services on node

Hi :slightly_smiling_face:

We have a bit of a weird problem with networking right now. We have a service running on a node that is also acting as one of our workers. When we try to access that service through it’s external IP address, we get a “no route to host” error.

A bit of context:

Used version: 5.8.0
Nodes:

  • node-01:
    • web
    • worker
    • service
  • node-02:
    • worker

Problem:

  • Accessing that service from the worker running on node-02 works.
  • Accessing that service from the worker running on node-01 does not work (“no route to host”)
  • DNS resolution works as the resolved IP for “service” is correct.

The workers are started with --garden-config /srv/concourse/etc/garden.ini and the garden.ini contains the following settings:

What we’ve tried so far:

[server]
max-containers = 1024
log-level = error
network-pool = 172.16.0.0/12
allow-host-access = true

At this point we are not sure, what else we can do but move the service somewhere else. Any help would be appreciated :slightly_smiling_face: