Container collection happens based on this criteria
In my Concourse’s case, all of the required fields for GC are null except for worker_resource_config_check_session_id
According to these docs, the worker_resource_config_check_sessions should go away when either its corresponding resource_config_check_sessions record goes away or it’s corresponding worker_base_resource_types record goes away. I can confirm that resource_config_check_sessions is going away and containers are still sticking around by using the following query.
select * from worker_resource_config_check_sessions ws where ws.resource_config_check_session_id not in ( select rs.id from resource_config_check_sessions rs )
The amount of these steadily increases until the workers no longer have available containers. I change the ‘select *’ to a ‘delete’ and run that delete query periodically to keep the pipelines running. I have a hard time believing I found such an obvious bug so it has to be something on my end. I have manually set gc_interval to 30s on atc. Previously it was not set, and should have been using the default that is also 30s.
Let me know what other information I can provide.