Task #16423
closedIPv6 connectivity issue preventing GitLab LRZ webhooks from reaching Hephaestus servers
0%
Description
Summary: IPv6 connectivity issue preventing GitLab LRZ webhooks from reaching Hephaestus servers
Priority: High
Description:
I'm trying to configure GitLab webhooks from gitlab.lrz.de to send events to our Hephaestus application. The webhooks are failing with the following error shown in GitLab:
{quote}Hook execution failed. Failed to open TCP connection to 2a09:80c0:89::80:443
(No route to host - connect(2) for "2a09:80c0:89::80" port 443)
{quote}
Problem Analysis (disclaimer: analysis performed with assistance from Claude Opus 4.5)
Our domains resolve to both IPv4 and IPv6 addresses:
$ dig +short staging.hephaestus.aet.cit.tum.de A
131.159.89.80
$ dig +short staging.hephaestus.aet.cit.tum.de AAAA
2a09:80c0:89::80
When GitLab's servers resolve our domain, they receive both A and AAAA records. Per standard behavior (RFC 6724), they prefer IPv6 and attempt to connect to 2a09:80c0:89::80:443. This fails because there appears to be no IPv6 route between LRZ's network and our servers.
IPv4 connectivity works correctly - I verified that {{curl -4 [https://staging.hephaestus.aet.cit.tum.de/webhooks/health]}} returns {{{}200 OK{}}}.
Affected domains:
- staging.hephaestus.aet.cit.tum.de (AAAA: 2a09:80c0:89::80)
- hephaestus.aet.cit.tum.de (AAAA: 2a09:80c0:89::7)
Impact:
I cannot receive GitLab webhooks from LRZ, which blocks our integration with LRZ-hosted GitLab projects.
Request:
I would appreciate your advice on the best solution. Some options Opus and me considered:
Remove the AAAA records for these domains, forcing all clients to use IPv4¶
Fix IPv6 routing between LRZ and our subnet (2a09:80c0:89::/48) if this is feasible¶
Alternative approach we haven't considered?¶
I don't have strong requirements for IPv6 on these domains, so removing the AAAA records would be acceptable if that's the simplest solution. However, I wanted to check with you first in case there are implications I'm not aware of, or if fixing the routing is straightforward.
Thank you for your help!
ML Updated by Matthias Linhuber about 2 months ago
For some reason your VMs lost their DHCP IPv6 address. A sudo netplan apply reconfigured the interface and autoresolved the IPv6 address