[Solved] IPv6 connectivity issue to `nginx.org` due to tunnel MTU

[Alexander Huynh]

Hello,

I think this is a repeat of "packages.nginx.org IPv6 SSL is broken" last month [0]. I solved the problem for myself, so I'm posting here in hopes it can help others down the line.

I was trying to connect to `https://nginx.org/`, and I found the website hung before I was impatient enough to give up browsing to it [3].

I did a bit of digging and found that it occurs only on IPv6. My setup is similar to Sergio's:

  * no native IPv6, but rather via an HE.net tunnel on tunnelbroker.net
  * tunnel MTU set to 1480 on tunnelbroker.net

Additionally, I have this relevant setting on my EdgeRouter, matching the remote end on HE:

  * # show interfaces tunnel tun0 mtu ⟶ mtu 1480

After getting primed on MTU and MSS, I found some documentation [1] on how to clamp MSS on my router. I tried and nothing changed, so I fiddled around on my local machine and found that locally setting the MTU to 1480 worked, via macOS's `networksetup` commands:

  * original local MTU is 1500 according to: networksetup -getMTU en0
  * [3] suddenly worked after executing: networksetup -setMTU en0 1480
  * [3] stopped working after executing: networksetup -setMTU en0 1481

More searching yielded that there are two MSS clamping options on my EdgeRouter: one for IPv4 and one for IPv6 [2]. Deleting the erroneous v4 commands and adding the following TCP MSS clamping options directed the router to rewrite the MSS field, allowing the subsequent smaller packets to fit within my tunnel:

  * set firewall options mss-clamp6 mss 1420
  * set firewall options mss-clamp6 interface-type all

The 1420 is calculated as: the tunnel MTU of 1480 bytes, minus 40 bytes for the IPv6 header, minus 20 bytes for the IP header.

Things started working again from there, though with my local interface MTU incorrectly set to 1500, I've only solved IPv6 TCP.

Anywho, I do have to thank the nginx website for strictly adhering to standards. If it weren't for trying to maximize MTU, this problem wouldn't have been exposed on my end. Hopefully this information helps someone further down the line, if they run into a similar issue.

Thanks for reading!
-- 
Alex

[0] http://mailman.nginx.org/pipermail/nginx/2020-September/059964.html
[1] https://community.ui.com/questions/EdgeRouter-and-MTU-Setting/54051cd0-38fc-4de9-a499-32af37a851b3
[2] https://docs.vyos.io/en/latest/routing/mss-clamp.html
[3] % date; curl -svJLo /dev/null --connect-to '::[2a05:d014:edb:5704::6]:443' 'https://nginx.org'; date
Tue Oct 20 20:03:22 EDT 2020
* Connecting to hostname: 2a05:d014:edb:5704::6
* Connecting to port: 443
*   Trying 2a05:d014:edb:5704::6...
* TCP_NODELAY set
* Connected to 2a05:d014:edb:5704::6 (2a05:d014:edb:5704::6) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [223 bytes data]
### hangs here
* LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to nginx.org:443 
* Closing connection 0
Tue Oct 20 20:06:24 EDT 2020