Nginx reload leading to ELB 502 on AWS Elastic Load Balancer

[gab]

# Issue Summary

* After executing Nginx soft reload with "service nginx reload", nginx is
able to close a lot of connections gracefully, but some connections aren't
closed gracefully and Nginx is sending an RST packet. For these connections,
Nginx didn't send FIN packet, and it didn't send "Connection: Close" header.
The connections are HTTP/1.1 keep alive connections.

# Expected Behaviour

* After executing Nginx soft reload, Nginx is gracefully closing all
connections by sending a "Connection: close" header in the response or a FIN
packet.

# Supporting Data

I have tcpdump of 5 such connections where nginx didn't close the connection
gracefully after nginx reload.
Here's the tcpdump -
https://drive.google.com/file/d/1UquhmJET9i8ShEizu8453iUKpprutILV/view?usp=sharing

If we analyse one such connection, we see that nginx didn't send FIN packet
on this connection, refer this image - https://i.imgur.com/zqyLOLc.png

If we see the response of second last request, we see nginx didn't send
"Connection: close" header either, refer this image -
https://i.imgur.com/P2uu722.png

In this image I have plotted FIN packets sent by nginx over time -
https://i.imgur.com/5lNAmnk.png

Nginx was reloaded on 2022-01-12 13:57:44 UTC.

FIN packet graph (https://i.imgur.com/5lNAmnk.png) shows that Nginx was able
to close a lot of connections gracefully at the time of reload, but it
wasn't able to close the 5 connections. TCP dump of which I've shared above
(https://drive.google.com/file/d/1UquhmJET9i8ShEizu8453iUKpprutILV/view?usp=sharing).

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,293375,293375#msg-293375