<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">Sébastien,</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Nginx should keep active connections
open and wait for a request to complete before closing.<br>
</div>
<div class="moz-cite-prefix">A reload starts a new set of workers
while old workers wait for old connections to shut down.</div>
<div class="moz-cite-prefix">The only exception I'm aware of is
having worker_shutdown_timeout configured: in this case a worker
will wait till this timeout and forcibly close a connection. Be
default there is no timeout.<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">It would be curious to see error log of
nginx at IP_1 (the reloaded one) while the reload happens. It may
explain the reason for connection resets.<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Kind regards,</div>
<div class="moz-cite-prefix">Igor.<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">On 25/03/2024 12:31, Sébastien Rebecchi
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CA+ts6nrhzgVDbufP9ywNQxNcvOQzJMZmsaeLn4KObLO+V1sK8Q@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div style="padding:0px 0px 0px 2px">
<div
style="color:rgb(0,0,0);font-family:monospace;font-size:10pt">
<p style="margin:0px">Hello</p>
<p style="margin:0px"><br>
</p>
<p style="margin:0px">I have an issue with <span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">nginx</span>
closing prematurely connections when reload is performed.</p>
<p style="margin:0px"><br>
</p>
<p style="margin:0px">I have some <span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">nginx</span>
servers configured to proxy_pass requests to an upstream
group. This group itself is composed of several servers
which are <span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">nginx</span>
themselves, and is configured to use <span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">keepalive</span>
connections.</p>
<p style="margin:0px">When I trigger a reload (-s reload) on
an <span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">nginx</span>
of one of the servers which is target of the upstream, I
see in error logs of all servers in front that connection
was reset by the <span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">nginx</span>
which was reloaded.</p>
<p style="margin:0px"><br>
</p>
<p style="margin:0px">Here configuration of upstream group
(IPs are hidden replaced by IP_X):</p>
<p style="margin:0px">--- BEGIN ---</p>
<p style="margin:0px">upstream data_api {</p>
<p style="margin:0px"> random;</p>
<p style="margin:0px"><br>
</p>
<p style="margin:0px"> server IP_1:80 max_fails=3
fail_timeout=30s;</p>
<p style="margin:0px"> server IP_2:80 max_fails=3
fail_timeout=30s;</p>
<p style="margin:0px"> server IP_3:80 max_fails=3
fail_timeout=30s;</p>
<p style="margin:0px"> server IP_4:80 max_fails=3
fail_timeout=30s;</p>
<p style="margin:0px"> server IP_5:80 max_fails=3
fail_timeout=30s;</p>
<p style="margin:0px"> server IP_6:80 max_fails=3
fail_timeout=30s;</p>
<p style="margin:0px"> server IP_7:80 max_fails=3
fail_timeout=30s;</p>
<p style="margin:0px"> server IP_8:80 max_fails=3
fail_timeout=30s;</p>
<p style="margin:0px"> server IP_9:80 max_fails=3
fail_timeout=30s;</p>
<p style="margin:0px"> server IP_10:80 max_fails=3
fail_timeout=30s;</p>
<p style="margin:0px"><br>
</p>
<p style="margin:0px"> <span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">keepalive</span>
20;</p>
<p style="margin:0px">}</p>
<p style="margin:0px">--- END ---</p>
<p style="margin:0px"><br>
</p>
<p style="margin:0px">Here configuration of the location
using this upstream:</p>
<p style="margin:0px">--- BEGIN ---</p>
<p style="margin:0px">location / {</p>
<p style="margin:0px"> proxy_pass <a href="http://data_api"
moz-do-not-send="true" class="moz-txt-link-freetext">http://data_api</a>;</p>
<p style="margin:0px"><br>
</p>
<p style="margin:0px"> proxy_http_version 1.1;</p>
<p style="margin:0px"> proxy_set_header Connection "";</p>
<p style="margin:0px"><br>
</p>
<p style="margin:0px"> proxy_set_header Host $host;</p>
<p style="margin:0px"> proxy_set_header X-Real-IP $real_ip;</p>
<p style="margin:0px"> proxy_set_header X-Forwarded-For
$proxy_add_x_forwarded_for;</p>
<p style="margin:0px"><br>
</p>
<p style="margin:0px"> proxy_connect_timeout 2s;</p>
<p style="margin:0px"> proxy_send_timeout 6s;</p>
<p style="margin:0px"> proxy_read_timeout 10s;</p>
<p style="margin:0px"><br>
</p>
<p style="margin:0px"> proxy_next_upstream error timeout <span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">http</span>_502
<span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">http</span>_504;</p>
<p style="margin:0px">}</p>
<p style="margin:0px">--- END ---</p>
<p style="margin:0px"><br>
</p>
<p style="margin:0px">And here the kind of error messages I
get when I reload <span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">nginx</span>
of "IP_1":</p>
<p style="margin:0px">--- BEGIN ---</p>
<p style="margin:0px">2024/03/25 11:24:25 [error] 3758170#0:
*1795895162 <span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">recv</span>()
failed (104: Connection reset by peer) while reading
response header from upstream, client: CLIENT_IP_HIDDEN,
server: SERVER_HIDDEN, request: "POST
/REQUEST_LOCATION_HIDDEN HTTP/2.0", upstream: "<a
href="http://IP_1:80/REQUEST_LOCATION_HIDDEN"
moz-do-not-send="true">http://IP_1:80/REQUEST_LOCATION_HIDDEN</a>",
host: "HOST_HIDDEN", referrer: "REFERRER_HIDDEN"</p>
<p style="margin:0px">--- END ---</p>
<p style="margin:0px"><br>
</p>
<p style="margin:0px">I thought -s reload was doing graceful
shutdown of connections. Is it due to the fact that <span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">nginx</span>
can not handle that when using <span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">keepalive</span>
connections? Is it a bug?</p>
<p style="margin:0px">I am using <span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">nginx</span>
1.24.0 everywhere, no particular </p>
<p style="margin:0px"><br>
</p>
<p style="margin:0px">Thank you for any help.</p>
<p style="margin:0px"><br>
</p>
<p style="margin:0px"><span
style="text-decoration-line:underline;text-decoration-style:wavy;text-decoration-color:rgb(255,128,64)">Sébastien</span></p>
</div>
</div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
nginx mailing list
<a class="moz-txt-link-abbreviated" href="mailto:nginx@nginx.org">nginx@nginx.org</a>
<a class="moz-txt-link-freetext" href="https://mailman.nginx.org/mailman/listinfo/nginx">https://mailman.nginx.org/mailman/listinfo/nginx</a>
</pre>
</blockquote>
<p><br>
</p>
</body>
</html>