<div dir="ltr"><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">Hello,<br><br></div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">Reading from <a href="https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream">upstream</a> docs, on upstream pool exhaustion, every backend should be tried once, and then if all fail the response should be crafted based on the one from the last server attempt.<br></div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">So far so good.<br><br></div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">I recently faced a server farm which implements a dull nightly restart of every node, not sequencing it, resulting in the possibility of having all nodes offline at the same time.<br><br></div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">However, I collected log entries which did not match what I was expected.<br></div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">For 6 backend nodes, I got:<br></div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">- log format: <span style="font-family:monospace,monospace">$status $body_bytes_sent $request_time $upstream_addr $upstream_response_time</span><br></div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">- log entry: <span style="font-family:monospace,monospace">502 568 0.001 <IP address 1>:<port>, <IP address 2>:<port>, <IP address 3>:<port>, <IP address 4>:<port>, <IP address 5>:<port>, <IP address 6>:<port>, php-fpm 0.000, 0.000, 0.000, 0.000, 0.001, 0.000, 0.000</span><br></div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">I got 7 entries for <span style="font-family:monospace,monospace">$upstream_addr</span> & <span style="font-family:monospace,monospace">$upstream_response_time</span>, instead of the expected 6.<br></div><br><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default">Here are the interesting parts of the configuration:<br></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace">upstream php-fpm {<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> server <machine 1>:<port> down;<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> server <machine 2>:<port> down;<br> [...]<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> server <machine N-5>:<port>;<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> server <machine N-4>:<port>;<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> server <machine N-3>:<port>;<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> server <machine N-2>:<port>;<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> server <machine N-1>:<port>;<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> server <machine N>:<port>;<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> keepalive 128;<br></span></div><span style="font-family:monospace,monospace">}<br><br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace">server {<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> set $fpm_pool "php-fpm$fpm_pool_ID";<br> [...]<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> location ~ \.php$ {<br> [...]<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> fastcgi_read_timeout 600;<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> fastcgi_keep_conn on;<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> fastcgi_index index.php;<br><br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> include fastcgi_params;<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;<br> [...]<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> fastcgi_pass $fpm_pool;<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace"> }<br></span></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace">}</span><br></div><br><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default">The question is:<br></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default"><span style="font-family:monospace,monospace">php-fpm</span> being an upstream group name, how come has it been tried as a domain name in the end?<br>Stated otherwise, is this because the upstream group is considered 'down', thus somehow removed from the possibilities, and nginx trying one last time the name as a domain name to see if something answers?<br></div><div style="font-size:small;color:rgb(51,51,153)" class="gmail_default">This 7th request is definitely strange to my point of view. Is it a bug or a feature?<br></div><div><div class="gmail_signature"><font size="1"><span style="color:rgb(102,102,102)">---<br></span><b><span style="color:rgb(102,102,102)">B. R.</span></b><span style="color:rgb(102,102,102)"></span></font></div></div>
</div>