<div dir="ltr">Thanks for the detailed response and taking the time to show some test examples.<div><br></div><div>We did think that perhaps it was buffering.<br></div><div>However, in our case, the "large" request is gigs in size, so there is no way that it is buffering that whole thing. I think our buffers are pretty small.</div><div>Unless there is some absolute black magic that will buffer what it can, close the upstream, then open it again to ask for more of that file. :)</div><div><br></div><div>I'm going to try what Peter suggested and check the netstat to at least confirm whether it is actually connected to the upstream still.<br></div><div>Worst case I'll then simulate the same on our end with the limit_rate.</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Dec 23, 2020 at 4:26 PM Maxim Dounin <<a href="mailto:mdounin@mdounin.ru">mdounin@mdounin.ru</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello!<br>
<br>
On Wed, Dec 23, 2020 at 12:36:16PM -0500, kenneth.s.brooks wrote:<br>
<br>
> Thanks for the response.<br>
> <br>
> I understand what you are saying about the worker processes. We have only a<br>
> single worker process.<br>
> <br>
> I have 2 upstream servers.<br>
> <br>
> To validate:<br>
> I am sending a request for a LARGE file. I see it hit server1. Server1 is<br>
> now serving that request for the next couple of minutes.<br>
<br>
Note that the fact that server1 is actually serving the request <br>
needs some additional verification. As a web accelerator nginx <br>
normally ensures that upstream servers are free to serve additional <br>
requests as soon as possible, so if the limiting factor is the <br>
client speed rather than connection between nginx and the upstream <br>
server, nginx will happily buffer the response and will serve it <br>
to the client itself. And there will be no active connections to <br>
server1, so least_conn will behave much like round-robin.<br>
<br>
> I send a request for a very tiny file. I see it hit server2. It finishes<br>
> (server1 is still processing request #1)<br>
> I send a request for a very tiny file. I see it hit server1 (even tho it is<br>
> still serving request #1 and server2 is not serving anything)<br>
> <br>
> I repeat that over and over, and I'll see all the subsequent requests being<br>
> routed to server1, then 2, then 1 then 2.<br>
> If I submit another LARGE file request, if the last request went to server2,<br>
> then now I have 2 LARGE file requests being served by server1.<br>
> <br>
> If I submit more requests, they all continue to be equally distributed to<br>
> server1 and server2 (even though server1 has 2 active things it is<br>
> serving).<br>
<br>
This behaviour corresponds to no active connections to server1, as <br>
might happen if the file is not large enough and instead buffered <br>
by nginx. <br>
<br>
> Is there some sort of a 'fudge factor' or threshold? That there has to be n<br>
> number of requests that one server is handling more than another server? I<br>
> wouldn't think so, but I'm at a loss.<br>
<br>
No, nothing like this.<br>
<br>
Just in case, here is a simple configuration which demonstrates <br>
how least_conn works (by using limit_rate to slow down responses <br>
of one of the upstream servers):<br>
<br>
upstream u {<br>
server <a href="http://127.0.0.1:8081" rel="noreferrer" target="_blank">127.0.0.1:8081</a>;<br>
server <a href="http://127.0.0.1:8082" rel="noreferrer" target="_blank">127.0.0.1:8082</a>;<br>
least_conn;<br>
}<br>
<br>
server {<br>
listen 8080;<br>
<br>
location / {<br>
proxy_pass <a href="http://u" rel="noreferrer" target="_blank">http://u</a>;<br>
}<br>
}<br>
<br>
server {<br>
listen 8081;<br>
limit_rate 10;<br>
return 200 "slow\n";<br>
}<br>
<br>
server {<br>
listen 8082;<br>
return 200 "fast\n";<br>
}<br>
<br>
And a test:<br>
<br>
$ for i in 1 2 3 4; do curl -q <a href="http://127.0.0.1:8080/" rel="noreferrer" target="_blank">http://127.0.0.1:8080/</a> & sleep 0.1; done; sleep 15<br>
fast<br>
fast<br>
fast<br>
slow<br>
[1] Done curl -q <a href="http://127.0.0.1:8080/" rel="noreferrer" target="_blank">http://127.0.0.1:8080/</a><br>
[2] Done curl -q <a href="http://127.0.0.1:8080/" rel="noreferrer" target="_blank">http://127.0.0.1:8080/</a><br>
[3] Done curl -q <a href="http://127.0.0.1:8080/" rel="noreferrer" target="_blank">http://127.0.0.1:8080/</a><br>
[4] Done curl -q <a href="http://127.0.0.1:8080/" rel="noreferrer" target="_blank">http://127.0.0.1:8080/</a><br>
<br>
Note that requests are started with some delay ("sleep 0.1") to <br>
make sure fast backend will be able to respond before the next <br>
request starts. Note that only one of the requests is routed to <br>
the slow backed - all other requests are routed to the fast <br>
one. That is, least_conn works as expected.<br>
<br>
-- <br>
Maxim Dounin<br>
<a href="http://mdounin.ru/" rel="noreferrer" target="_blank">http://mdounin.ru/</a><br>
_______________________________________________<br>
nginx mailing list<br>
<a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
<a href="http://mailman.nginx.org/mailman/listinfo/nginx" rel="noreferrer" target="_blank">http://mailman.nginx.org/mailman/listinfo/nginx</a><br>
</blockquote></div>