Configuration tuning for best performance with upstream nodes down
mdounin at mdounin.ru
Wed Apr 9 17:30:59 MSD 2008
On Wed, Apr 09, 2008 at 09:58:06AM +0200, denis wrote:
>I've had some experience with Nginx working as a reverse proxy
>loadbalancer only for multiple apache backends. For most of the time
>performance was excellent.
>However during a few situations where backends would go down and stay
>down for a longer period of time it would seem Nginx was not behaving
>ideally. It would give lots of error messages about upstream timeouts
>(expected), but it would take a little long time before switching to the
>next upstream (imho anything that is noticeable by the user is a bit too
>Is a bit confusing to me; fail_timeout and max_fails seem to be the
>numbers to work with, but as it appears, tuning fail_timeout down would
>also mean that downed backends would be tried again faster?!
Yes. And to make things even more clear: fail_timeout has nothing
to do with upstream response time, it's just a time frame for
nginx to remember fails.
It looks like you really want to tune how fails are detected.
Take a look at ngx_http_proxy_module configuration, notably
proxy_connect_timeout and proxy_read_timeout.
Note: proxy_connect_timeout defaults to 60s - its way too high for
backends under you control. I recommend tuning this to something
Note: proxy_read_timeout defaults to 60s too, and depending on you
task this may be either too high or too small.
There is also proxy_send_timeout, but normally it doesn't
matter since most requests are small and will fit into socket
buffers (so nginx can't distingush between send and read problems).
>Let us say the scenario has backends that normally respond very quickly
>(<2secs), how would one best tune the appropriate parameters?
>An alternate way of doing this which I actually considered at one point,
>would be to automate checks (better checks than pure network connections
>in this case) and then removing the nodes from the pools in config, and
>forcing an nginx config refresh..
>Hope someone can give examples of config that minimizes worsening the
>user experience in fail situations, or otherwise enlighten me ;)
More information about the nginx