Nginx request processing is slow when logging disabled

[Maxim Dounin]

Hello!

On Thu, Jul 04, 2019 at 01:07:04PM +0530, Om Prakash Shanmugam wrote:

> Hello All,
> 
> I have an Nginx reverse proxy connected to uWSGI backend. I have configured
> nginx to log to a centralized remote syslog service as:
> 
> *error_log syslog:server=example.com:514
> <http://example.com:514>,tag=nginx_error debug;*
> 
> The problem here is that, when I remove the above line from my nginx.conf,
> the request processing time is becoming very high and It leads to client
> timeout ( returns HTTP 460 ).
> 
> When I enable logging in my nginx.conf, I do not get HTTP 460 at all. But,
> there's an extra overhead introduced which Increases the CPU Utilization.
> What I suspect is that the nginx is sending the HTTP requests to my uWSGI
> backend little slowly and my uWSGI backend is able to handle them
> gracefully, write the response back to nginx successfully. The average
> response time of the backend also spikes to 5x when logging is enabled.
> 
> Once I disable logging, the CPU Utilization decreases while the requests
> are flooded to uWSGI backend and the backend takes time to return the
> response within the defined client timeout period. If the request takes
> time to process and if the client ( Android/iOS app) hasn't received the
> response, It aborts the connection either when the timeout is reached or if
> the user cancels the request.
> 
> I'd like to know whether I have to add a proxy buffer to my nginx to queue
> up requests and send it to my backend instead of overflooding it. Or any
> other solutions are appreciated.

So, from your description it looks like the problem is that your 
backend can't cope with load.

You may want to configure your backend to better handle many 
requests - in particular, consider configuring a reasonable number 
of worker processes / threads, and make sure this number is low 
enough for your server to handle.  To control number of requests 
queued to your backend, consider tuning your backend's listening 
queue.

Alternatively, you may want to add more backends to handle the 
load.

-- 
Maxim Dounin
http://mdounin.ru/