High number connections-writing stuck

[Marcelo MD]

Hi,

Just getting back to you. Sorry about the delay.

This behaviour was caused by a custom patch in a module. The patch was
ending the requests without finalizing them, leaking connections.

Eventually Nginx just exploded =) Nothing like production workload to
stress things out.

Thanks a lot.



On Sun, Dec 20, 2015 at 11:04 AM, Valentin V. Bartenev <vbart at nginx.com>
wrote:

> On Friday 18 December 2015 15:55:47 Marcelo MD wrote:
> > Hi,
> >
> > Recently we added a 'thread_pool' directive to our main configuration. A
> > few hours later we saw a huge increase in the connections_writing stat as
> > reported by stub_status module. This number reached +- 3800 and is stuck
> > there since. The server in question is operating normally, but this is
> very
> > strange.
> >
> > Any hints on what this could be?
> >
> >
> > Some info:
> >
> > - Here is a graph of the stats reported, for a server with thread_pool
> and
> > another without: http://imgur.com/a/lF2EL
> >
> > - I don`t have older data anymore, but the jump from <100 to +- 3800
> > connections_writing happened in two sharp jumps. The first one following
> a
> > reload;
> >
> > - The machines' hardware and software are identical except for the
> > thread_pool directive in their nginx.conf. They live in two different
> data
> > centers;
> >
> > - Both machines are performing normally. Nothing unusual in CPU or RAM
> > usage. Nginx performance is about the same.
> >
> > - Reloading Nginx with 'nginx -s reload' does nothing. Restarting the
> > process brings connections_writing down.
> [..]
>
> As I understand from your message everything works well.  You should also
> check the error_log, if it doesn't have anything suspicious then there is
> nothing to worry about.
>
> The increased number of writing connections can be explained by increased
> concurrency.  Now nginx processing cycle doesn't block on disk and can
> accept more connections at the same time.  All the connections that were
> waiting in listen socket before are waiting now in thread pool.
>
>   wbr, Valentin V. Bartenev
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>



-- 
Marcelo Mallmann Dias
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20160118/b5798af3/attachment.html>