mail proxy auth_http stats

[Maxim Dounin]

Hello!

On Tue, Oct 26, 2010 at 10:34:31PM +0530, Naresh V wrote:

> Hi,
> 
> I have a mail proxy configured alongside the following:
> 
> http {
>         perl_modules /etc/nginx/perl/lib;
>         perl_require mailauth.pm;   // there's a Perl::DBI call here
> 
>         server {
>                 listen 81;
>                 location /auth {
>                          perl mailauth::handler;
>                 }
>                 location /stats {
>                          stub_status on;
>                          access_log off;
>                 }
> }
> 
> and my mail proxy uses this server :81/auth for auth_http
> 
> when i curl :81/stats, I always get only 1 in writing:
> 
> 
> Active connections: 149
> server accepts handled requests
>  4945 4945 2465
> Reading: 0 Writing: 1 Waiting: 148
> 
> I initially thought that this was due to some sort of a limitation of
> using the embedded-perl interpretor of nginx and so i removed this
> logic out and served it out of a different nginx+thin+rackup script
> (an ActiveRecord call) which implemented the same redirection logic i
> had wired in earlier in mailauth.pm.
> 
> 
> So now I have one instance of nginx on server A acting solely as a
> mail proxy and another on server B serving as auth_http server.
> 
> This time, stats from server B was like:
> 
> Active connections: 1
> server accepts handled requests
>    1985 1985 1985
> Reading: 0 Writing: 1 Waiting: 0
> 
> 
> I don't follow why "writing" is always 1.
> I've even setup multiple mail proxy nginx instances to use the same
> server B's :81/auth as auth_http URI and yet I have writing = 1.
> 
> I've seen other nginx+tomcat setups where stats are like:
> 
> Active connections: 39
> server accepts handled requests
>  421439 421439 986972
> Reading: 0 Writing: 28 Waiting: 11
> 
> (writing > 1)
> 
> 
> Can someone explain why this is the case?

1. Mail connections are only counted in active stats (and waiting, 
as it's calculated as "active - (reading + writing)").  Only 
connections which may contribute to "writing" is http connections.  

2. Your request to stub_status will be in "writing" state when 
response is generated.  That's why "writing" is always at least 1. 

3. With embedded perl (as well as stub_status) and small responses 
you have no chance to have two requests in "writing" state in the 
same worker, as requests are processed one-by-one.  As soon as 
nginx got full request it increments "writing", calls perl to 
generate response, write()'s result to socket, decremets 
"writing".  There are no points where nginx may switch to another 
request between incrementing and decrementing writing as it's not 
returns to event loops while processing request.

With many workers there are chances to see several requests in 
"writing" state (up to 1 per worker) but this requires high 
concurency which is unlikely to happen in your case (as auth_http 
requests usually happen only once per mail connection).

For http proxy/fastcgi/... setups (or just while serving big 
static responses, even without aio) situation is a bit different: 
nginx increments "writing", starts handling request and at some 
point it gets EAGAIN (e.g. waiting for connection to backend).  
Then it goes to event loop and starts processing other requests 
(if any).  This way you may see multiple request in "writing" 
state even in single worker.  Though this still requres at least 
one active request to be present in addtional to your status 
request which is unlikely to happen in your case (as auth_http... 
see above).

Maxim Dounin