How can the nginx detect the stale event using 'instance'?

[Maxim Dounin]

Hello!

On Wed, Feb 22, 2012 at 08:59:18AM +0900, 김수영 wrote:

> Dear Maxim,
> 
> Thanks for your comments.
> 
> Could you please explain it more details?
> 
> I have only seen the epoll module code not kqueue module. The routine is similar to kqueue.
> 
> -------------
> (ngx_epoll_add_event)
>     ee.data.ptr = (void *) ((uintptr_t) c | ev->instance);
> 
> (ngx_epoll_process_events)
>     instance = (uintptr_t) c & 1;
>     if (c->fd == -1 || rev->instance != instance) {...}
> -------------

The same here: instance bit is stored in user data passed to 
kernel, and when kernel report event the instance bit returned by 
kernel is checked against one in the event structure.

> 
> I'm wondering who set the instance bit.. I just found in ngx_event_process_init() and ngx_get_connection().
> 
> -------------
> (ngx_event_process_init)
>     rev = cycle->read_events;
>     for (i = 0; i < cycle->connection_n; i++) {
>         rev[i].closed = 1;
>         rev[i].instance = 1;
> #if (NGX_THREADS)
>         rev[i].lock = &c[i].lock;
>         rev[i].own_lock = &c[i].lock;
> #endif
>     }

This will initalize instance bit.

> 
> (ngx_get_connection)
>     instance = rev->instance;
>     ngx_memzero(rev, sizeof(ngx_event_t));
>     ngx_memzero(wev, sizeof(ngx_event_t));
>     rev->instance = !instance;
>     wev->instance = !instance;

And this will reverse existing bit when new connection is 
created. 

> -------------
> 
> BUT, I guess that ngx_event_process_init() may be called once 
> 
> and ngx_get_connection() may be called when a new connection is accepted(I'm not sure-_-;).
> 
> When ngx_event_process_init() is called, instance bit is 1
> 
> and then when a new connection is accepted ngx_get_connection(), 
> instance bit is 0. (next ngx_get_connection() will set the 
> instance bit to 1).

The ngx_get_connection() will always reverse the bit, i.e. on 
first use of a connection structure it will be set to 0, on second 
connection structure use it will be set to 1, then again to 0 and 
so on.  On each use of the connection structure instance bit will 
be different from one used during previous use of the same 
connection structure (for another connection).

> And then what happen? I can't trace how the nginx handle it, anymore..-_-;

And then (even if connection isn't closed) if the instance bit 
returned by kernel doesn't match one stored in the event structure - 
we may safely drop this event as we are sure the connection 
structure was reused for another connection.

Maxim Dounin

> 
> Please let me know next scenario..^^
> 
> -----Original Message-----
> From: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Maxim Dounin
> Sent: Tuesday, February 21, 2012 10:28 PM
> To: nginx-devel at nginx.org
> Subject: Re: How can the nginx detect the stale event using 'instance'?
> 
> Hello!
> 
> On Tue, Feb 21, 2012 at 06:51:19PM +0900, 김수영 wrote:
> 
> > These days, I am studying the nginx source code..
> > 
> > I think this code is very beautiful, and fully modulized.. BUT so it 
> > is difficult to follow the code.-_-;
> > 
> > BTW, In struct ngx_event_s, there is ‘unsigned instance:1;’..
> > 
> > I can’t understand how can the nginx detect the stale events using 
> > it.-_-;
> > 
> > Could you please explain the detection mechanism or give me a hint?
> 
> The "instance" bit is passed in a event user data to the kernel and when the kernel returns the event it's checked against one recorded in the event structure.  If there is no match - this means the event reported by the kernel is stale (i.e. we've already closed the descriptor in the very same event loop iteration).
> 
> See e.g. src/event/modules/ngx_kqueue_module.c for details, in particular ngx_kqueue_set_event() and ngx_kqueue_process_events().
> 
> Maxim Dounin
> 
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel