nginx-devel Digest, Vol 33, Issue 20

Joy zhou yizhou136 at gmail.com
Sat Jul 21 13:35:26 UTC 2012


it well be locked in the case of  multiple threads.

2012/7/20 <nginx-devel-request at nginx.org>

> Send nginx-devel mailing list submissions to
>         nginx-devel at nginx.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://mailman.nginx.org/mailman/listinfo/nginx-devel
> or, via email, send a message with subject or body 'help' to
>         nginx-devel-request at nginx.org
>
> You can reach the person managing the list at
>         nginx-devel-owner at nginx.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of nginx-devel digest..."
>
>
> Today's Topics:
>
>    1. Re: various body filter questions (Maxim Dounin)
>    2. a little bug in ngx_event_expire_timers (Joy zhou)
>    3. Re: a little bug in ngx_event_expire_timers (Ruslan Ermilov)
>    4. [nginx] svn commit:  r4757 - trunk/src/http/modules (ru at nginx.com)
>    5. Re: a little bug in ngx_event_expire_timers (Pass86)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 19 Jul 2012 17:52:24 +0400
> From: Maxim Dounin <mdounin at mdounin.ru>
> To: nginx-devel at nginx.org
> Subject: Re: various body filter questions
> Message-ID: <20120719135224.GC31671 at mdounin.ru>
> Content-Type: text/plain; charset=us-ascii
>
> Hello!
>
> On Thu, Jul 19, 2012 at 10:48:59AM +0100, Robert Lemmen wrote:
>
> > hi everyone,
> >
> > I need to run an nginx with a custom body filter, for the sake of
> > argument let's say I want to replace "robert" with "bob" in every file
> > that ends with ".txt". I played around a bit and looked at the various
> > tutorials and documentation on the net. I have managed to get it
> > basically working, but am looking for your input to get it "correct":
> >
> > a) I create a module context and store it via ngx_http_set_ctx(). how do
> > I clean it up? or do I not need to?
>
> Usually you don't need to.  If you really want for some reason,
> you may use ngx_http_set_ctx() with NULL.
>
> > does everything allocated with
> > ngx_pcalloc(r->pool, ...) get freed when the request is sorted?
>
> Yes.  Everything allocated from a pool is freed on the pool
> destruction.
>
> > b) I would like to handle HEAD requests as well, but I can't just do it
> > with a filter. In the header filter I don't have the actual data in the
> > file, so I can't do the substitutions, so I don't know what the
> > content-length will be. what to do? I was thinking doing a GET
>
> Usual aproach is to just remove Content-Length header from an
> answer.  As even with GET you can't forecast response length
> before you've done processing full response, which in turn means
> your module will just don't work with anything but small
> responses.
>
> [...]
>
> > c) in the body filter, I want to read the buffer chain and modify it or
> > create a new one. but I don't get all the details of the buffer chain.
> > assuming "ngx_chain_t *in" as a parameter of the filter:
> > - if in->next != 0 (and so on), does that mean these buffers are parts
> >   of the actual file? so e.g. in->buf could hold the first half of teh
> >   file and in->next->buf the second half?
>
> Not file, response.  Otherwise yes, there may be more than one
> buffer in a chain passed to a body filter.  Moreover, it may be
> more than one call to a body filter.  You have to handle all
> buffers in the input chain(s) passed to your body filter.
>
> > - what are the semantics of the different fields in in->buf? what's the
> >   difference between pos/last and start/end? what do the different flags
> >   (temporary, memory, mmap, recycled, in_file, flush, sync) mean?
>
> The pos/last are bounds of memory used in a buffer, start/end -
> memory region allocated.  Various flags specify various buffer
> properties, try looking code for details.
>
> > - what's the difference between last_buf, last_in_chain? is it not the
> >   same as in->next == 0?
>
> The last_in_chain flag marks last buffer of a subrequest response,
> last_buf - last buffer in a main request response.  It's not the
> same as in->next == NULL as the latter means nothing, see above.
>
> > - it might be easier for me to read the buffers and create new ones,
> >   rather than changing the existing ones. do I need to do any sort of
> >   cleanup?
>
> Yes.  If you don't pass original buffer to a next filter and
> instead copy it's contents you have to mark buffer as sent by
> moving b->pos to b->last.  Failure to do so will result in hang
> on large responses as eventually underlying module will run out of
> buffers and will wait for previously passed buffers to be sent.
>
> > - are the buffers always complete? how would I know? I assume that if I
> >   download a 10GB file nginx doesn't just copy it into memory...
>
> Single buffer chain passed to a body filter doesn't necessary
> represent full response.
>
> Number and size of buffers used to copy files to memory (once one
> of a filters needs it) may be controlled with output_buffers
> directive, see http://nginx.org/r/output_buffers.
>
> > - how does sendfile work in nginx? if I do a body filter I of course
> >   need to make sure sendfile is never used.
>
> As long as buffers aren't with file-based content sendfile is not
> used.  You shouldn't care about this as long as you correctly
> modify buffer chain(s).
>
> Maxim Dounin
>
>
>
> ------------------------------
>
> Message: 2
> Date: Fri, 20 Jul 2012 12:11:51 +0800
> From: Joy zhou <yizhou136 at gmail.com>
> To: nginx-devel at nginx.org
> Subject: a little bug in ngx_event_expire_timers
> Message-ID:
>         <CAPw8Emfgs43CydaC6mafOgFHrRKD=
> DvqwPdBFQJY7KrUusvM3w at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> hi everyone:
>
>   I have found a little bug at method ngx_event_expire_timers  of  the file
> "src/event/ngx_event_timer.c "
>
>
>      84     for ( ;; ) {
>      85
>      86         ngx_mutex_lock(ngx_event_timer_mutex);
>      87
>      88         root = ngx_event_timer_rbtree.root;
>      89
>      90         if (root == sentinel) {
>      91             return;
>      92         }
>
>                [...]
>             }
>
>    The above code should add  "ngx_mutex_unlock(ngx_event_timer_mutex);" at
> 90 line,
>    Because the ngx_event_timer_mutex has locked at begin , but returned
> without unlock , is it right?
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120720/cbcae94d/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 3
> Date: Fri, 20 Jul 2012 10:07:26 +0400
> From: Ruslan Ermilov <ru at nginx.com>
> To: nginx-devel at nginx.org
> Subject: Re: a little bug in ngx_event_expire_timers
> Message-ID: <20120720060726.GA47268 at lo0.su>
> Content-Type: text/plain; charset=iso-8859-1
>
> On Fri, Jul 20, 2012 at 12:11:51PM +0800, Joy zhou wrote:
> >    hi everyone:
> >    ??
> >    ? I have found a little bug at method ngx_event_expire_timers? of? the
> >    file "src/event/ngx_event_timer.c "
> >    ??
> >    ??
> >    ???? 84???? for ( ;; ) {
> >    ???? 85
> >    ???? 86???????? ngx_mutex_lock(ngx_event_timer_mutex);
> >    ???? 87
> >    ???? 88???????? root = ngx_event_timer_rbtree.root;
> >    ???? 89
> >    ???? 90???????? if (root == sentinel) {
> >    ???? 91???????????? return;
> >    ???? 92???????? }
> >    ????
> >    ?????????????? [...]
> >    ??????????? }
> >    ????????????
> >    ?? The above code should add?
> >    "ngx_mutex_unlock(ngx_event_timer_mutex);" at 90 line,
> >    ?? Because the ngx_event_timer_mutex has locked at begin , but
> returned
> >    without unlock , is it right????????
>
> Only theoretically.  Practically, ngx_mutex_lock() is a no-op in nginx.
>
>
>
> ------------------------------
>
> Message: 4
> Date: Fri, 20 Jul 2012 08:22:00 +0000
> From: ru at nginx.com
> To: nginx-devel at nginx.org
> Subject: [nginx] svn commit:  r4757 - trunk/src/http/modules
> Message-ID: <20120720082200.1F2B33FA05C at mail.nginx.com>
> Content-Type: text/plain; charset=UTF-8
>
> Author: ru
> Date: 2012-07-20 08:21:59 +0000 (Fri, 20 Jul 2012)
> New Revision: 4757
> URL: http://trac.nginx.org/nginx/changeset/4757/nginx
>
> Log:
> Fixed debugging messages to account that limit_zone was renamed to
> limit_conn.
>
>
> Modified:
>    trunk/src/http/modules/ngx_http_limit_conn_module.c
>
> Modified: trunk/src/http/modules/ngx_http_limit_conn_module.c
> ===================================================================
> --- trunk/src/http/modules/ngx_http_limit_conn_module.c 2012-07-17
> 04:47:34 UTC (rev 4756)
> +++ trunk/src/http/modules/ngx_http_limit_conn_module.c 2012-07-20
> 08:21:59 UTC (rev 4757)
> @@ -238,7 +238,7 @@
>          }
>
>          ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
> -                       "limit zone: %08XD %d", node->key, lc->conn);
> +                       "limit conn: %08XD %d", node->key, lc->conn);
>
>          ngx_shmtx_unlock(&shpool->mutex);
>
> @@ -358,7 +358,7 @@
>      ngx_shmtx_lock(&shpool->mutex);
>
>      ngx_log_debug2(NGX_LOG_DEBUG_HTTP, lccln->shm_zone->shm.log, 0,
> -                   "limit zone cleanup: %08XD %d", node->key, lc->conn);
> +                   "limit conn cleanup: %08XD %d", node->key, lc->conn);
>
>      lc->conn--;
>
>
>
>
> ------------------------------
>
> Message: 5
> Date: Fri, 20 Jul 2012 18:35:00 +0800
> From: Pass86 <pass86 at gmail.com>
> To: "nginx-devel at nginx.org" <nginx-devel at nginx.org>
> Subject: Re: a little bug in ngx_event_expire_timers
> Message-ID: <4B94046F-69DC-4AC3-8068-866D1BCFFFCF at gmail.com>
> Content-Type: text/plain;       charset=GB2312
>
> not using lock, why coding?
>
> ???? iPhone
>
> ? 2012-7-20?14:07?Ruslan Ermilov <ru at nginx.com> ???
>
> > On Fri, Jul 20, 2012 at 12:11:51PM +0800, Joy zhou wrote:
> >>   hi everyone:
> >>
> >>     I have found a little bug at method ngx_event_expire_timers  of  the
> >>   file "src/event/ngx_event_timer.c "
> >>
> >>
> >>        84     for ( ;; ) {
> >>        85
> >>        86         ngx_mutex_lock(ngx_event_timer_mutex);
> >>        87
> >>        88         root = ngx_event_timer_rbtree.root;
> >>        89
> >>        90         if (root == sentinel) {
> >>        91             return;
> >>        92         }
> >>
> >>                  [...]
> >>               }
> >>
> >>      The above code should add
> >>   "ngx_mutex_unlock(ngx_event_timer_mutex);" at 90 line,
> >>      Because the ngx_event_timer_mutex has locked at begin , but
> returned
> >>   without unlock , is it right?
> >
> > Only theoretically.  Practically, ngx_mutex_lock() is a no-op in nginx.
> >
> > _______________________________________________
> > nginx-devel mailing list
> > nginx-devel at nginx.org
> > http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
>
>
> ------------------------------
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
> End of nginx-devel Digest, Vol 33, Issue 20
> *******************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120721/552b76c1/attachment.html>


More information about the nginx-devel mailing list