From nick at webthing.com  Sat Dec  1 00:19:36 2012
From: nick at webthing.com (Nick Kew)
Date: Sat, 1 Dec 2012 00:19:36 +0000
Subject: Connection API?
Message-ID: <20121201001936.283a7d4a@baldur>

I'm looking to port a module to nginx.  The module currently
exists for Apache Trafficserver and Apache HTTPD.

Part of the module hooks into the start and end of the
TCP connection, and maintains connection data in a connection
context.  As far as I can tell, there's no equivalent API in
nginx: there's no ctx to store context in ngx_connection_t,
nor are there hooks to insert my modules functions at the start
and end of a connection.

(a) Is there really no connection API, or am I just missing it?
(b) If there really is no such API, would the nginx core team
    be receptive to a patch introducing it?  I would envisage
    a ctx in the connection struct, and a hookable
    start-connection event.

-- 
Nick Kew


From kyprizel at gmail.com  Sat Dec  1 00:27:08 2012
From: kyprizel at gmail.com (kyprizel)
Date: Sat, 1 Dec 2012 04:27:08 +0400
Subject: Connection API?
In-Reply-To: <20121201001936.283a7d4a@baldur>
References: <20121201001936.283a7d4a@baldur>
Message-ID: <CAAk1GzLNUGmXujgGbfD3K9+0Wo_uxQ5z-fx8XzS7+yRh2Voh3w@mail.gmail.com>

Hi, Nick
nice to see that you're interested in porting Ironbee to NGINX :)

On Sat, Dec 1, 2012 at 4:19 AM, Nick Kew <nick at webthing.com> wrote:
> I'm looking to port a module to nginx.  The module currently
> exists for Apache Trafficserver and Apache HTTPD.
>
> Part of the module hooks into the start and end of the
> TCP connection, and maintains connection data in a connection
> context.  As far as I can tell, there's no equivalent API in
> nginx: there's no ctx to store context in ngx_connection_t,
> nor are there hooks to insert my modules functions at the start
> and end of a connection.
>
> (a) Is there really no connection API, or am I just missing it?
> (b) If there really is no such API, would the nginx core team
>     be receptive to a patch introducing it?  I would envisage
>     a ctx in the connection struct, and a hookable
>     start-connection event.
>
> --
> Nick Kew
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel


From lawrence.pit at gmail.com  Sat Dec  1 11:33:48 2012
From: lawrence.pit at gmail.com (Lawrence Pit)
Date: Sat, 01 Dec 2012 22:33:48 +1100
Subject: [PATCH] Feature: $msec and $start_msec http variables,
 useful for setting in proxy headers before sending request to backend
 servers so processing and queue times can be measured.
Message-ID: <63A1809C-D39E-4761-95E3-FBCA3180F854@gmail.com>

Hello,

Below is a patch based on my own requirements to monitor request and 
queue times within nginx. It's based off stable-1.2 (currently 1.2.5)

$start_msec is the time in milliseconds resolution when the first bytes 
were read from the client.

$msec is the current time in milliseconds resolution.

I use it like this:

server {
   proxy_set_header X-Request-Start $start_msec;
   proxy_set_header X-Queue-Start $msec;
   ?
}

I chose the name "msec" as the counterpart of the variable "msec" as 
used by the log formatter.

After I finished this patch I checked the 1.3 branch, and noticed it 
already has an $msec http variable defined. But it's output is with a 
dot character before the milliseconds part. Why is that? I can 
understand why it's formatted like that in a log, but not for use in an 
http variable when it's probably going to be used for calculations.

New Relic has decribed something similar:
https://newrelic.com/docs/features/tracking-front-end-time

Aside: note that New Relic requires the time in microseconds. The only 
way I could find to do this (with my patch) was a workaround using the 
set command. With the 1.3 branch version of $msec there's no way I think 
(or is there?).

server {
   set $usec 000;
   proxy_set_header X-Request-Start 't=$start_msec$usec';
   proxy_set_header X-Queue-Start 't=$msec$usec';
   ?
}

It would be nice if proxy_set_header handles strings and interpolation 
in the same way that log_format works, e.g. :

   proxy_set_header X-Something 't=$msec' '000';

But that's for another patch another day. ;p


Cheers,
Lawrence



---
  src/http/ngx_http_variables.c | 60 
+++++++++++++++++++++++++++++++++++++++++++
  1 file changed, 60 insertions(+)

diff --git a/src/http/ngx_http_variables.c 
b/src/http/ngx_http_variables.c
index b0949c7..9cb1330 100644
--- a/src/http/ngx_http_variables.c
+++ b/src/http/ngx_http_variables.c
@@ -108,6 +108,10 @@ static ngx_int_t 
ngx_http_variable_hostname(ngx_http_request_t *r,
      ngx_http_variable_value_t *v, uintptr_t data);
  static ngx_int_t ngx_http_variable_pid(ngx_http_request_t *r,
      ngx_http_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_http_variable_msec(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_http_variable_start_msec(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data);

  /*
   * TODO:
@@ -285,6 +289,12 @@ static ngx_http_variable_t  
ngx_http_core_variables[] = {
      { ngx_string("pid"), NULL, ngx_http_variable_pid,
        0, 0, 0 },

+    { ngx_string("msec"), NULL, ngx_http_variable_msec,
+      0, 0, 0 },
+
+    { ngx_string("start_msec"), NULL, ngx_http_variable_start_msec,
+      0, 0, 0 },
+
  #if (NGX_HAVE_TCP_INFO)
      { ngx_string("tcpinfo_rtt"), NULL, ngx_http_variable_tcpinfo,
        0, NGX_HTTP_VAR_NOCACHEABLE, 0 },
@@ -1915,6 +1925,56 @@ ngx_http_variable_pid(ngx_http_request_t *r,
  }


+static ngx_int_t
+ngx_http_variable_msec(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data)
+{
+    u_char           *p;
+    ngx_time_t       *tp;
+    ngx_msec_int_t   ms;
+
+    p = ngx_pnalloc(r->pool, NGX_INT64_LEN);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
+
+    tp = ngx_timeofday();
+    ms = (ngx_msec_int_t) (tp->sec * 1000 + tp->msec);
+
+    v->len = ngx_sprintf(p, "%L", ms) - p;
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+    v->data = p;
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_http_variable_start_msec(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data)
+{
+    u_char           *p;
+    ngx_msec_int_t   ms;
+
+    p = ngx_pnalloc(r->pool, NGX_INT64_LEN);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
+
+    ms = (ngx_msec_int_t) (r->start_sec * 1000 + r->start_msec);
+
+    v->len = ngx_sprintf(p, "%L", ms) - p;
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+    v->data = p;
+
+    return NGX_OK;
+}
+
+
  void *
  ngx_http_map_find(ngx_http_request_t *r, ngx_http_map_t *map, 
ngx_str_t *match)
  {
-- 
1.7.11.1


From ru at nginx.com  Sun Dec  2 17:57:14 2012
From: ru at nginx.com (Ruslan Ermilov)
Date: Sun, 2 Dec 2012 21:57:14 +0400
Subject: [PATCH] Feature: $msec and $start_msec http variables,
 useful for setting in proxy headers before sending request to backend
 servers so processing and queue times can be measured.
In-Reply-To: <63A1809C-D39E-4761-95E3-FBCA3180F854@gmail.com>
References: <63A1809C-D39E-4761-95E3-FBCA3180F854@gmail.com>
Message-ID: <20121202175714.GA97426@lo0.su>

On Sat, Dec 01, 2012 at 10:33:48PM +1100, Lawrence Pit wrote:
[...]
> After I finished this patch I checked the 1.3 branch, and noticed it 
> already has an $msec http variable defined. But it's output is with a 
> dot character before the milliseconds part. Why is that?

The idea is to make all "log only" variables the normal ones.

> New Relic has decribed something similar:
> https://newrelic.com/docs/features/tracking-front-end-time
> 
> Aside: note that New Relic requires the time in microseconds. The only 
> way I could find to do this (with my patch) was a workaround using the 
> set command. With the 1.3 branch version of $msec there's no way I think 
> (or is there?).

http {
    map $msec $s {
	~(?<x>.*)\. $x;
    }

    map $msec $ms {
	~\.(?<x>.*) $x;
    }

    server {
	return 200 "msec=$msec usec=${s}${ms}000\n";
    }
}

$ curl http://localhost:8000/
msec=1354471012.591 usec=1354471012591000


From mdounin at mdounin.ru  Mon Dec  3 14:42:48 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 3 Dec 2012 18:42:48 +0400
Subject: Connection API?
In-Reply-To: <20121201001936.283a7d4a@baldur>
References: <20121201001936.283a7d4a@baldur>
Message-ID: <20121203144248.GY40452@mdounin.ru>

Hello!

On Sat, Dec 01, 2012 at 12:19:36AM +0000, Nick Kew wrote:

> I'm looking to port a module to nginx.  The module currently
> exists for Apache Trafficserver and Apache HTTPD.
> 
> Part of the module hooks into the start and end of the
> TCP connection, and maintains connection data in a connection
> context.  As far as I can tell, there's no equivalent API in
> nginx: there's no ctx to store context in ngx_connection_t,
> nor are there hooks to insert my modules functions at the start
> and end of a connection.
> 
> (a) Is there really no connection API, or am I just missing it?
> (b) If there really is no such API, would the nginx core team
>     be receptive to a patch introducing it?  I would envisage
>     a ctx in the connection struct, and a hookable
>     start-connection event.

As of now connection end may be hooked by using connection's pool 
cleanup handler.  There is no connection start hook in the current 
http module API.

It's probably worth adding, but I doubt we need full-featured API 
here.  In particular, full ctx array would be huge compared to a 
connection structure size which is kept small to minimize 
keepalive connections costs.  And the demand for a connection start 
hook looks extremelay low.

-- 
Maxim Dounin
http://nginx.com/support.html


From nick at webthing.com  Mon Dec  3 15:57:24 2012
From: nick at webthing.com (Nick Kew)
Date: Mon, 3 Dec 2012 15:57:24 +0000
Subject: Connection API?
In-Reply-To: <20121203144248.GY40452@mdounin.ru>
References: <20121201001936.283a7d4a@baldur>
 <20121203144248.GY40452@mdounin.ru>
Message-ID: <20121203155724.094c4d52@baldur>

On Mon, 3 Dec 2012 18:42:48 +0400
Maxim Dounin <mdounin at mdounin.ru> wrote:

> As of now connection end may be hooked by using connection's pool 
> cleanup handler.  There is no connection start hook in the current 
> http module API.
> 
> It's probably worth adding, but I doubt we need full-featured API 
> here.  In particular, full ctx array would be huge compared to a 
> connection structure size which is kept small to minimize 
> keepalive connections costs.  And the demand for a connection start 
> hook looks extremelay low.

Thanks for the reply.  You mean you're happy to add a new-connection
hook, but not a context?

In an HTTP context I can work around that: my module maintains a
pool of connection contexts, and associates each new HTTP request
with its connection in a post_read_request hook.

Can I assume that if requests r1 and r2 come on the same connection
then r1->connection == r2->connection?  That would enable me to save
a little overhead by using the connection pointer as a hash key,
rather than having to compute a key from the connection data.

-- 
Nick Kew


From ru at nginx.com  Mon Dec  3 16:00:26 2012
From: ru at nginx.com (ru at nginx.com)
Date: Mon,  3 Dec 2012 16:00:26 +0000
Subject: [nginx] svn commit: r4946 - in trunk: auto/lib/perl
 src/http/modules/perl
Message-ID: <20121203160026.9C3D53F9C0D@mail.nginx.com>

Author: ru
Date: 2012-12-03 16:00:26 +0000 (Mon, 03 Dec 2012)
New Revision: 4946
URL: http://trac.nginx.org/nginx/changeset/4946/nginx

Log:
Fixed build with embedded perl in certain setups (ticket #48).


Modified:
   trunk/auto/lib/perl/make
   trunk/src/http/modules/perl/Makefile.PL

Modified: trunk/auto/lib/perl/make
===================================================================
--- trunk/auto/lib/perl/make	2012-11-30 11:26:50 UTC (rev 4945)
+++ trunk/auto/lib/perl/make	2012-12-03 16:00:26 UTC (rev 4946)
@@ -26,9 +26,7 @@
 
 	cd $NGX_OBJS/src/http/modules/perl				\
 		&& NGX_PM_CFLAGS="\$(NGX_PM_CFLAGS) -g $NGX_CC_OPT"	\
-			NGX_PCRE=$PCRE					\
-			NGX_OBJS=$NGX_OBJS				\
-			NGX_OPENSSL=$OPENSSL				\
+			NGX_INCS="$CORE_INCS $NGX_OBJS $HTTP_INCS"	\
 		$NGX_PERL Makefile.PL					\
 			LIB=$NGX_PERL_MODULES				\
 			INSTALLSITEMAN3DIR=$NGX_PERL_MODULES_MAN

Modified: trunk/src/http/modules/perl/Makefile.PL
===================================================================
--- trunk/src/http/modules/perl/Makefile.PL	2012-11-30 11:26:50 UTC (rev 4945)
+++ trunk/src/http/modules/perl/Makefile.PL	2012-12-03 16:00:26 UTC (rev 4946)
@@ -16,20 +16,9 @@
     CCFLAGS           => "$ENV{NGX_PM_CFLAGS}",
     OPTIMIZE          => '-O',
 
-    INC               => "-I ../../../../../src/core " .
-                         "-I ../../../../../src/event " .
-                         "-I ../../../../../src/os/unix " .
-                         "-I ../../../../../src/http " .
-                         "-I ../../../../../src/http/modules " .
-                         "-I ../../../../../src/http/modules/perl " .
-                         "-I ../../../../../$ENV{NGX_OBJS} " .
-                         ($ENV{NGX_PCRE} =~ /^(YES|NO)/ ? "" :
-                             ($ENV{NGX_PCRE} =~ m#^/# ? "-I $ENV{NGX_PCRE} " :
-                                  "-I ../../../../../$ENV{NGX_PCRE} ")) .
-                         ($ENV{NGX_OPENSSL} =~ /^(YES|NO)/ ? "" :
-                             ($ENV{NGX_OPENSSL} =~ m#^/# ?
-                                  "-I $ENV{NGX_OPENSSL}/.openssl/include " :
-                      "-I ../../../../../$ENV{NGX_OPENSSL}/.openssl/include ")),
+    INC               => join(" ", map {
+                             m#^/# ? "-I $_" : "-I ../../../../../$_"
+                         } (split /\s+/, $ENV{NGX_INCS})),
 
     depend => {
         'nginx.c'     =>


From ruslan.khusnullin at gmail.com  Mon Dec  3 17:27:36 2012
From: ruslan.khusnullin at gmail.com (Ruslan Khusnullin)
Date: Mon, 3 Dec 2012 21:27:36 +0400
Subject: ngx_chain_get_free_buf crashes with free=0 after return
 next_body_filter
Message-ID: <CAG6Cn5bHnj_fV4EErrj1oogqfCw=7kmdtHiJr-p1C81=SgY8OA@mail.gmail.com>

Hello, I cannot find an issue root cause and hope for your help.

My nginx plugin crashes and core dumps on the line containing return in
body_filter function:

ngx_http_output_body_filter_pt my_next_body_filter;
static ngx_int_t ngx_http_my_init (ngx_conf_t * cf) {
    my_next_body_filter = ngx_http_top_body_filter;
    ngx_http_top_body_filter = ngx_http_my_body_filter;
    return NGX_OK;
}

static ngx_http_module_t ngx_http_my_module_ctx = {
    NULL,                                /* preconfiguration */
    ngx_http_my_init,            /* postconfiguration */
    ...
};

static ngx_int_t ngx_http_my_body_filter (ngx_http_request_t * r,
ngx_chain_t * in) {
    ngx_http_my_loc_conf_t * cf;
    cf = ngx_http_get_module_loc_conf (r, ngx_http_my_module);
    if (cf == NULL) return NGX_HTTP_INTERNAL_SERVER_ERROR;
    if (! cf->enabled || in == NULL || r->header_only) {
        return my_next_body_filter (r, in);
    }
    ...
}

I saw the same (or very similar) return statement in other official modules
and they seem no crashing. I can't understand why my module is crashing.

Here is what I've catched from gdb:
(gdb) bt
#0  ngx_chain_get_free_buf (p=0x80f5020, free=0x0) at src/core/ngx_buf.c:160
#1  0x08089fbe in ngx_http_chunked_body_filter (r=0x80de458, in=0xbfffe798)
at src/http/modules/ngx_http_chunked_filter_module.c:150
#2  0x0808ee07 in ngx_http_gzip_body_filter (r=0x80de458, in=0xbfffe798) at
src/http/modules/ngx_http_gzip_filter_module.c:324
#3  0x0808fc1a in ngx_http_postpone_filter (r=0x80de458, in=0x0) at
src/http/ngx_http_postpone_filter_module.c:83
#4  0x0809023d in ngx_http_ssi_body_filter (r=0x80de458, in=0x219) at
src/http/modules/ngx_http_ssi_filter_module.c:395
#5  0x08093ffd in ngx_http_charset_body_filter (r=0x80de458, in=0xbfffe798)
at src/http/modules/ngx_http_charset_filter_module.c:553
#6  0x080a65cb in ngx_http_my_body_filter (r=0x80de458, in=0xbfffe798) at
../../my/ngx_http_my_module.c:552
#7  0x0804fb22 in ngx_output_chain (ctx=0x80f778c, in=0xbfffe798) at
src/core/ngx_output_chain.c:66
#8  0x0807be9b in ngx_http_copy_filter (r=0x80de458, in=0xbfffe798) at
src/http/ngx_http_copy_filter_module.c:143
#9  0x0808a2a6 in ngx_http_range_body_filter (r=0x80de458, in=0xbfffe798)
at src/http/modules/ngx_http_range_filter_module.c:559
#10 0x0806f061 in ngx_http_output_filter (r=0x80de458, in=0xbfffe798) at
src/http/ngx_http_core_module.c:1912
#11 0x08089464 in ngx_http_static_handler (r=0x80de458) at
src/http/modules/ngx_http_static_module.c:266
#12 0x0807342f in ngx_http_core_content_phase (r=0x80de458, ph=0x80ecd3c)
at src/http/ngx_http_core_module.c:1403
#13 0x0806ecf5 in ngx_http_core_run_phases (r=0x80de458) at
src/http/ngx_http_core_module.c:877
#14 0x0806edfd in ngx_http_handler (r=0x219) at
src/http/ngx_http_core_module.c:860
#15 0x080725bd in ngx_http_internal_redirect (r=0x80de458, uri=0xbfffe884,
args=0xbfffe87c) at src/http/ngx_http_core_module.c:2545
#16 0x080752a1 in ngx_http_send_error_page (r=0x80de458, error=500) at
src/http/ngx_http_special_response.c:569
#17 ngx_http_special_response_handler (r=0x80de458, error=500) at
src/http/ngx_http_special_response.c:415
#18 0x0807779b in ngx_http_finalize_request (r=0x80de458, rc=500) at
src/http/ngx_http_request.c:2003
#19 0x080842b2 in ngx_http_upstream_finalize_request (r=0x80de458,
u=0x80f56ac, rc=500) at src/http/ngx_http_upstream.c:3095
#20 0x080856c4 in ngx_http_upstream_process_non_buffered_request
(r=0x80de458, do_write=<value optimized out>) at
src/http/ngx_http_upstream.c:2437
#21 0x08085994 in ngx_http_upstream_process_non_buffered_downstream
(r=0x80de458) at src/http/ngx_http_upstream.c:2368
#22 0x08086d5e in ngx_http_upstream_send_response (r=0x80de458,
u=0x80f56ac) at src/http/ngx_http_upstream.c:2141
#23 ngx_http_upstream_process_header (r=0x80de458, u=0x80f56ac) at
src/http/ngx_http_upstream.c:1644
#24 0x08084db5 in ngx_http_upstream_handler (ev=0x0) at
src/http/ngx_http_upstream.c:935
#25 0x080687b1 in ngx_epoll_process_events (cycle=0x80d6cd8, timer=60000,
flags=<value optimized out>) at src/event/modules/ngx_epoll_module.c:683
#26 0x0806152b in ngx_process_events_and_timers (cycle=0x80d6cd8) at
src/event/ngx_event.c:247
#27 0x08067427 in ngx_worker_process_cycle (cycle=0x80d6cd8, data=0x0) at
src/os/unix/ngx_process_cycle.c:810
#28 0x08065d01 in ngx_spawn_process (cycle=0x80d6cd8, proc=0x806736f
<ngx_worker_process_cycle>, data=0x0, name=0x80ae6c9 "worker process",
respawn=-3) at src/os/unix/ngx_process.c:198
#29 0x08066a7d in ngx_start_worker_processes (cycle=0x80d6cd8, n=1,
type=-3) at src/os/unix/ngx_process_cycle.c:365
#30 0x080679ae in ngx_master_process_cycle (cycle=0x80d6cd8) at
src/os/unix/ngx_process_cycle.c:137
#31 0x0804d381 in main (argc=1, argv=0xbfffedc4) at src/core/nginx.c:412

So ngx_chain_get_free_buf gets null pointer and I can't understand why it
does. The situation is: there is upstream configured and my body_filter
works with it, it should be "invisible" (do not modify contents) if it's
not enabled in config or whatelse (you can see conditions in the code
above).

Any ideas on what I'm doing wrong?

nginx-1.2.5
Linux 2.6.18 i686
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121203/1c11c231/attachment.html>

From mdounin at mdounin.ru  Mon Dec  3 17:31:52 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 3 Dec 2012 21:31:52 +0400
Subject: Connection API?
In-Reply-To: <20121203155724.094c4d52@baldur>
References: <20121201001936.283a7d4a@baldur>
 <20121203144248.GY40452@mdounin.ru>
 <20121203155724.094c4d52@baldur>
Message-ID: <20121203173152.GA40452@mdounin.ru>

Hello!

On Mon, Dec 03, 2012 at 03:57:24PM +0000, Nick Kew wrote:

> On Mon, 3 Dec 2012 18:42:48 +0400
> Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> > As of now connection end may be hooked by using connection's pool 
> > cleanup handler.  There is no connection start hook in the current 
> > http module API.
> > 
> > It's probably worth adding, but I doubt we need full-featured API 
> > here.  In particular, full ctx array would be huge compared to a 
> > connection structure size which is kept small to minimize 
> > keepalive connections costs.  And the demand for a connection start 
> > hook looks extremelay low.
> 
> Thanks for the reply.  You mean you're happy to add a new-connection
> hook, but not a context?

Something like this.  I would like to see use case first though, 
to make sure it indeed requires connection start hook.

Simple way to store module-specific per-connection data without 
ctx array I could think of is something based on connection pool 
cleanup handler.  I.e. one can register connection pool cleanup 
handler, and then find associated data by iterating over 
connection pool cleanup handlers installed.

It will be more costly than lookup via ctx array, but shouldn't be 
too costly as there shouldn't be more than one such lookup per 
request and I expect only few such modules.  Data lookup will be 
also free in connection pool cleanup handler itself.

> In an HTTP context I can work around that: my module maintains a
> pool of connection contexts, and associates each new HTTP request
> with its connection in a post_read_request hook.
> 
> Can I assume that if requests r1 and r2 come on the same connection
> then r1->connection == r2->connection?  That would enable me to save
> a little overhead by using the connection pointer as a hash key,
> rather than having to compute a key from the connection data.

As of now its always true.  Note though, that with SPDY patches 
Valentin is working on it will likely change.  I've just talked to 
him and he suggests to compare r[12]->connection->fd instead as a 
more bulletproof solution.

-- 
Maxim Dounin
http://nginx.com/support.html


From ibobrik at gmail.com  Mon Dec  3 17:55:10 2012
From: ibobrik at gmail.com (ivan babrou)
Date: Mon, 3 Dec 2012 21:55:10 +0400
Subject: image_filter enhancement
In-Reply-To: <20121127160328.GZ40452@mdounin.ru>
References: <CANWdNRDP_6+WHtCawX8KnoH4GFHHQeHC2tXY6-8gDctYq0Dnrg@mail.gmail.com>
 <20121106181601.GS40452@mdounin.ru>
 <CANWdNRDYuMDZOpcEwtsryfUpTBCsdB6VsSJfF+qUJV34y5j08g@mail.gmail.com>
 <20121114234732.GC40452@mdounin.ru>
 <CANWdNRB=bRCEjEBM+ksak8c6pEZw+VC1XfKBrPLEJ5zXv+B_=g@mail.gmail.com>
 <20121127160328.GZ40452@mdounin.ru>
Message-ID: <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>

Reworked again. Looks much better now! :)

I renamed configuration directive to image_filter_crop_offset.

diff --git a/ngx_http_image_filter_module.c b/ngx_http_image_filter_module.c
index c853c33..4d0ce95 100644
--- a/ngx_http_image_filter_module.c
+++ b/ngx_http_image_filter_module.c
@@ -32,6 +32,11 @@
 #define NGX_HTTP_IMAGE_GIF       2
 #define NGX_HTTP_IMAGE_PNG       3

+#define NGX_HTTP_IMAGE_OFFSET_CENTER    0
+#define NGX_HTTP_IMAGE_OFFSET_LEFT      1
+#define NGX_HTTP_IMAGE_OFFSET_RIGHT     2
+#define NGX_HTTP_IMAGE_OFFSET_TOP       3
+#define NGX_HTTP_IMAGE_OFFSET_BOTTOM    4

 #define NGX_HTTP_IMAGE_BUFFERED  0x08

@@ -43,11 +48,15 @@ typedef struct {
     ngx_uint_t                   angle;
     ngx_uint_t                   jpeg_quality;
     ngx_uint_t                   sharpen;
+    ngx_uint_t                   crop_offset_x;
+    ngx_uint_t                   crop_offset_y;

     ngx_flag_t                   transparency;

     ngx_http_complex_value_t    *wcv;
     ngx_http_complex_value_t    *hcv;
+    ngx_http_complex_value_t    *oxv;
+    ngx_http_complex_value_t    *oyv;
     ngx_http_complex_value_t    *acv;
     ngx_http_complex_value_t    *jqcv;
     ngx_http_complex_value_t    *shcv;
@@ -66,6 +75,8 @@ typedef struct {
     ngx_uint_t                   height;
     ngx_uint_t                   max_width;
     ngx_uint_t                   max_height;
+    ngx_uint_t                   crop_offset_x;
+    ngx_uint_t                   crop_offset_y;
     ngx_uint_t                   angle;

     ngx_uint_t                   phase;
@@ -110,6 +121,8 @@ static char
*ngx_http_image_filter_jpeg_quality(ngx_conf_t *cf,
     ngx_command_t *cmd, void *conf);
 static char *ngx_http_image_filter_sharpen(ngx_conf_t *cf, ngx_command_t *cmd,
     void *conf);
+static char *ngx_http_image_filter_offset(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf);
 static ngx_int_t ngx_http_image_filter_init(ngx_conf_t *cf);


@@ -150,6 +163,13 @@ static ngx_command_t  ngx_http_image_filter_commands[] = {
       offsetof(ngx_http_image_filter_conf_t, buffer_size),
       NULL },

+    { ngx_string("image_filter_crop_offset"),
+      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
+      ngx_http_image_filter_offset,
+      NGX_HTTP_LOC_CONF_OFFSET,
+      0,
+      NULL },
+
       ngx_null_command
 };

@@ -737,7 +757,8 @@ ngx_http_image_resize(ngx_http_request_t *r,
ngx_http_image_filter_ctx_t *ctx)
 {
     int                            sx, sy, dx, dy, ox, oy, ax, ay, size,
                                    colors, palette, transparent, sharpen,
-                                   red, green, blue, t;
+                                   red, green, blue, t,
+                                   crop_offset_x, crop_offset_y;
     u_char                        *out;
     ngx_buf_t                     *b;
     ngx_uint_t                     resize;
@@ -932,8 +953,27 @@ transparent:
                 return NULL;
             }

-            ox /= 2;
-            oy /= 2;
+            crop_offset_x = conf->crop_offset_x;
+            crop_offset_y = conf->crop_offset_y;
+
+            crop_offset_x = ngx_http_image_filter_get_value(r, conf->oxv,
+                                                            crop_offset_x);
+            crop_offset_y = ngx_http_image_filter_get_value(r, conf->oyv,
+                                                            crop_offset_y);
+
+            if (crop_offset_x == NGX_HTTP_IMAGE_OFFSET_LEFT) {
+                ox = 0;
+
+            } else if (crop_offset_x == NGX_HTTP_IMAGE_OFFSET_CENTER) {
+                ox /= 2;
+            }
+
+            if (crop_offset_y == NGX_HTTP_IMAGE_OFFSET_TOP) {
+                oy = 0;
+
+            } else if (crop_offset_y == NGX_HTTP_IMAGE_OFFSET_CENTER) {
+                oy /= 2;
+            }

             ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                            "image crop: %d x %d @ %d x %d",
@@ -1149,10 +1189,27 @@ ngx_http_image_filter_value(ngx_str_t *value)
         return (ngx_uint_t) -1;
     }

-    n = ngx_atoi(value->data, value->len);
+    if (ngx_strcmp(value->data, "center") == 0) {
+        return NGX_HTTP_IMAGE_OFFSET_CENTER;
+
+    } else if (ngx_strcmp(value->data, "left") == 0) {
+        return NGX_HTTP_IMAGE_OFFSET_LEFT;
+
+    } else if (ngx_strcmp(value->data, "right") == 0) {
+        return NGX_HTTP_IMAGE_OFFSET_RIGHT;
+
+    } else if (ngx_strcmp(value->data, "top") == 0) {
+        return NGX_HTTP_IMAGE_OFFSET_TOP;

-    if (n > 0) {
-        return (ngx_uint_t) n;
+    } else if (ngx_strcmp(value->data, "bottom") == 0) {
+        return NGX_HTTP_IMAGE_OFFSET_BOTTOM;
+
+    } else {
+        n = ngx_atoi(value->data, value->len);
+
+        if (n > 0) {
+            return (ngx_uint_t) n;
+        }
     }

     return 0;
@@ -1175,6 +1232,8 @@ ngx_http_image_filter_create_conf(ngx_conf_t *cf)
     conf->angle = NGX_CONF_UNSET_UINT;
     conf->transparency = NGX_CONF_UNSET;
     conf->buffer_size = NGX_CONF_UNSET_SIZE;
+    conf->crop_offset_x = NGX_CONF_UNSET_UINT;
+    conf->crop_offset_y = NGX_CONF_UNSET_UINT;

     return conf;
 }
@@ -1223,6 +1282,17 @@ ngx_http_image_filter_merge_conf(ngx_conf_t
*cf, void *parent, void *child)
     ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size,
                               1 * 1024 * 1024);

+    ngx_conf_merge_uint_value(conf->crop_offset_x,
prev->crop_offset_x, NGX_HTTP_IMAGE_OFFSET_CENTER);
+    ngx_conf_merge_uint_value(conf->crop_offset_y,
prev->crop_offset_y, NGX_HTTP_IMAGE_OFFSET_CENTER);
+
+    if (conf->oxv == NULL) {
+        conf->oxv = prev->oxv;
+    }
+
+    if (conf->oyv == NULL) {
+        conf->oyv = prev->oyv;
+    }
+
     return NGX_CONF_OK;
 }

@@ -1474,6 +1544,68 @@ ngx_http_image_filter_sharpen(ngx_conf_t *cf,
ngx_command_t *cmd,
 }


+static char *
+ngx_http_image_filter_offset(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf)
+{
+    ngx_http_image_filter_conf_t *imcf = conf;
+
+    ngx_str_t                         *value;
+    ngx_http_complex_value_t           cv;
+    ngx_http_compile_complex_value_t   ccv;
+
+    value = cf->args->elts;
+
+    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
+
+    ccv.cf = cf;
+    ccv.value = &value[1];
+    ccv.complex_value = &cv;
+    ccv.zero = 1;
+
+    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
+        return NGX_CONF_ERROR;
+    }
+
+    if (cv.lengths == NULL) {
+        imcf->crop_offset_x = ngx_http_image_filter_value(&value[1]);
+
+    } else {
+        imcf->oxv = ngx_palloc(cf->pool, sizeof(ngx_http_complex_value_t));
+        if (imcf->oxv == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        *imcf->oxv = cv;
+    }
+
+    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
+
+    ccv.cf = cf;
+    ccv.value = &value[2];
+    ccv.complex_value = &cv;
+    ccv.zero = 1;
+
+    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
+        return NGX_CONF_ERROR;
+    }
+
+    if (cv.lengths == NULL) {
+        imcf->crop_offset_y = ngx_http_image_filter_value(&value[2]);
+
+    } else {
+        imcf->oyv = ngx_palloc(cf->pool, sizeof(ngx_http_complex_value_t));
+        if (imcf->oyv == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        *imcf->oyv = cv;
+    }
+
+    return NGX_CONF_OK;
+}
+
+
 static ngx_int_t
 ngx_http_image_filter_init(ngx_conf_t *cf)
 {

On 27 November 2012 20:03, Maxim Dounin <mdounin at mdounin.ru> wrote:
> Hello!
>
> On Tue, Nov 27, 2012 at 12:10:13AM +0400, ivan babrou wrote:
>
>> I tried to rework patch. Now it supports configuration through
>> variables. Variables ox and oy renamed to crop_offset_x and
>> crop_offset_y.
>>
>> I'm confused about this line:
>> +    value.data[value.len] = 0;
>>
>> Probably I don't understand something about memory management in nginx.
>
> See below.
>
>>
>>
>> diff --git a/ngx_http_image_filter_module.c b/ngx_http_image_filter_module.c
>> index c853c33..15f8d17 100644
>> --- a/ngx_http_image_filter_module.c
>> +++ b/ngx_http_image_filter_module.c
>> @@ -32,6 +32,14 @@
>>  #define NGX_HTTP_IMAGE_GIF       2
>>  #define NGX_HTTP_IMAGE_PNG       3
>>
>> +#define NGX_HTTP_IMAGE_OFFSET_TYPE_HORIZONTAL 0
>> +#define NGX_HTTP_IMAGE_OFFSET_TYPE_VERTICAL 1
>> +
>> +#define NGX_HTTP_IMAGE_OFFSET_CENTER    0
>> +#define NGX_HTTP_IMAGE_OFFSET_LEFT      1
>> +#define NGX_HTTP_IMAGE_OFFSET_RIGHT     2
>> +#define NGX_HTTP_IMAGE_OFFSET_TOP       3
>> +#define NGX_HTTP_IMAGE_OFFSET_BOTTOM    4
>>
>>  #define NGX_HTTP_IMAGE_BUFFERED  0x08
>>
>> @@ -43,11 +51,15 @@ typedef struct {
>>      ngx_uint_t                   angle;
>>      ngx_uint_t                   jpeg_quality;
>>      ngx_uint_t                   sharpen;
>> +    ngx_uint_t                   crop_offset_x;
>> +    ngx_uint_t                   crop_offset_y;
>>
>>      ngx_flag_t                   transparency;
>>
>>      ngx_http_complex_value_t    *wcv;
>>      ngx_http_complex_value_t    *hcv;
>> +    ngx_http_complex_value_t    *oxv;
>> +    ngx_http_complex_value_t    *oyv;
>
> Probably you mean "oxcv", "oycv" - "cv" here is abbreviation of
> "complex value".
>
>>      ngx_http_complex_value_t    *acv;
>>      ngx_http_complex_value_t    *jqcv;
>>      ngx_http_complex_value_t    *shcv;
>> @@ -66,6 +78,8 @@ typedef struct {
>>      ngx_uint_t                   height;
>>      ngx_uint_t                   max_width;
>>      ngx_uint_t                   max_height;
>> +    ngx_uint_t                   crop_offset_x;
>> +    ngx_uint_t                   crop_offset_y;
>>      ngx_uint_t                   angle;
>>
>>      ngx_uint_t                   phase;
>> @@ -98,6 +112,15 @@ static u_char
>> *ngx_http_image_out(ngx_http_request_t *r, ngx_uint_t type,
>>  static void ngx_http_image_cleanup(void *data);
>>  static ngx_uint_t ngx_http_image_filter_get_value(ngx_http_request_t *r,
>>      ngx_http_complex_value_t *cv, ngx_uint_t v);
>> +static void
>> +ngx_http_image_filter_crop_offset(ngx_http_request_t *r,
>> +    ngx_http_complex_value_t *cv, ngx_uint_t t, ngx_uint_t *v);
>> +static void
>> +ngx_http_image_filter_vertical_crop_offset(ngx_str_t *value,
>> +    ngx_uint_t *v);
>> +static void
>> +ngx_http_image_filter_horizontal_crop_offset(ngx_str_t *value,
>> +    ngx_uint_t *v);
>>  static ngx_uint_t ngx_http_image_filter_value(ngx_str_t *value);
>>
>>
>> @@ -110,6 +133,8 @@ static char
>> *ngx_http_image_filter_jpeg_quality(ngx_conf_t *cf,
>>      ngx_command_t *cmd, void *conf);
>>  static char *ngx_http_image_filter_sharpen(ngx_conf_t *cf, ngx_command_t *cmd,
>>      void *conf);
>> +static char *ngx_http_image_filter_offset(ngx_conf_t *cf, ngx_command_t *cmd,
>> +    void *conf);
>>  static ngx_int_t ngx_http_image_filter_init(ngx_conf_t *cf);
>>
>>
>> @@ -150,6 +175,13 @@ static ngx_command_t  ngx_http_image_filter_commands[] = {
>>        offsetof(ngx_http_image_filter_conf_t, buffer_size),
>>        NULL },
>>
>> +    { ngx_string("image_filter_offset"),
>> +      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
>> +      ngx_http_image_filter_offset,
>> +      NGX_HTTP_LOC_CONF_OFFSET,
>> +      0,
>> +      NULL },
>> +
>>        ngx_null_command
>>  };
>>
>> @@ -529,6 +561,22 @@ ngx_http_image_process(ngx_http_request_t *r)
>>          return NULL;
>>      }
>>
>> +    if (conf->oxv == NULL) {
>> +        ctx->crop_offset_x = conf->crop_offset_x;
>> +    } else {
>> +        ngx_http_image_filter_crop_offset(r, conf->oxv,
>> +
>> NGX_HTTP_IMAGE_OFFSET_TYPE_HORIZONTAL,
>> +                                          &ctx->crop_offset_x);
>> +    }
>> +
>> +    if (conf->oyv == NULL) {
>> +        ctx->crop_offset_y = conf->crop_offset_y;
>> +    } else {
>> +        ngx_http_image_filter_crop_offset(r, conf->oyv,
>> +                                          NGX_HTTP_IMAGE_OFFSET_TYPE_VERTICAL,
>> +                                          &ctx->crop_offset_y);
>> +    }
>> +
>
> 1) You want to follow pattern in the previous code, i.e.
>    foo = ngx_http_image_filter_get_value(r, conf->foocv, conf->foo);
>
>    (With additional support of top/bottom/left/right literals.)
>
> 2) I don't think you need to calculate offsets to decide whether
>    image is ok as is or we need to resize it.  Hence it's should
>    be ok to move the code to where it's needed.
>
> 3) With (2) fixed you probably don't need variables in ctx
>    anymore.
>
>>      if (rc == NGX_OK
>>          && ctx->width <= ctx->max_width
>>          && ctx->height <= ctx->max_height
>> @@ -932,8 +980,17 @@ transparent:
>>                  return NULL;
>>              }
>>
>> -            ox /= 2;
>> -            oy /= 2;
>> +            if (ctx->crop_offset_x == NGX_HTTP_IMAGE_OFFSET_LEFT) {
>> +                ox = 0;
>> +            } else if (ctx->crop_offset_x == NGX_HTTP_IMAGE_OFFSET_CENTER) {
>> +                ox /= 2;
>> +            }
>> +
>> +            if (ctx->crop_offset_y == NGX_HTTP_IMAGE_OFFSET_TOP) {
>> +                oy = 0;
>> +            } else if (ctx->crop_offset_y == NGX_HTTP_IMAGE_OFFSET_CENTER) {
>> +                oy /= 2;
>> +            }
>
> Note: this needs empty lines before "} else if ..." to match
> style.  It should be changed anyway though, due to reasons
> outlined above.
>
>>
>>              ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
>>                             "image crop: %d x %d @ %d x %d",
>> @@ -1139,6 +1196,52 @@ ngx_http_image_filter_get_value(ngx_http_request_t *r,
>>      return ngx_http_image_filter_value(&val);
>>  }
>>
>> +static void
>> +ngx_http_image_filter_crop_offset(ngx_http_request_t *r,
>> +    ngx_http_complex_value_t *cv, ngx_uint_t t, ngx_uint_t *v)
>> +{
>> +    ngx_str_t value;
>> +
>> +    if (ngx_http_complex_value(r, cv, &value) != NGX_OK) {
>> +        return;
>> +    }
>> +
>> +    value.data[value.len] = 0;
>
> This is incorrect and will result in memory corruption.
>
> If you need null-terminated string, right aproach is to set
> ccv.zero flag while calling ngx_http_compile_complex_value().  On
> the other hand, it might be good idea to don't assume
> null-termination instead.
>
>> +
>> +    if (t == NGX_HTTP_IMAGE_OFFSET_TYPE_HORIZONTAL) {
>> +        ngx_http_image_filter_horizontal_crop_offset(&value, v);
>> +    } else if (t == NGX_HTTP_IMAGE_OFFSET_TYPE_VERTICAL) {
>> +        ngx_http_image_filter_vertical_crop_offset(&value, v);
>> +    }
>> +}
>> +
>> +
>> +static void
>> +ngx_http_image_filter_horizontal_crop_offset(ngx_str_t *value,
>> +    ngx_uint_t *v)
>> +{
>> +    if (ngx_strcmp(value->data, "center") == 0) {
>> +        *v = NGX_HTTP_IMAGE_OFFSET_CENTER;
>> +    } else if (ngx_strcmp(value->data, "left") == 0) {
>> +        *v = NGX_HTTP_IMAGE_OFFSET_LEFT;
>> +    } else if (ngx_strcmp(value->data, "right") == 0) {
>> +        *v = NGX_HTTP_IMAGE_OFFSET_RIGHT;
>> +    }
>> +}
>> +
>> +static void
>> +ngx_http_image_filter_vertical_crop_offset(ngx_str_t *value,
>> +    ngx_uint_t *v)
>> +{
>> +    if (ngx_strcmp(value->data, "center") == 0) {
>> +        *v = NGX_HTTP_IMAGE_OFFSET_CENTER;
>> +    } else if (ngx_strcmp(value->data, "top") == 0) {
>> +        *v = NGX_HTTP_IMAGE_OFFSET_TOP;
>> +    } else if (ngx_strcmp(value->data, "bottom") == 0) {
>> +        *v = NGX_HTTP_IMAGE_OFFSET_BOTTOM;
>> +    }
>> +}
>> +
>
> I don't think we need so many functions to do this simple task.
> It would be good idea simplify this.
>
> The NGX_HTTP_IMAGE_OFFSET_TYPE_HORIZONTAL and
> NGX_HTTP_IMAGE_OFFSET_TYPE_VERTICAL seem to be unneeded, just a
> boolean value would do the trick.
>
>>  static ngx_uint_t
>>  ngx_http_image_filter_value(ngx_str_t *value)
>> @@ -1175,6 +1278,8 @@ ngx_http_image_filter_create_conf(ngx_conf_t *cf)
>>      conf->angle = NGX_CONF_UNSET_UINT;
>>      conf->transparency = NGX_CONF_UNSET;
>>      conf->buffer_size = NGX_CONF_UNSET_SIZE;
>> +    conf->crop_offset_x = NGX_CONF_UNSET_UINT;
>> +    conf->crop_offset_y = NGX_CONF_UNSET_UINT;
>>
>>      return conf;
>>  }
>> @@ -1223,6 +1328,17 @@ ngx_http_image_filter_merge_conf(ngx_conf_t
>> *cf, void *parent, void *child)
>>      ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size,
>>                                1 * 1024 * 1024);
>>
>> +    ngx_conf_merge_uint_value(conf->crop_offset_x,
>> prev->crop_offset_x, NGX_HTTP_IMAGE_OFFSET_CENTER);
>> +    ngx_conf_merge_uint_value(conf->crop_offset_y,
>> prev->crop_offset_y, NGX_HTTP_IMAGE_OFFSET_CENTER);
>> +
>> +    if (conf->oxv == NULL) {
>> +        conf->oxv = prev->oxv;
>> +    }
>> +
>> +    if (conf->oyv == NULL) {
>> +        conf->oyv = prev->oyv;
>> +    }
>> +
>
> Style: please keep lines shorter than 80 chars.
>
> Additionally, looking at the code I tend to think it's incorrect.
> That is, it's pattern you've followed is incorrect, not your code.
> E.g. the following config will result in $arg_q incorrectly used
> for quality in /image/, instead of "80" explicitly set:
>
>     image_filter_jpeg_quality $arg_q;
>
>     location /image/ {
>         image_filter crop $arg_w $arg_h;
>         image_filter_jpeg_quality 80;
>     }
>
> This needs fixing.
>
>>      return NGX_CONF_OK;
>>  }
>>
>> @@ -1474,6 +1590,64 @@ ngx_http_image_filter_sharpen(ngx_conf_t *cf,
>> ngx_command_t *cmd,
>>  }
>>
>>
>> +static char *
>> +ngx_http_image_filter_offset(ngx_conf_t *cf, ngx_command_t *cmd,
>> +    void *conf)
>> +{
>> +    ngx_http_image_filter_conf_t *imcf = conf;
>> +
>> +    ngx_str_t                         *value;
>> +    ngx_http_complex_value_t           cv;
>> +    ngx_http_compile_complex_value_t   ccv;
>> +
>> +    value = cf->args->elts;
>> +
>> +    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
>> +
>> +    ccv.cf = cf;
>> +    ccv.value = &value[1];
>> +    ccv.complex_value = &cv;
>> +
>> +    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
>> +        return NGX_CONF_ERROR;
>> +    }
>> +
>> +    if (cv.lengths == NULL) {
>> +        ngx_http_image_filter_horizontal_crop_offset(&value[1],
>> &imcf->crop_offset_x);
>> +    } else {
>> +        imcf->oxv = ngx_palloc(cf->pool, sizeof(ngx_http_complex_value_t));
>> +        if (imcf->oxv == NULL) {
>> +            return NGX_CONF_ERROR;
>> +        }
>> +
>> +        *imcf->oxv = cv;
>> +    }
>> +
>> +    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
>> +
>> +    ccv.cf = cf;
>> +    ccv.value = &value[2];
>> +    ccv.complex_value = &cv;
>> +
>> +    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
>> +        return NGX_CONF_ERROR;
>> +    }
>> +
>> +    if (cv.lengths == NULL) {
>> +        ngx_http_image_filter_vertical_crop_offset(&value[2],
>> &imcf->crop_offset_y);
>> +    } else {
>> +        imcf->oyv = ngx_palloc(cf->pool, sizeof(ngx_http_complex_value_t));
>> +        if (imcf->oyv == NULL) {
>> +            return NGX_CONF_ERROR;
>> +        }
>> +
>> +        *imcf->oyv = cv;
>> +    }
>> +
>> +    return NGX_CONF_OK;
>> +}
>> +
>> +
>>  static ngx_int_t
>>  ngx_http_image_filter_init(ngx_conf_t *cf)
>>  {
>>
>>
>> On 15 November 2012 03:47, Maxim Dounin <mdounin at mdounin.ru> wrote:
>> > offsets via variables.  On the other hand, his patch asks for
>>
>>
>>
>> --
>> Regards, Ian Babrou
>> http://bobrik.name http://twitter.com/ibobrik skype:i.babrou
>>
>> _______________________________________________
>> nginx-devel mailing list
>> nginx-devel at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
> --
> Maxim Dounin
> http://nginx.com/support.html
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel



-- 
Regards, Ian Babrou
http://bobrik.name http://twitter.com/ibobrik skype:i.babrou


From mdounin at mdounin.ru  Mon Dec  3 18:09:49 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 3 Dec 2012 22:09:49 +0400
Subject: ngx_chain_get_free_buf crashes with free=0 after return
 next_body_filter
In-Reply-To: <CAG6Cn5bHnj_fV4EErrj1oogqfCw=7kmdtHiJr-p1C81=SgY8OA@mail.gmail.com>
References: <CAG6Cn5bHnj_fV4EErrj1oogqfCw=7kmdtHiJr-p1C81=SgY8OA@mail.gmail.com>
Message-ID: <20121203180949.GB40452@mdounin.ru>

Hello!

On Mon, Dec 03, 2012 at 09:27:36PM +0400, Ruslan Khusnullin wrote:

> Hello, I cannot find an issue root cause and hope for your help.
> 
> My nginx plugin crashes and core dumps on the line containing return in
> body_filter function:
> 
> ngx_http_output_body_filter_pt my_next_body_filter;
> static ngx_int_t ngx_http_my_init (ngx_conf_t * cf) {
>     my_next_body_filter = ngx_http_top_body_filter;
>     ngx_http_top_body_filter = ngx_http_my_body_filter;
>     return NGX_OK;
> }
> 
> static ngx_http_module_t ngx_http_my_module_ctx = {
>     NULL,                                /* preconfiguration */
>     ngx_http_my_init,            /* postconfiguration */
>     ...
> };
> 
> static ngx_int_t ngx_http_my_body_filter (ngx_http_request_t * r,
> ngx_chain_t * in) {
>     ngx_http_my_loc_conf_t * cf;
>     cf = ngx_http_get_module_loc_conf (r, ngx_http_my_module);
>     if (cf == NULL) return NGX_HTTP_INTERNAL_SERVER_ERROR;

Note: returning NGX_HTTP_INTERNAL_SERVER_ERROR is completely wrong 
here, and it will result in undefined behaviour if ever returned.

You should either return NGX_ERROR here (probably logging an alert 
message) or just assume it should never happen unless process 
memory is corrupted and just dereference the pointer (this way 
you'll get proper segmentation fault instead of undefined 
behviour).

>     if (! cf->enabled || in == NULL || r->header_only) {
>         return my_next_body_filter (r, in);
>     }
>     ...
> }
> 
> I saw the same (or very similar) return statement in other official modules
> and they seem no crashing. I can't understand why my module is crashing.
> 
> Here is what I've catched from gdb:
> (gdb) bt
> #0  ngx_chain_get_free_buf (p=0x80f5020, free=0x0) at src/core/ngx_buf.c:160
> #1  0x08089fbe in ngx_http_chunked_body_filter (r=0x80de458, in=0xbfffe798)
> at src/http/modules/ngx_http_chunked_filter_module.c:150

This part of the backtrace suggests the r->chunked flag was 
somehow set outside of chunked filter, with chunked filter context 
not set (== NULL).

[...]

> #16 0x080752a1 in ngx_http_send_error_page (r=0x80de458, error=500) at
> src/http/ngx_http_special_response.c:569
> #17 ngx_http_special_response_handler (r=0x80de458, error=500) at
> src/http/ngx_http_special_response.c:415
> #18 0x0807779b in ngx_http_finalize_request (r=0x80de458, rc=500) at
> src/http/ngx_http_request.c:2003
> #19 0x080842b2 in ngx_http_upstream_finalize_request (r=0x80de458,
> u=0x80f56ac, rc=500) at src/http/ngx_http_upstream.c:3095
> #20 0x080856c4 in ngx_http_upstream_process_non_buffered_request
> (r=0x80de458, do_write=<value optimized out>) at
> src/http/ngx_http_upstream.c:2437

Here the request is finalized with rc=500 according to the 
backtrace, but the line it claims to contain the call clearly uses 
rc=0.  This is probably an optimization artifact, but 500 here 
suggest the problem in your code outlined above might be related.

[...]

> So ngx_chain_get_free_buf gets null pointer and I can't understand why it
> does. The situation is: there is upstream configured and my body_filter
> works with it, it should be "invisible" (do not modify contents) if it's
> not enabled in config or whatelse (you can see conditions in the code
> above).
> 
> Any ideas on what I'm doing wrong?
> 
> nginx-1.2.5
> Linux 2.6.18 i686

See above for a possible explanation.

-- 
Maxim Dounin
http://nginx.com/support.html


From ruslan.khusnullin at gmail.com  Mon Dec  3 18:34:22 2012
From: ruslan.khusnullin at gmail.com (Ruslan Khusnullin)
Date: Mon, 3 Dec 2012 22:34:22 +0400
Subject: ngx_chain_get_free_buf crashes with free=0 after return
 next_body_filter
In-Reply-To: <20121203180949.GB40452@mdounin.ru>
References: <CAG6Cn5bHnj_fV4EErrj1oogqfCw=7kmdtHiJr-p1C81=SgY8OA@mail.gmail.com>
 <20121203180949.GB40452@mdounin.ru>
Message-ID: <CAG6Cn5Yawu2Yz0RHyC93fbGS-0zSWEk7BrZnH+1+OVPKS5jqEg@mail.gmail.com>

On Mon, Dec 3, 2012 at 10:09 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Note: returning NGX_HTTP_INTERNAL_SERVER_ERROR is completely wrong
> here, and it will result in undefined behaviour if ever returned.
>
> You should either return NGX_ERROR here (probably logging an alert
> message) or just assume it should never happen unless process
> memory is corrupted and just dereference the pointer (this way
> you'll get proper segmentation fault instead of undefined
> behviour).

Thank you for this, I have replaced all NGX_HTTP_INTERNAL_SERVER_ERROR
with NGX_ERROR in body_filter and now it doesn't crash.

So is returning NGX_HTTP_*_ERROR valid only in a handler function?

>> Here is what I've catched from gdb:
>> (gdb) bt
>> #0  ngx_chain_get_free_buf (p=0x80f5020, free=0x0) at src/core/ngx_buf.c:160
>> #1  0x08089fbe in ngx_http_chunked_body_filter (r=0x80de458, in=0xbfffe798)
>> at src/http/modules/ngx_http_chunked_filter_module.c:150
>
> This part of the backtrace suggests the r->chunked flag was
> somehow set outside of chunked filter, with chunked filter context
> not set (== NULL).

Strange, I didn't touch other modules' contexts but may be it is
really related to invallid returning above.

Could you please also comment below piece of code? I've got it from
previous coders and I'm not sure it needs to be there. It is a version
of ngx_chain_add_copy that really copies data instead of copying
pointers. Does it make sense? The module copies all chunks from
upstream cache with this function and then when meeting buf->last_buf,
it composes whole response in one memory block and modifies it. I
guess that all the chunks remain in memory for all request time or I
really need to make a copy of data in my context?

static ngx_int_t
chain_add_copy (ngx_pool_t * pool, ngx_chain_t ** chain, ngx_chain_t *
in, size_t * data_size) {
    ngx_chain_t * cl, ** ll;
    size_t len;

    ll = chain;
    for (cl = * chain; cl != NULL; cl = cl->next) {
        ll = & cl->next;
    }

    while (in) {
        len = in->buf->last - in->buf->pos;

        cl = ngx_alloc_chain_link (pool);
        if (cl == NULL) {
            return NGX_ERROR;
        }

        cl->buf = ngx_create_temp_buf (pool, len);
        if (cl->buf == NULL) {
            return NGX_ERROR;
        }

        cl->buf->last = ngx_copy (cl->buf->last, in->buf->pos, len);
        * data_size += len;

        * ll = cl;
        ll = & cl->next;
        in = in->next;
    }

    * ll = NULL;

    return NGX_OK;
}

Thanks in advance.


From nick at webthing.com  Mon Dec  3 18:37:56 2012
From: nick at webthing.com (Nick Kew)
Date: Mon, 3 Dec 2012 18:37:56 +0000
Subject: Connection API?
In-Reply-To: <20121203173152.GA40452@mdounin.ru>
References: <20121201001936.283a7d4a@baldur>
 <20121203144248.GY40452@mdounin.ru>
 <20121203155724.094c4d52@baldur>
 <20121203173152.GA40452@mdounin.ru>
Message-ID: <20121203183756.11b51d11@baldur>

On Mon, 3 Dec 2012 21:31:52 +0400
Maxim Dounin <mdounin at mdounin.ru> wrote:

> > Thanks for the reply.  You mean you're happy to add a new-connection
> > hook, but not a context?
> 
> Something like this.  I would like to see use case first though, 
> to make sure it indeed requires connection start hook.

The basic use case is for a module that hooks in a library
that needs to be notified of new connections.

There are a couple of reasons for this.  One is that it has
its own connection struct that needs to be initialised before
it can process requests on the connection.  To do that
per-request would require a lookup table of existing
connections, and a mutex to prevent a race condition
between requests on a connection.  In other words, quite
a lot of extra complexity.

The other reason is, this is a security tool, and we'd like
to know about any connections that are NOT followed by a
valid (and notifiable) HTTP request, and might be part of
an attack on a server.

> As of now its always true.  Note though, that with SPDY patches 
> Valentin is working on it will likely change.  I've just talked to 
> him and he suggests to compare r[12]->connection->fd instead as a 
> more bulletproof solution.

Surely an fd will be recycled/re-used?

OK, I can work around that, too: just a little extra work to re-use
the library's connection initialisation to generate a key.
Though it means holding a mutex rather longer!

-- 
Nick Kew


From mdounin at mdounin.ru  Mon Dec  3 19:06:13 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 3 Dec 2012 23:06:13 +0400
Subject: Connection API?
In-Reply-To: <20121203183756.11b51d11@baldur>
References: <20121201001936.283a7d4a@baldur>
 <20121203144248.GY40452@mdounin.ru>
 <20121203155724.094c4d52@baldur>
 <20121203173152.GA40452@mdounin.ru>
 <20121203183756.11b51d11@baldur>
Message-ID: <20121203190613.GE40452@mdounin.ru>

Hello!

On Mon, Dec 03, 2012 at 06:37:56PM +0000, Nick Kew wrote:

> On Mon, 3 Dec 2012 21:31:52 +0400
> Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> > > Thanks for the reply.  You mean you're happy to add a new-connection
> > > hook, but not a context?
> > 
> > Something like this.  I would like to see use case first though, 
> > to make sure it indeed requires connection start hook.
> 
> The basic use case is for a module that hooks in a library
> that needs to be notified of new connections.
> 
> There are a couple of reasons for this.  One is that it has
> its own connection struct that needs to be initialised before
> it can process requests on the connection.  To do that
> per-request would require a lookup table of existing
> connections, and a mutex to prevent a race condition
> between requests on a connection.  In other words, quite
> a lot of extra complexity.

You don't need mutext as there are no threads.  And you don't need 
a lookup table at all as long a you'll follow already suggested 
way to store per-connection data.  That is, this part doesn't 
seems to be relevant.

> The other reason is, this is a security tool, and we'd like
> to know about any connections that are NOT followed by a
> valid (and notifiable) HTTP request, and might be part of
> an attack on a server.

This one looks better.

> > As of now its always true.  Note though, that with SPDY patches 
> > Valentin is working on it will likely change.  I've just talked to 
> > him and he suggests to compare r[12]->connection->fd instead as a 
> > more bulletproof solution.
> 
> Surely an fd will be recycled/re-used?

Much like r[12]->connection, which is reused as well.  You 
never asked if r1->connection == r2->connection means r1 and r2 
are on the same connection, and obviously the answer is no.

> OK, I can work around that, too: just a little extra work to re-use
> the library's connection initialisation to generate a key.
> Though it means holding a mutex rather longer!

There is already r->connection->number which is incremented 
each time new connection is accepted, and it's highly unlikely 
that two distinct connections to have the same fd and number.

-- 
Maxim Dounin
http://nginx.com/support.html


From mdounin at mdounin.ru  Mon Dec  3 19:16:54 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 3 Dec 2012 23:16:54 +0400
Subject: ngx_chain_get_free_buf crashes with free=0 after return
 next_body_filter
In-Reply-To: <CAG6Cn5Yawu2Yz0RHyC93fbGS-0zSWEk7BrZnH+1+OVPKS5jqEg@mail.gmail.com>
References: <CAG6Cn5bHnj_fV4EErrj1oogqfCw=7kmdtHiJr-p1C81=SgY8OA@mail.gmail.com>
 <20121203180949.GB40452@mdounin.ru>
 <CAG6Cn5Yawu2Yz0RHyC93fbGS-0zSWEk7BrZnH+1+OVPKS5jqEg@mail.gmail.com>
Message-ID: <20121203191654.GF40452@mdounin.ru>

Hello!

On Mon, Dec 03, 2012 at 10:34:22PM +0400, Ruslan Khusnullin wrote:

> On Mon, Dec 3, 2012 at 10:09 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> > Note: returning NGX_HTTP_INTERNAL_SERVER_ERROR is completely wrong
> > here, and it will result in undefined behaviour if ever returned.
> >
> > You should either return NGX_ERROR here (probably logging an alert
> > message) or just assume it should never happen unless process
> > memory is corrupted and just dereference the pointer (this way
> > you'll get proper segmentation fault instead of undefined
> > behviour).
> 
> Thank you for this, I have replaced all NGX_HTTP_INTERNAL_SERVER_ERROR
> with NGX_ERROR in body_filter and now it doesn't crash.
> 
> So is returning NGX_HTTP_*_ERROR valid only in a handler function?

Yes.

[...]

> Could you please also comment below piece of code? I've got it from
> previous coders and I'm not sure it needs to be there. It is a version
> of ngx_chain_add_copy that really copies data instead of copying
> pointers. Does it make sense? The module copies all chunks from
> upstream cache with this function and then when meeting buf->last_buf,
> it composes whole response in one memory block and modifies it. I
> guess that all the chunks remain in memory for all request time or I
> really need to make a copy of data in my context?

In general you have to make a copy if you need buffer data in 
memory longer than a single buffer invocation.  E.g. with

    output_buffers 1 4k;

you won't be able process files bigger than 4k if you'll keep 
buffers busy till request end - as there is only one buffer 
configured to read a file from disk and serve it to a client.

In most cases it's possible to do something better than just a 
dumb copy of all buffers (e.g. one may want to short-circut 
situation where the last buffer is readily available), but in a 
worst case copy is unavoidable (unless you'll change your code to 
do proper stream processing).

-- 
Maxim Dounin
http://nginx.com/support.html


From ibobrik at gmail.com  Tue Dec  4 05:35:07 2012
From: ibobrik at gmail.com (ivan babrou)
Date: Tue, 4 Dec 2012 09:35:07 +0400
Subject: image_filter enhancement
In-Reply-To: <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>
References: <CANWdNRDP_6+WHtCawX8KnoH4GFHHQeHC2tXY6-8gDctYq0Dnrg@mail.gmail.com>
 <20121106181601.GS40452@mdounin.ru>
 <CANWdNRDYuMDZOpcEwtsryfUpTBCsdB6VsSJfF+qUJV34y5j08g@mail.gmail.com>
 <20121114234732.GC40452@mdounin.ru>
 <CANWdNRB=bRCEjEBM+ksak8c6pEZw+VC1XfKBrPLEJ5zXv+B_=g@mail.gmail.com>
 <20121127160328.GZ40452@mdounin.ru>
 <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>
Message-ID: <CANWdNRBXuGyFdo1NGUeHba2QnSYokMJpD61tW-VNcOo=TV-9gg@mail.gmail.com>

Renamed oxv and oyv to oxcv and oycv too.

diff --git a/ngx_http_image_filter_module.c b/ngx_http_image_filter_module.c
index c853c33..325140e 100644
--- a/ngx_http_image_filter_module.c
+++ b/ngx_http_image_filter_module.c
@@ -32,6 +32,11 @@
 #define NGX_HTTP_IMAGE_GIF       2
 #define NGX_HTTP_IMAGE_PNG       3

+#define NGX_HTTP_IMAGE_OFFSET_CENTER    0
+#define NGX_HTTP_IMAGE_OFFSET_LEFT      1
+#define NGX_HTTP_IMAGE_OFFSET_RIGHT     2
+#define NGX_HTTP_IMAGE_OFFSET_TOP       3
+#define NGX_HTTP_IMAGE_OFFSET_BOTTOM    4

 #define NGX_HTTP_IMAGE_BUFFERED  0x08

@@ -43,11 +48,15 @@ typedef struct {
     ngx_uint_t                   angle;
     ngx_uint_t                   jpeg_quality;
     ngx_uint_t                   sharpen;
+    ngx_uint_t                   crop_offset_x;
+    ngx_uint_t                   crop_offset_y;

     ngx_flag_t                   transparency;

     ngx_http_complex_value_t    *wcv;
     ngx_http_complex_value_t    *hcv;
+    ngx_http_complex_value_t    *oxcv;
+    ngx_http_complex_value_t    *oycv;
     ngx_http_complex_value_t    *acv;
     ngx_http_complex_value_t    *jqcv;
     ngx_http_complex_value_t    *shcv;
@@ -66,6 +75,8 @@ typedef struct {
     ngx_uint_t                   height;
     ngx_uint_t                   max_width;
     ngx_uint_t                   max_height;
+    ngx_uint_t                   crop_offset_x;
+    ngx_uint_t                   crop_offset_y;
     ngx_uint_t                   angle;

     ngx_uint_t                   phase;
@@ -110,6 +121,8 @@ static char
*ngx_http_image_filter_jpeg_quality(ngx_conf_t *cf,
     ngx_command_t *cmd, void *conf);
 static char *ngx_http_image_filter_sharpen(ngx_conf_t *cf, ngx_command_t *cmd,
     void *conf);
+static char *ngx_http_image_filter_offset(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf);
 static ngx_int_t ngx_http_image_filter_init(ngx_conf_t *cf);


@@ -150,6 +163,13 @@ static ngx_command_t  ngx_http_image_filter_commands[] = {
       offsetof(ngx_http_image_filter_conf_t, buffer_size),
       NULL },

+    { ngx_string("image_filter_crop_offset"),
+      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
+      ngx_http_image_filter_offset,
+      NGX_HTTP_LOC_CONF_OFFSET,
+      0,
+      NULL },
+
       ngx_null_command
 };

@@ -737,7 +757,8 @@ ngx_http_image_resize(ngx_http_request_t *r,
ngx_http_image_filter_ctx_t *ctx)
 {
     int                            sx, sy, dx, dy, ox, oy, ax, ay, size,
                                    colors, palette, transparent, sharpen,
-                                   red, green, blue, t;
+                                   red, green, blue, t,
+                                   crop_offset_x, crop_offset_y;
     u_char                        *out;
     ngx_buf_t                     *b;
     ngx_uint_t                     resize;
@@ -932,8 +953,27 @@ transparent:
                 return NULL;
             }

-            ox /= 2;
-            oy /= 2;
+            crop_offset_x = conf->crop_offset_x;
+            crop_offset_y = conf->crop_offset_y;
+
+            crop_offset_x = ngx_http_image_filter_get_value(r, conf->oxcv,
+                                                            crop_offset_x);
+            crop_offset_y = ngx_http_image_filter_get_value(r, conf->oycv,
+                                                            crop_offset_y);
+
+            if (crop_offset_x == NGX_HTTP_IMAGE_OFFSET_LEFT) {
+                ox = 0;
+
+            } else if (crop_offset_x == NGX_HTTP_IMAGE_OFFSET_CENTER) {
+                ox /= 2;
+            }
+
+            if (crop_offset_y == NGX_HTTP_IMAGE_OFFSET_TOP) {
+                oy = 0;
+
+            } else if (crop_offset_y == NGX_HTTP_IMAGE_OFFSET_CENTER) {
+                oy /= 2;
+            }

             ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                            "image crop: %d x %d @ %d x %d",
@@ -1149,10 +1189,27 @@ ngx_http_image_filter_value(ngx_str_t *value)
         return (ngx_uint_t) -1;
     }

-    n = ngx_atoi(value->data, value->len);
+    if (ngx_strcmp(value->data, "center") == 0) {
+        return NGX_HTTP_IMAGE_OFFSET_CENTER;
+
+    } else if (ngx_strcmp(value->data, "left") == 0) {
+        return NGX_HTTP_IMAGE_OFFSET_LEFT;
+
+    } else if (ngx_strcmp(value->data, "right") == 0) {
+        return NGX_HTTP_IMAGE_OFFSET_RIGHT;
+
+    } else if (ngx_strcmp(value->data, "top") == 0) {
+        return NGX_HTTP_IMAGE_OFFSET_TOP;

-    if (n > 0) {
-        return (ngx_uint_t) n;
+    } else if (ngx_strcmp(value->data, "bottom") == 0) {
+        return NGX_HTTP_IMAGE_OFFSET_BOTTOM;
+
+    } else {
+        n = ngx_atoi(value->data, value->len);
+
+        if (n > 0) {
+            return (ngx_uint_t) n;
+        }
     }

     return 0;
@@ -1175,6 +1232,8 @@ ngx_http_image_filter_create_conf(ngx_conf_t *cf)
     conf->angle = NGX_CONF_UNSET_UINT;
     conf->transparency = NGX_CONF_UNSET;
     conf->buffer_size = NGX_CONF_UNSET_SIZE;
+    conf->crop_offset_x = NGX_CONF_UNSET_UINT;
+    conf->crop_offset_y = NGX_CONF_UNSET_UINT;

     return conf;
 }
@@ -1223,6 +1282,17 @@ ngx_http_image_filter_merge_conf(ngx_conf_t
*cf, void *parent, void *child)
     ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size,
                               1 * 1024 * 1024);

+    ngx_conf_merge_uint_value(conf->crop_offset_x,
prev->crop_offset_x, NGX_HTTP_IMAGE_OFFSET_CENTER);
+    ngx_conf_merge_uint_value(conf->crop_offset_y,
prev->crop_offset_y, NGX_HTTP_IMAGE_OFFSET_CENTER);
+
+    if (conf->oxcv == NULL) {
+        conf->oxcv = prev->oxcv;
+    }
+
+    if (conf->oycv == NULL) {
+        conf->oycv = prev->oycv;
+    }
+
     return NGX_CONF_OK;
 }

@@ -1474,6 +1544,68 @@ ngx_http_image_filter_sharpen(ngx_conf_t *cf,
ngx_command_t *cmd,
 }


+static char *
+ngx_http_image_filter_offset(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf)
+{
+    ngx_http_image_filter_conf_t *imcf = conf;
+
+    ngx_str_t                         *value;
+    ngx_http_complex_value_t           cv;
+    ngx_http_compile_complex_value_t   ccv;
+
+    value = cf->args->elts;
+
+    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
+
+    ccv.cf = cf;
+    ccv.value = &value[1];
+    ccv.complex_value = &cv;
+    ccv.zero = 1;
+
+    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
+        return NGX_CONF_ERROR;
+    }
+
+    if (cv.lengths == NULL) {
+        imcf->crop_offset_x = ngx_http_image_filter_value(&value[1]);
+
+    } else {
+        imcf->oxcv = ngx_palloc(cf->pool, sizeof(ngx_http_complex_value_t));
+        if (imcf->oxcv == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        *imcf->oxcv = cv;
+    }
+
+    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
+
+    ccv.cf = cf;
+    ccv.value = &value[2];
+    ccv.complex_value = &cv;
+    ccv.zero = 1;
+
+    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
+        return NGX_CONF_ERROR;
+    }
+
+    if (cv.lengths == NULL) {
+        imcf->crop_offset_y = ngx_http_image_filter_value(&value[2]);
+
+    } else {
+        imcf->oycv = ngx_palloc(cf->pool, sizeof(ngx_http_complex_value_t));
+        if (imcf->oycv == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        *imcf->oycv = cv;
+    }
+
+    return NGX_CONF_OK;
+}
+
+
 static ngx_int_t
 ngx_http_image_filter_init(ngx_conf_t *cf)
 {

On 3 December 2012 21:55, ivan babrou <ibobrik at gmail.com> wrote:
> Reworked again. Looks much better now! :)
>
> I renamed configuration directive to image_filter_crop_offset.
>
> diff --git a/ngx_http_image_filter_module.c b/ngx_http_image_filter_module.c
> index c853c33..4d0ce95 100644
> --- a/ngx_http_image_filter_module.c
> +++ b/ngx_http_image_filter_module.c
> @@ -32,6 +32,11 @@
>  #define NGX_HTTP_IMAGE_GIF       2
>  #define NGX_HTTP_IMAGE_PNG       3
>
> +#define NGX_HTTP_IMAGE_OFFSET_CENTER    0
> +#define NGX_HTTP_IMAGE_OFFSET_LEFT      1
> +#define NGX_HTTP_IMAGE_OFFSET_RIGHT     2
> +#define NGX_HTTP_IMAGE_OFFSET_TOP       3
> +#define NGX_HTTP_IMAGE_OFFSET_BOTTOM    4
>
>  #define NGX_HTTP_IMAGE_BUFFERED  0x08
>
> @@ -43,11 +48,15 @@ typedef struct {
>      ngx_uint_t                   angle;
>      ngx_uint_t                   jpeg_quality;
>      ngx_uint_t                   sharpen;
> +    ngx_uint_t                   crop_offset_x;
> +    ngx_uint_t                   crop_offset_y;
>
>      ngx_flag_t                   transparency;
>
>      ngx_http_complex_value_t    *wcv;
>      ngx_http_complex_value_t    *hcv;
> +    ngx_http_complex_value_t    *oxv;
> +    ngx_http_complex_value_t    *oyv;
>      ngx_http_complex_value_t    *acv;
>      ngx_http_complex_value_t    *jqcv;
>      ngx_http_complex_value_t    *shcv;
> @@ -66,6 +75,8 @@ typedef struct {
>      ngx_uint_t                   height;
>      ngx_uint_t                   max_width;
>      ngx_uint_t                   max_height;
> +    ngx_uint_t                   crop_offset_x;
> +    ngx_uint_t                   crop_offset_y;
>      ngx_uint_t                   angle;
>
>      ngx_uint_t                   phase;
> @@ -110,6 +121,8 @@ static char
> *ngx_http_image_filter_jpeg_quality(ngx_conf_t *cf,
>      ngx_command_t *cmd, void *conf);
>  static char *ngx_http_image_filter_sharpen(ngx_conf_t *cf, ngx_command_t *cmd,
>      void *conf);
> +static char *ngx_http_image_filter_offset(ngx_conf_t *cf, ngx_command_t *cmd,
> +    void *conf);
>  static ngx_int_t ngx_http_image_filter_init(ngx_conf_t *cf);
>
>
> @@ -150,6 +163,13 @@ static ngx_command_t  ngx_http_image_filter_commands[] = {
>        offsetof(ngx_http_image_filter_conf_t, buffer_size),
>        NULL },
>
> +    { ngx_string("image_filter_crop_offset"),
> +      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
> +      ngx_http_image_filter_offset,
> +      NGX_HTTP_LOC_CONF_OFFSET,
> +      0,
> +      NULL },
> +
>        ngx_null_command
>  };
>
> @@ -737,7 +757,8 @@ ngx_http_image_resize(ngx_http_request_t *r,
> ngx_http_image_filter_ctx_t *ctx)
>  {
>      int                            sx, sy, dx, dy, ox, oy, ax, ay, size,
>                                     colors, palette, transparent, sharpen,
> -                                   red, green, blue, t;
> +                                   red, green, blue, t,
> +                                   crop_offset_x, crop_offset_y;
>      u_char                        *out;
>      ngx_buf_t                     *b;
>      ngx_uint_t                     resize;
> @@ -932,8 +953,27 @@ transparent:
>                  return NULL;
>              }
>
> -            ox /= 2;
> -            oy /= 2;
> +            crop_offset_x = conf->crop_offset_x;
> +            crop_offset_y = conf->crop_offset_y;
> +
> +            crop_offset_x = ngx_http_image_filter_get_value(r, conf->oxv,
> +                                                            crop_offset_x);
> +            crop_offset_y = ngx_http_image_filter_get_value(r, conf->oyv,
> +                                                            crop_offset_y);
> +
> +            if (crop_offset_x == NGX_HTTP_IMAGE_OFFSET_LEFT) {
> +                ox = 0;
> +
> +            } else if (crop_offset_x == NGX_HTTP_IMAGE_OFFSET_CENTER) {
> +                ox /= 2;
> +            }
> +
> +            if (crop_offset_y == NGX_HTTP_IMAGE_OFFSET_TOP) {
> +                oy = 0;
> +
> +            } else if (crop_offset_y == NGX_HTTP_IMAGE_OFFSET_CENTER) {
> +                oy /= 2;
> +            }
>
>              ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
>                             "image crop: %d x %d @ %d x %d",
> @@ -1149,10 +1189,27 @@ ngx_http_image_filter_value(ngx_str_t *value)
>          return (ngx_uint_t) -1;
>      }
>
> -    n = ngx_atoi(value->data, value->len);
> +    if (ngx_strcmp(value->data, "center") == 0) {
> +        return NGX_HTTP_IMAGE_OFFSET_CENTER;
> +
> +    } else if (ngx_strcmp(value->data, "left") == 0) {
> +        return NGX_HTTP_IMAGE_OFFSET_LEFT;
> +
> +    } else if (ngx_strcmp(value->data, "right") == 0) {
> +        return NGX_HTTP_IMAGE_OFFSET_RIGHT;
> +
> +    } else if (ngx_strcmp(value->data, "top") == 0) {
> +        return NGX_HTTP_IMAGE_OFFSET_TOP;
>
> -    if (n > 0) {
> -        return (ngx_uint_t) n;
> +    } else if (ngx_strcmp(value->data, "bottom") == 0) {
> +        return NGX_HTTP_IMAGE_OFFSET_BOTTOM;
> +
> +    } else {
> +        n = ngx_atoi(value->data, value->len);
> +
> +        if (n > 0) {
> +            return (ngx_uint_t) n;
> +        }
>      }
>
>      return 0;
> @@ -1175,6 +1232,8 @@ ngx_http_image_filter_create_conf(ngx_conf_t *cf)
>      conf->angle = NGX_CONF_UNSET_UINT;
>      conf->transparency = NGX_CONF_UNSET;
>      conf->buffer_size = NGX_CONF_UNSET_SIZE;
> +    conf->crop_offset_x = NGX_CONF_UNSET_UINT;
> +    conf->crop_offset_y = NGX_CONF_UNSET_UINT;
>
>      return conf;
>  }
> @@ -1223,6 +1282,17 @@ ngx_http_image_filter_merge_conf(ngx_conf_t
> *cf, void *parent, void *child)
>      ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size,
>                                1 * 1024 * 1024);
>
> +    ngx_conf_merge_uint_value(conf->crop_offset_x,
> prev->crop_offset_x, NGX_HTTP_IMAGE_OFFSET_CENTER);
> +    ngx_conf_merge_uint_value(conf->crop_offset_y,
> prev->crop_offset_y, NGX_HTTP_IMAGE_OFFSET_CENTER);
> +
> +    if (conf->oxv == NULL) {
> +        conf->oxv = prev->oxv;
> +    }
> +
> +    if (conf->oyv == NULL) {
> +        conf->oyv = prev->oyv;
> +    }
> +
>      return NGX_CONF_OK;
>  }
>
> @@ -1474,6 +1544,68 @@ ngx_http_image_filter_sharpen(ngx_conf_t *cf,
> ngx_command_t *cmd,
>  }
>
>
> +static char *
> +ngx_http_image_filter_offset(ngx_conf_t *cf, ngx_command_t *cmd,
> +    void *conf)
> +{
> +    ngx_http_image_filter_conf_t *imcf = conf;
> +
> +    ngx_str_t                         *value;
> +    ngx_http_complex_value_t           cv;
> +    ngx_http_compile_complex_value_t   ccv;
> +
> +    value = cf->args->elts;
> +
> +    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
> +
> +    ccv.cf = cf;
> +    ccv.value = &value[1];
> +    ccv.complex_value = &cv;
> +    ccv.zero = 1;
> +
> +    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
> +        return NGX_CONF_ERROR;
> +    }
> +
> +    if (cv.lengths == NULL) {
> +        imcf->crop_offset_x = ngx_http_image_filter_value(&value[1]);
> +
> +    } else {
> +        imcf->oxv = ngx_palloc(cf->pool, sizeof(ngx_http_complex_value_t));
> +        if (imcf->oxv == NULL) {
> +            return NGX_CONF_ERROR;
> +        }
> +
> +        *imcf->oxv = cv;
> +    }
> +
> +    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
> +
> +    ccv.cf = cf;
> +    ccv.value = &value[2];
> +    ccv.complex_value = &cv;
> +    ccv.zero = 1;
> +
> +    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
> +        return NGX_CONF_ERROR;
> +    }
> +
> +    if (cv.lengths == NULL) {
> +        imcf->crop_offset_y = ngx_http_image_filter_value(&value[2]);
> +
> +    } else {
> +        imcf->oyv = ngx_palloc(cf->pool, sizeof(ngx_http_complex_value_t));
> +        if (imcf->oyv == NULL) {
> +            return NGX_CONF_ERROR;
> +        }
> +
> +        *imcf->oyv = cv;
> +    }
> +
> +    return NGX_CONF_OK;
> +}
> +
> +
>  static ngx_int_t
>  ngx_http_image_filter_init(ngx_conf_t *cf)
>  {
>
> On 27 November 2012 20:03, Maxim Dounin <mdounin at mdounin.ru> wrote:
>> Hello!
>>
>> On Tue, Nov 27, 2012 at 12:10:13AM +0400, ivan babrou wrote:
>>
>>> I tried to rework patch. Now it supports configuration through
>>> variables. Variables ox and oy renamed to crop_offset_x and
>>> crop_offset_y.
>>>
>>> I'm confused about this line:
>>> +    value.data[value.len] = 0;
>>>
>>> Probably I don't understand something about memory management in nginx.
>>
>> See below.
>>
>>>
>>>
>>> diff --git a/ngx_http_image_filter_module.c b/ngx_http_image_filter_module.c
>>> index c853c33..15f8d17 100644
>>> --- a/ngx_http_image_filter_module.c
>>> +++ b/ngx_http_image_filter_module.c
>>> @@ -32,6 +32,14 @@
>>>  #define NGX_HTTP_IMAGE_GIF       2
>>>  #define NGX_HTTP_IMAGE_PNG       3
>>>
>>> +#define NGX_HTTP_IMAGE_OFFSET_TYPE_HORIZONTAL 0
>>> +#define NGX_HTTP_IMAGE_OFFSET_TYPE_VERTICAL 1
>>> +
>>> +#define NGX_HTTP_IMAGE_OFFSET_CENTER    0
>>> +#define NGX_HTTP_IMAGE_OFFSET_LEFT      1
>>> +#define NGX_HTTP_IMAGE_OFFSET_RIGHT     2
>>> +#define NGX_HTTP_IMAGE_OFFSET_TOP       3
>>> +#define NGX_HTTP_IMAGE_OFFSET_BOTTOM    4
>>>
>>>  #define NGX_HTTP_IMAGE_BUFFERED  0x08
>>>
>>> @@ -43,11 +51,15 @@ typedef struct {
>>>      ngx_uint_t                   angle;
>>>      ngx_uint_t                   jpeg_quality;
>>>      ngx_uint_t                   sharpen;
>>> +    ngx_uint_t                   crop_offset_x;
>>> +    ngx_uint_t                   crop_offset_y;
>>>
>>>      ngx_flag_t                   transparency;
>>>
>>>      ngx_http_complex_value_t    *wcv;
>>>      ngx_http_complex_value_t    *hcv;
>>> +    ngx_http_complex_value_t    *oxv;
>>> +    ngx_http_complex_value_t    *oyv;
>>
>> Probably you mean "oxcv", "oycv" - "cv" here is abbreviation of
>> "complex value".
>>
>>>      ngx_http_complex_value_t    *acv;
>>>      ngx_http_complex_value_t    *jqcv;
>>>      ngx_http_complex_value_t    *shcv;
>>> @@ -66,6 +78,8 @@ typedef struct {
>>>      ngx_uint_t                   height;
>>>      ngx_uint_t                   max_width;
>>>      ngx_uint_t                   max_height;
>>> +    ngx_uint_t                   crop_offset_x;
>>> +    ngx_uint_t                   crop_offset_y;
>>>      ngx_uint_t                   angle;
>>>
>>>      ngx_uint_t                   phase;
>>> @@ -98,6 +112,15 @@ static u_char
>>> *ngx_http_image_out(ngx_http_request_t *r, ngx_uint_t type,
>>>  static void ngx_http_image_cleanup(void *data);
>>>  static ngx_uint_t ngx_http_image_filter_get_value(ngx_http_request_t *r,
>>>      ngx_http_complex_value_t *cv, ngx_uint_t v);
>>> +static void
>>> +ngx_http_image_filter_crop_offset(ngx_http_request_t *r,
>>> +    ngx_http_complex_value_t *cv, ngx_uint_t t, ngx_uint_t *v);
>>> +static void
>>> +ngx_http_image_filter_vertical_crop_offset(ngx_str_t *value,
>>> +    ngx_uint_t *v);
>>> +static void
>>> +ngx_http_image_filter_horizontal_crop_offset(ngx_str_t *value,
>>> +    ngx_uint_t *v);
>>>  static ngx_uint_t ngx_http_image_filter_value(ngx_str_t *value);
>>>
>>>
>>> @@ -110,6 +133,8 @@ static char
>>> *ngx_http_image_filter_jpeg_quality(ngx_conf_t *cf,
>>>      ngx_command_t *cmd, void *conf);
>>>  static char *ngx_http_image_filter_sharpen(ngx_conf_t *cf, ngx_command_t *cmd,
>>>      void *conf);
>>> +static char *ngx_http_image_filter_offset(ngx_conf_t *cf, ngx_command_t *cmd,
>>> +    void *conf);
>>>  static ngx_int_t ngx_http_image_filter_init(ngx_conf_t *cf);
>>>
>>>
>>> @@ -150,6 +175,13 @@ static ngx_command_t  ngx_http_image_filter_commands[] = {
>>>        offsetof(ngx_http_image_filter_conf_t, buffer_size),
>>>        NULL },
>>>
>>> +    { ngx_string("image_filter_offset"),
>>> +      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
>>> +      ngx_http_image_filter_offset,
>>> +      NGX_HTTP_LOC_CONF_OFFSET,
>>> +      0,
>>> +      NULL },
>>> +
>>>        ngx_null_command
>>>  };
>>>
>>> @@ -529,6 +561,22 @@ ngx_http_image_process(ngx_http_request_t *r)
>>>          return NULL;
>>>      }
>>>
>>> +    if (conf->oxv == NULL) {
>>> +        ctx->crop_offset_x = conf->crop_offset_x;
>>> +    } else {
>>> +        ngx_http_image_filter_crop_offset(r, conf->oxv,
>>> +
>>> NGX_HTTP_IMAGE_OFFSET_TYPE_HORIZONTAL,
>>> +                                          &ctx->crop_offset_x);
>>> +    }
>>> +
>>> +    if (conf->oyv == NULL) {
>>> +        ctx->crop_offset_y = conf->crop_offset_y;
>>> +    } else {
>>> +        ngx_http_image_filter_crop_offset(r, conf->oyv,
>>> +                                          NGX_HTTP_IMAGE_OFFSET_TYPE_VERTICAL,
>>> +                                          &ctx->crop_offset_y);
>>> +    }
>>> +
>>
>> 1) You want to follow pattern in the previous code, i.e.
>>    foo = ngx_http_image_filter_get_value(r, conf->foocv, conf->foo);
>>
>>    (With additional support of top/bottom/left/right literals.)
>>
>> 2) I don't think you need to calculate offsets to decide whether
>>    image is ok as is or we need to resize it.  Hence it's should
>>    be ok to move the code to where it's needed.
>>
>> 3) With (2) fixed you probably don't need variables in ctx
>>    anymore.
>>
>>>      if (rc == NGX_OK
>>>          && ctx->width <= ctx->max_width
>>>          && ctx->height <= ctx->max_height
>>> @@ -932,8 +980,17 @@ transparent:
>>>                  return NULL;
>>>              }
>>>
>>> -            ox /= 2;
>>> -            oy /= 2;
>>> +            if (ctx->crop_offset_x == NGX_HTTP_IMAGE_OFFSET_LEFT) {
>>> +                ox = 0;
>>> +            } else if (ctx->crop_offset_x == NGX_HTTP_IMAGE_OFFSET_CENTER) {
>>> +                ox /= 2;
>>> +            }
>>> +
>>> +            if (ctx->crop_offset_y == NGX_HTTP_IMAGE_OFFSET_TOP) {
>>> +                oy = 0;
>>> +            } else if (ctx->crop_offset_y == NGX_HTTP_IMAGE_OFFSET_CENTER) {
>>> +                oy /= 2;
>>> +            }
>>
>> Note: this needs empty lines before "} else if ..." to match
>> style.  It should be changed anyway though, due to reasons
>> outlined above.
>>
>>>
>>>              ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
>>>                             "image crop: %d x %d @ %d x %d",
>>> @@ -1139,6 +1196,52 @@ ngx_http_image_filter_get_value(ngx_http_request_t *r,
>>>      return ngx_http_image_filter_value(&val);
>>>  }
>>>
>>> +static void
>>> +ngx_http_image_filter_crop_offset(ngx_http_request_t *r,
>>> +    ngx_http_complex_value_t *cv, ngx_uint_t t, ngx_uint_t *v)
>>> +{
>>> +    ngx_str_t value;
>>> +
>>> +    if (ngx_http_complex_value(r, cv, &value) != NGX_OK) {
>>> +        return;
>>> +    }
>>> +
>>> +    value.data[value.len] = 0;
>>
>> This is incorrect and will result in memory corruption.
>>
>> If you need null-terminated string, right aproach is to set
>> ccv.zero flag while calling ngx_http_compile_complex_value().  On
>> the other hand, it might be good idea to don't assume
>> null-termination instead.
>>
>>> +
>>> +    if (t == NGX_HTTP_IMAGE_OFFSET_TYPE_HORIZONTAL) {
>>> +        ngx_http_image_filter_horizontal_crop_offset(&value, v);
>>> +    } else if (t == NGX_HTTP_IMAGE_OFFSET_TYPE_VERTICAL) {
>>> +        ngx_http_image_filter_vertical_crop_offset(&value, v);
>>> +    }
>>> +}
>>> +
>>> +
>>> +static void
>>> +ngx_http_image_filter_horizontal_crop_offset(ngx_str_t *value,
>>> +    ngx_uint_t *v)
>>> +{
>>> +    if (ngx_strcmp(value->data, "center") == 0) {
>>> +        *v = NGX_HTTP_IMAGE_OFFSET_CENTER;
>>> +    } else if (ngx_strcmp(value->data, "left") == 0) {
>>> +        *v = NGX_HTTP_IMAGE_OFFSET_LEFT;
>>> +    } else if (ngx_strcmp(value->data, "right") == 0) {
>>> +        *v = NGX_HTTP_IMAGE_OFFSET_RIGHT;
>>> +    }
>>> +}
>>> +
>>> +static void
>>> +ngx_http_image_filter_vertical_crop_offset(ngx_str_t *value,
>>> +    ngx_uint_t *v)
>>> +{
>>> +    if (ngx_strcmp(value->data, "center") == 0) {
>>> +        *v = NGX_HTTP_IMAGE_OFFSET_CENTER;
>>> +    } else if (ngx_strcmp(value->data, "top") == 0) {
>>> +        *v = NGX_HTTP_IMAGE_OFFSET_TOP;
>>> +    } else if (ngx_strcmp(value->data, "bottom") == 0) {
>>> +        *v = NGX_HTTP_IMAGE_OFFSET_BOTTOM;
>>> +    }
>>> +}
>>> +
>>
>> I don't think we need so many functions to do this simple task.
>> It would be good idea simplify this.
>>
>> The NGX_HTTP_IMAGE_OFFSET_TYPE_HORIZONTAL and
>> NGX_HTTP_IMAGE_OFFSET_TYPE_VERTICAL seem to be unneeded, just a
>> boolean value would do the trick.
>>
>>>  static ngx_uint_t
>>>  ngx_http_image_filter_value(ngx_str_t *value)
>>> @@ -1175,6 +1278,8 @@ ngx_http_image_filter_create_conf(ngx_conf_t *cf)
>>>      conf->angle = NGX_CONF_UNSET_UINT;
>>>      conf->transparency = NGX_CONF_UNSET;
>>>      conf->buffer_size = NGX_CONF_UNSET_SIZE;
>>> +    conf->crop_offset_x = NGX_CONF_UNSET_UINT;
>>> +    conf->crop_offset_y = NGX_CONF_UNSET_UINT;
>>>
>>>      return conf;
>>>  }
>>> @@ -1223,6 +1328,17 @@ ngx_http_image_filter_merge_conf(ngx_conf_t
>>> *cf, void *parent, void *child)
>>>      ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size,
>>>                                1 * 1024 * 1024);
>>>
>>> +    ngx_conf_merge_uint_value(conf->crop_offset_x,
>>> prev->crop_offset_x, NGX_HTTP_IMAGE_OFFSET_CENTER);
>>> +    ngx_conf_merge_uint_value(conf->crop_offset_y,
>>> prev->crop_offset_y, NGX_HTTP_IMAGE_OFFSET_CENTER);
>>> +
>>> +    if (conf->oxv == NULL) {
>>> +        conf->oxv = prev->oxv;
>>> +    }
>>> +
>>> +    if (conf->oyv == NULL) {
>>> +        conf->oyv = prev->oyv;
>>> +    }
>>> +
>>
>> Style: please keep lines shorter than 80 chars.
>>
>> Additionally, looking at the code I tend to think it's incorrect.
>> That is, it's pattern you've followed is incorrect, not your code.
>> E.g. the following config will result in $arg_q incorrectly used
>> for quality in /image/, instead of "80" explicitly set:
>>
>>     image_filter_jpeg_quality $arg_q;
>>
>>     location /image/ {
>>         image_filter crop $arg_w $arg_h;
>>         image_filter_jpeg_quality 80;
>>     }
>>
>> This needs fixing.
>>
>>>      return NGX_CONF_OK;
>>>  }
>>>
>>> @@ -1474,6 +1590,64 @@ ngx_http_image_filter_sharpen(ngx_conf_t *cf,
>>> ngx_command_t *cmd,
>>>  }
>>>
>>>
>>> +static char *
>>> +ngx_http_image_filter_offset(ngx_conf_t *cf, ngx_command_t *cmd,
>>> +    void *conf)
>>> +{
>>> +    ngx_http_image_filter_conf_t *imcf = conf;
>>> +
>>> +    ngx_str_t                         *value;
>>> +    ngx_http_complex_value_t           cv;
>>> +    ngx_http_compile_complex_value_t   ccv;
>>> +
>>> +    value = cf->args->elts;
>>> +
>>> +    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
>>> +
>>> +    ccv.cf = cf;
>>> +    ccv.value = &value[1];
>>> +    ccv.complex_value = &cv;
>>> +
>>> +    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
>>> +        return NGX_CONF_ERROR;
>>> +    }
>>> +
>>> +    if (cv.lengths == NULL) {
>>> +        ngx_http_image_filter_horizontal_crop_offset(&value[1],
>>> &imcf->crop_offset_x);
>>> +    } else {
>>> +        imcf->oxv = ngx_palloc(cf->pool, sizeof(ngx_http_complex_value_t));
>>> +        if (imcf->oxv == NULL) {
>>> +            return NGX_CONF_ERROR;
>>> +        }
>>> +
>>> +        *imcf->oxv = cv;
>>> +    }
>>> +
>>> +    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
>>> +
>>> +    ccv.cf = cf;
>>> +    ccv.value = &value[2];
>>> +    ccv.complex_value = &cv;
>>> +
>>> +    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
>>> +        return NGX_CONF_ERROR;
>>> +    }
>>> +
>>> +    if (cv.lengths == NULL) {
>>> +        ngx_http_image_filter_vertical_crop_offset(&value[2],
>>> &imcf->crop_offset_y);
>>> +    } else {
>>> +        imcf->oyv = ngx_palloc(cf->pool, sizeof(ngx_http_complex_value_t));
>>> +        if (imcf->oyv == NULL) {
>>> +            return NGX_CONF_ERROR;
>>> +        }
>>> +
>>> +        *imcf->oyv = cv;
>>> +    }
>>> +
>>> +    return NGX_CONF_OK;
>>> +}
>>> +
>>> +
>>>  static ngx_int_t
>>>  ngx_http_image_filter_init(ngx_conf_t *cf)
>>>  {
>>>
>>>
>>> On 15 November 2012 03:47, Maxim Dounin <mdounin at mdounin.ru> wrote:
>>> > offsets via variables.  On the other hand, his patch asks for
>>>
>>>
>>>
>>> --
>>> Regards, Ian Babrou
>>> http://bobrik.name http://twitter.com/ibobrik skype:i.babrou
>>>
>>> _______________________________________________
>>> nginx-devel mailing list
>>> nginx-devel at nginx.org
>>> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>>
>> --
>> Maxim Dounin
>> http://nginx.com/support.html
>>
>> _______________________________________________
>> nginx-devel mailing list
>> nginx-devel at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
>
>
> --
> Regards, Ian Babrou
> http://bobrik.name http://twitter.com/ibobrik skype:i.babrou



-- 
Regards, Ian Babrou
http://bobrik.name http://twitter.com/ibobrik skype:i.babrou


From lawrence.pit at gmail.com  Tue Dec  4 05:57:32 2012
From: lawrence.pit at gmail.com (Lawrence Pit)
Date: Tue, 04 Dec 2012 16:57:32 +1100
Subject: nginx-devel Digest, Vol 38, Issue 2
In-Reply-To: <mailman.7.1354536001.47351.nginx-devel@nginx.org>
References: <mailman.7.1354536001.47351.nginx-devel@nginx.org>
Message-ID: <9A20C610-45EA-429A-88E8-7EC856DA2DC5@gmail.com>

>> After I finished this patch I checked the 1.3 branch, and noticed it
>> already has an $msec http variable defined. But it's output is with a
>> dot character before the milliseconds part. Why is that?
>
> http {
>  map $msec $s {
> 	~(?<x>.*)\. $x;
>  }
>
>  map $msec $ms {
> 	~\.(?<x>.*) $x;
>  }
>
>  server {
> 	return 200 "msec=$msec usec=${s}${ms}000\n";
>  }
> }
>
> $ curl http://localhost:8000/
> msec=1354471012.591 usec=1354471012591000

That's cool, impressive. Though ugly ;)  If it needs to do that for 
every request it's relatively expensive compared to what I'd rather do:

    proxy_set_header X-Queue-Start "${msec}000";

Is there any objection to not output the dot character starting from the 
1.3 release?

My reason for not using the dot character is that it seems like an 
arbitrary view concern to me and when the value is not used for logging 
purposes it's most likely to be used for calculations or conversions; in 
which case not having to parse out the dot character makes things 
simpler.


Cheers,
Lawrence


From ruslan.khusnullin at gmail.com  Tue Dec  4 09:18:24 2012
From: ruslan.khusnullin at gmail.com (Ruslan Khusnullin)
Date: Tue, 4 Dec 2012 13:18:24 +0400
Subject: ngx_chain_get_free_buf crashes with free=0 after return
 next_body_filter
In-Reply-To: <20121203191654.GF40452@mdounin.ru>
References: <CAG6Cn5bHnj_fV4EErrj1oogqfCw=7kmdtHiJr-p1C81=SgY8OA@mail.gmail.com>
 <20121203180949.GB40452@mdounin.ru>
 <CAG6Cn5Yawu2Yz0RHyC93fbGS-0zSWEk7BrZnH+1+OVPKS5jqEg@mail.gmail.com>
 <20121203191654.GF40452@mdounin.ru>
Message-ID: <CAG6Cn5ax8oBcjLEJ_GgUcGGB68UpVonRCnhPJ28vXS7ok4UFEw@mail.gmail.com>

Thanks, Maxim.

One more question: is there a way to install body/header filter only
for locations with directive instead of installing them for all
locations?


From ru at nginx.com  Tue Dec  4 10:06:25 2012
From: ru at nginx.com (Ruslan Ermilov)
Date: Tue, 4 Dec 2012 14:06:25 +0400
Subject: nginx-devel Digest, Vol 38, Issue 2
In-Reply-To: <9A20C610-45EA-429A-88E8-7EC856DA2DC5@gmail.com>
References: <mailman.7.1354536001.47351.nginx-devel@nginx.org>
 <9A20C610-45EA-429A-88E8-7EC856DA2DC5@gmail.com>
Message-ID: <20121204100625.GA54091@lo0.su>

On Tue, Dec 04, 2012 at 04:57:32PM +1100, Lawrence Pit wrote:
> >> After I finished this patch I checked the 1.3 branch, and noticed it
> >> already has an $msec http variable defined. But it's output is with a
> >> dot character before the milliseconds part. Why is that?
> >
> > http {
> >  map $msec $s {
> > 	~(?<x>.*)\. $x;
> >  }
> >
> >  map $msec $ms {
> > 	~\.(?<x>.*) $x;
> >  }
> >
> >  server {
> > 	return 200 "msec=$msec usec=${s}${ms}000\n";
> >  }
> > }
> >
> > $ curl http://localhost:8000/
> > msec=1354471012.591 usec=1354471012591000
> 
> That's cool, impressive. Though ugly ;)  If it needs to do that for 
> every request it's relatively expensive compared to what I'd rather do:
> 
>     proxy_set_header X-Queue-Start "${msec}000";
> 
> Is there any objection to not output the dot character starting from the 
> 1.3 release?

We absolutely don't want two different $msec variables, and $msec
was documented to be holding a value in seconds with a milliseconds
resolution for ages.

Log-only variables are there only for historical reasons.  The current
trend seems to be to make most if not all of them generic variables, in
particular to make such manipulations possible, though of course some
of them continue to make sense only when logging (but that is at the
user's discretion).

> My reason for not using the dot character is that it seems like an 
> arbitrary view concern to me and when the value is not used for logging 
> purposes it's most likely to be used for calculations or conversions; in 
> which case not having to parse out the dot character makes things 
> simpler.


From mdounin at mdounin.ru  Tue Dec  4 10:17:41 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 4 Dec 2012 14:17:41 +0400
Subject: ngx_chain_get_free_buf crashes with free=0 after return
 next_body_filter
In-Reply-To: <CAG6Cn5ax8oBcjLEJ_GgUcGGB68UpVonRCnhPJ28vXS7ok4UFEw@mail.gmail.com>
References: <CAG6Cn5bHnj_fV4EErrj1oogqfCw=7kmdtHiJr-p1C81=SgY8OA@mail.gmail.com>
 <20121203180949.GB40452@mdounin.ru>
 <CAG6Cn5Yawu2Yz0RHyC93fbGS-0zSWEk7BrZnH+1+OVPKS5jqEg@mail.gmail.com>
 <20121203191654.GF40452@mdounin.ru>
 <CAG6Cn5ax8oBcjLEJ_GgUcGGB68UpVonRCnhPJ28vXS7ok4UFEw@mail.gmail.com>
Message-ID: <20121204101741.GG40452@mdounin.ru>

Hello!

On Tue, Dec 04, 2012 at 01:18:24PM +0400, Ruslan Khusnullin wrote:

> Thanks, Maxim.
> 
> One more question: is there a way to install body/header filter only
> for locations with directive instead of installing them for all
> locations?

No.  Filter chains are global, and you have to install filters into 
them always (and check if you want to run your filter or not during 
particular call of the filter).

-- 
Maxim Dounin
http://nginx.com/support.html


From mdounin at mdounin.ru  Tue Dec  4 10:42:21 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 4 Dec 2012 14:42:21 +0400
Subject: nginx-devel Digest, Vol 38, Issue 2
In-Reply-To: <9A20C610-45EA-429A-88E8-7EC856DA2DC5@gmail.com>
References: <mailman.7.1354536001.47351.nginx-devel@nginx.org>
 <9A20C610-45EA-429A-88E8-7EC856DA2DC5@gmail.com>
Message-ID: <20121204104221.GH40452@mdounin.ru>

Hello!

On Tue, Dec 04, 2012 at 04:57:32PM +1100, Lawrence Pit wrote:

> >>After I finished this patch I checked the 1.3 branch, and noticed it
> >>already has an $msec http variable defined. But it's output is with a
> >>dot character before the milliseconds part. Why is that?
> >
> >http {
> > map $msec $s {
> >	~(?<x>.*)\. $x;
> > }
> >
> > map $msec $ms {
> >	~\.(?<x>.*) $x;
> > }
> >
> > server {
> >	return 200 "msec=$msec usec=${s}${ms}000\n";
> > }
> >}
> >
> >$ curl http://localhost:8000/
> >msec=1354471012.591 usec=1354471012591000
> 
> That's cool, impressive. Though ugly ;)  If it needs to do that for
> every request it's relatively expensive compared to what I'd rather
> do:
> 
>    proxy_set_header X-Queue-Start "${msec}000";
> 
> Is there any objection to not output the dot character starting from
> the 1.3 release?
> 
> My reason for not using the dot character is that it seems like an
> arbitrary view concern to me and when the value is not used for
> logging purposes it's most likely to be used for calculations or
> conversions; in which case not having to parse out the dot character
> makes things simpler.

I consider dot character very valuable here, as it simplifies both 
reading such numbers and calculations with them, as well as 
eliminates measurment units confusion (much like what Time::HiRes 
does with time() in Perl).  The format also matches all other time 
variables nginx exports, like $request_time, 
$upstream_response_time and so on.

In any case, the $msec variable appeared in nginx 0.3.8, 7 years ago 
(log_format only at that time), and there are no plans to change it's 
semantics.

-- 
Maxim Dounin
http://nginx.com/support.html


From mdounin at mdounin.ru  Tue Dec  4 13:08:41 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 4 Dec 2012 17:08:41 +0400
Subject: image_filter enhancement
In-Reply-To: <CANWdNRBXuGyFdo1NGUeHba2QnSYokMJpD61tW-VNcOo=TV-9gg@mail.gmail.com>
References: <CANWdNRDP_6+WHtCawX8KnoH4GFHHQeHC2tXY6-8gDctYq0Dnrg@mail.gmail.com>
 <20121106181601.GS40452@mdounin.ru>
 <CANWdNRDYuMDZOpcEwtsryfUpTBCsdB6VsSJfF+qUJV34y5j08g@mail.gmail.com>
 <20121114234732.GC40452@mdounin.ru>
 <CANWdNRB=bRCEjEBM+ksak8c6pEZw+VC1XfKBrPLEJ5zXv+B_=g@mail.gmail.com>
 <20121127160328.GZ40452@mdounin.ru>
 <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>
 <CANWdNRBXuGyFdo1NGUeHba2QnSYokMJpD61tW-VNcOo=TV-9gg@mail.gmail.com>
Message-ID: <20121204130841.GI40452@mdounin.ru>

Hello!

On Tue, Dec 04, 2012 at 09:35:07AM +0400, ivan babrou wrote:

[...]

> @@ -932,8 +953,27 @@ transparent:
>                  return NULL;
>              }
> 
> -            ox /= 2;
> -            oy /= 2;
> +            crop_offset_x = conf->crop_offset_x;
> +            crop_offset_y = conf->crop_offset_y;

You may want to use shorter local variable names here.

[...]

> @@ -1149,10 +1189,27 @@ ngx_http_image_filter_value(ngx_str_t *value)
>          return (ngx_uint_t) -1;
>      }
> 
> -    n = ngx_atoi(value->data, value->len);
> +    if (ngx_strcmp(value->data, "center") == 0) {
> +        return NGX_HTTP_IMAGE_OFFSET_CENTER;
> +
> +    } else if (ngx_strcmp(value->data, "left") == 0) {
> +        return NGX_HTTP_IMAGE_OFFSET_LEFT;
> +
> +    } else if (ngx_strcmp(value->data, "right") == 0) {
> +        return NGX_HTTP_IMAGE_OFFSET_RIGHT;
> +
> +    } else if (ngx_strcmp(value->data, "top") == 0) {
> +        return NGX_HTTP_IMAGE_OFFSET_TOP;
> 
> -    if (n > 0) {
> -        return (ngx_uint_t) n;
> +    } else if (ngx_strcmp(value->data, "bottom") == 0) {
> +        return NGX_HTTP_IMAGE_OFFSET_BOTTOM;
> +
> +    } else {
> +        n = ngx_atoi(value->data, value->len);
> +
> +        if (n > 0) {
> +            return (ngx_uint_t) n;
> +        }
>      }

I'm not happy with this change as it degrades performance of other 
values evaluation.  It probably may be left as a single function, 
but I would rather fallback to all this string matching logic only 
if the value doesn't happen to be a number, i.e. if ngx_atoi() 
fails.

I also don't really see how you are going to distinguish constants 
returned from actual values specified by numbers.

[...]

> @@ -1223,6 +1282,17 @@ ngx_http_image_filter_merge_conf(ngx_conf_t
> *cf, void *parent, void *child)
>      ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size,
>                                1 * 1024 * 1024);
> 
> +    ngx_conf_merge_uint_value(conf->crop_offset_x,
> prev->crop_offset_x, NGX_HTTP_IMAGE_OFFSET_CENTER);
> +    ngx_conf_merge_uint_value(conf->crop_offset_y,
> prev->crop_offset_y, NGX_HTTP_IMAGE_OFFSET_CENTER);
> +
> +    if (conf->oxcv == NULL) {
> +        conf->oxcv = prev->oxcv;
> +    }
> +
> +    if (conf->oycv == NULL) {
> +        conf->oycv = prev->oycv;
> +    }

I believe I've already outlined in the previous review that this 
merge logic is wrong, as well as a style problem with lines longer 
than 80 chars here.

[...]

-- 
Maxim Dounin
http://nginx.com/support.html


From ruslan.khusnullin at gmail.com  Tue Dec  4 14:39:11 2012
From: ruslan.khusnullin at gmail.com (Ruslan Khusnullin)
Date: Tue, 4 Dec 2012 18:39:11 +0400
Subject: image_filter enhancement
In-Reply-To: <20121204130841.GI40452@mdounin.ru>
References: <CANWdNRDP_6+WHtCawX8KnoH4GFHHQeHC2tXY6-8gDctYq0Dnrg@mail.gmail.com>
 <20121106181601.GS40452@mdounin.ru>
 <CANWdNRDYuMDZOpcEwtsryfUpTBCsdB6VsSJfF+qUJV34y5j08g@mail.gmail.com>
 <20121114234732.GC40452@mdounin.ru>
 <CANWdNRB=bRCEjEBM+ksak8c6pEZw+VC1XfKBrPLEJ5zXv+B_=g@mail.gmail.com>
 <20121127160328.GZ40452@mdounin.ru>
 <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>
 <CANWdNRBXuGyFdo1NGUeHba2QnSYokMJpD61tW-VNcOo=TV-9gg@mail.gmail.com>
 <20121204130841.GI40452@mdounin.ru>
Message-ID: <CAG6Cn5brd19G5oi2r_5HGqCHWfNq46Q0R19PsXCJJyMwQAqYBQ@mail.gmail.com>

On Tue, Dec 4, 2012 at 5:08 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
>> -    n = ngx_atoi(value->data, value->len);
>> +    if (ngx_strcmp(value->data, "center") == 0) {
>> +        return NGX_HTTP_IMAGE_OFFSET_CENTER;
...
> I'm not happy with this change as it degrades performance of other
...

Is it safe to do ngx_strcmp (ngx_str_t.data, "string")? As far as I
know ngx_str_t.data may have no zero at the end. I always do
ngx_strncmp (ngx_str_t.data, "string", ngx_str_t.len). Does it make
sense?


From ibobrik at gmail.com  Tue Dec  4 14:58:17 2012
From: ibobrik at gmail.com (ivan babrou)
Date: Tue, 4 Dec 2012 18:58:17 +0400
Subject: image_filter enhancement
In-Reply-To: <20121204130841.GI40452@mdounin.ru>
References: <CANWdNRDP_6+WHtCawX8KnoH4GFHHQeHC2tXY6-8gDctYq0Dnrg@mail.gmail.com>
 <20121106181601.GS40452@mdounin.ru>
 <CANWdNRDYuMDZOpcEwtsryfUpTBCsdB6VsSJfF+qUJV34y5j08g@mail.gmail.com>
 <20121114234732.GC40452@mdounin.ru>
 <CANWdNRB=bRCEjEBM+ksak8c6pEZw+VC1XfKBrPLEJ5zXv+B_=g@mail.gmail.com>
 <20121127160328.GZ40452@mdounin.ru>
 <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>
 <CANWdNRBXuGyFdo1NGUeHba2QnSYokMJpD61tW-VNcOo=TV-9gg@mail.gmail.com>
 <20121204130841.GI40452@mdounin.ru>
Message-ID: <CANWdNRATaVeEC7W0UVos9AdDM4Xq5BNRQUDea+UZkCuZzGL8gA@mail.gmail.com>

>
> Additionally, looking at the code I tend to think it's incorrect.
> That is, it's pattern you've followed is incorrect, not your code.
> E.g. the following config will result in $arg_q incorrectly used
> for quality in /image/, instead of "80" explicitly set:
>     image_filter_jpeg_quality $arg_q;
>     location /image/ {
>         image_filter crop $arg_w $arg_h;
>         image_filter_jpeg_quality 80;
>     }
> This needs fixing.


As I see other config variables are buggy too. Should I fix them too?

Anyway, this is latest version of my patch where I fixed other issues:

diff --git a/ngx_http_image_filter_module.c b/ngx_http_image_filter_module.c
index c853c33..778fcd3 100644
--- a/ngx_http_image_filter_module.c
+++ b/ngx_http_image_filter_module.c
@@ -32,6 +32,11 @@
 #define NGX_HTTP_IMAGE_GIF       2
 #define NGX_HTTP_IMAGE_PNG       3

+#define NGX_HTTP_IMAGE_OFFSET_CENTER    0
+#define NGX_HTTP_IMAGE_OFFSET_LEFT      1
+#define NGX_HTTP_IMAGE_OFFSET_RIGHT     2
+#define NGX_HTTP_IMAGE_OFFSET_TOP       3
+#define NGX_HTTP_IMAGE_OFFSET_BOTTOM    4

 #define NGX_HTTP_IMAGE_BUFFERED  0x08

@@ -43,11 +48,15 @@ typedef struct {
     ngx_uint_t                   angle;
     ngx_uint_t                   jpeg_quality;
     ngx_uint_t                   sharpen;
+    ngx_uint_t                   crop_offset_x;
+    ngx_uint_t                   crop_offset_y;

     ngx_flag_t                   transparency;

     ngx_http_complex_value_t    *wcv;
     ngx_http_complex_value_t    *hcv;
+    ngx_http_complex_value_t    *oxcv;
+    ngx_http_complex_value_t    *oycv;
     ngx_http_complex_value_t    *acv;
     ngx_http_complex_value_t    *jqcv;
     ngx_http_complex_value_t    *shcv;
@@ -66,6 +75,8 @@ typedef struct {
     ngx_uint_t                   height;
     ngx_uint_t                   max_width;
     ngx_uint_t                   max_height;
+    ngx_uint_t                   crop_offset_x;
+    ngx_uint_t                   crop_offset_y;
     ngx_uint_t                   angle;

     ngx_uint_t                   phase;
@@ -110,6 +121,8 @@ static char
*ngx_http_image_filter_jpeg_quality(ngx_conf_t *cf,
     ngx_command_t *cmd, void *conf);
 static char *ngx_http_image_filter_sharpen(ngx_conf_t *cf, ngx_command_t
*cmd,
     void *conf);
+static char *ngx_http_image_filter_offset(ngx_conf_t *cf, ngx_command_t
*cmd,
+    void *conf);
 static ngx_int_t ngx_http_image_filter_init(ngx_conf_t *cf);


@@ -150,6 +163,13 @@ static ngx_command_t  ngx_http_image_filter_commands[]
= {
       offsetof(ngx_http_image_filter_conf_t, buffer_size),
       NULL },

+    { ngx_string("image_filter_crop_offset"),
+
 NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
+      ngx_http_image_filter_offset,
+      NGX_HTTP_LOC_CONF_OFFSET,
+      0,
+      NULL },
+
       ngx_null_command
 };

@@ -737,7 +757,8 @@ ngx_http_image_resize(ngx_http_request_t *r,
ngx_http_image_filter_ctx_t *ctx)
 {
     int                            sx, sy, dx, dy, ox, oy, ax, ay, size,
                                    colors, palette, transparent, sharpen,
-                                   red, green, blue, t;
+                                   red, green, blue, t,
+                                   offset_x, offset_y;
     u_char                        *out;
     ngx_buf_t                     *b;
     ngx_uint_t                     resize;
@@ -932,8 +953,24 @@ transparent:
                 return NULL;
             }

-            ox /= 2;
-            oy /= 2;
+            offset_x = ngx_http_image_filter_get_value(r, conf->oxcv,
+
conf->crop_offset_x);
+            offset_y = ngx_http_image_filter_get_value(r, conf->oycv,
+
conf->crop_offset_y);
+
+            if (offset_x == NGX_HTTP_IMAGE_OFFSET_LEFT) {
+                ox = 0;
+
+            } else if (offset_x == NGX_HTTP_IMAGE_OFFSET_CENTER) {
+                ox /= 2;
+            }
+
+            if (offset_y == NGX_HTTP_IMAGE_OFFSET_TOP) {
+                oy = 0;
+
+            } else if (offset_y == NGX_HTTP_IMAGE_OFFSET_CENTER) {
+                oy /= 2;
+            }

             ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                            "image crop: %d x %d @ %d x %d",
@@ -1151,7 +1188,24 @@ ngx_http_image_filter_value(ngx_str_t *value)

     n = ngx_atoi(value->data, value->len);

-    if (n > 0) {
+    if (n == NGX_ERROR) {
+        if (ngx_strcmp(value->data, "left") == 0) {
+            return NGX_HTTP_IMAGE_OFFSET_LEFT;
+
+        } else if (ngx_strcmp(value->data, "right") == 0) {
+            return NGX_HTTP_IMAGE_OFFSET_RIGHT;
+
+        } else if (ngx_strcmp(value->data, "top") == 0) {
+            return NGX_HTTP_IMAGE_OFFSET_TOP;
+
+        } else if (ngx_strcmp(value->data, "bottom") == 0) {
+            return NGX_HTTP_IMAGE_OFFSET_BOTTOM;
+
+        } else {
+            return NGX_HTTP_IMAGE_OFFSET_CENTER;
+        }
+
+    } else if (n > 0) {
         return (ngx_uint_t) n;
     }

@@ -1175,6 +1229,8 @@ ngx_http_image_filter_create_conf(ngx_conf_t *cf)
     conf->angle = NGX_CONF_UNSET_UINT;
     conf->transparency = NGX_CONF_UNSET;
     conf->buffer_size = NGX_CONF_UNSET_SIZE;
+    conf->crop_offset_x = NGX_CONF_UNSET_UINT;
+    conf->crop_offset_y = NGX_CONF_UNSET_UINT;

     return conf;
 }
@@ -1223,6 +1279,19 @@ ngx_http_image_filter_merge_conf(ngx_conf_t *cf,
void *parent, void *child)
     ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size,
                               1 * 1024 * 1024);

+    ngx_conf_merge_uint_value(conf->crop_offset_x, prev->crop_offset_x,
+                              NGX_HTTP_IMAGE_OFFSET_CENTER);
+    ngx_conf_merge_uint_value(conf->crop_offset_y, prev->crop_offset_y,
+                              NGX_HTTP_IMAGE_OFFSET_CENTER);
+
+    if (conf->oxcv == NULL) {
+        conf->oxcv = prev->oxcv;
+    }
+
+    if (conf->oycv == NULL) {
+        conf->oycv = prev->oycv;
+    }
+
     return NGX_CONF_OK;
 }

@@ -1474,6 +1543,68 @@ ngx_http_image_filter_sharpen(ngx_conf_t *cf,
ngx_command_t *cmd,
 }


+static char *
+ngx_http_image_filter_offset(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf)
+{
+    ngx_http_image_filter_conf_t *imcf = conf;
+
+    ngx_str_t                         *value;
+    ngx_http_complex_value_t           cv;
+    ngx_http_compile_complex_value_t   ccv;
+
+    value = cf->args->elts;
+
+    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
+
+    ccv.cf = cf;
+    ccv.value = &value[1];
+    ccv.complex_value = &cv;
+    ccv.zero = 1;
+
+    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
+        return NGX_CONF_ERROR;
+    }
+
+    if (cv.lengths == NULL) {
+        imcf->crop_offset_x = ngx_http_image_filter_value(&value[1]);
+
+    } else {
+        imcf->oxcv = ngx_palloc(cf->pool,
sizeof(ngx_http_complex_value_t));
+        if (imcf->oxcv == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        *imcf->oxcv = cv;
+    }
+
+    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
+
+    ccv.cf = cf;
+    ccv.value = &value[2];
+    ccv.complex_value = &cv;
+    ccv.zero = 1;
+
+    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
+        return NGX_CONF_ERROR;
+    }
+
+    if (cv.lengths == NULL) {
+        imcf->crop_offset_y = ngx_http_image_filter_value(&value[2]);
+
+    } else {
+        imcf->oycv = ngx_palloc(cf->pool,
sizeof(ngx_http_complex_value_t));
+        if (imcf->oycv == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        *imcf->oycv = cv;
+    }
+
+    return NGX_CONF_OK;
+}
+
+
 static ngx_int_t
 ngx_http_image_filter_init(ngx_conf_t *cf)
 {



On 4 December 2012 17:08, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Tue, Dec 04, 2012 at 09:35:07AM +0400, ivan babrou wrote:
>
> [...]
>
> > @@ -932,8 +953,27 @@ transparent:
> >                  return NULL;
> >              }
> >
> > -            ox /= 2;
> > -            oy /= 2;
> > +            crop_offset_x = conf->crop_offset_x;
> > +            crop_offset_y = conf->crop_offset_y;
>
> You may want to use shorter local variable names here.
>
> [...]
>
> > @@ -1149,10 +1189,27 @@ ngx_http_image_filter_value(ngx_str_t *value)
> >          return (ngx_uint_t) -1;
> >      }
> >
> > -    n = ngx_atoi(value->data, value->len);
> > +    if (ngx_strcmp(value->data, "center") == 0) {
> > +        return NGX_HTTP_IMAGE_OFFSET_CENTER;
> > +
> > +    } else if (ngx_strcmp(value->data, "left") == 0) {
> > +        return NGX_HTTP_IMAGE_OFFSET_LEFT;
> > +
> > +    } else if (ngx_strcmp(value->data, "right") == 0) {
> > +        return NGX_HTTP_IMAGE_OFFSET_RIGHT;
> > +
> > +    } else if (ngx_strcmp(value->data, "top") == 0) {
> > +        return NGX_HTTP_IMAGE_OFFSET_TOP;
> >
> > -    if (n > 0) {
> > -        return (ngx_uint_t) n;
> > +    } else if (ngx_strcmp(value->data, "bottom") == 0) {
> > +        return NGX_HTTP_IMAGE_OFFSET_BOTTOM;
> > +
> > +    } else {
> > +        n = ngx_atoi(value->data, value->len);
> > +
> > +        if (n > 0) {
> > +            return (ngx_uint_t) n;
> > +        }
> >      }
>
> I'm not happy with this change as it degrades performance of other
> values evaluation.  It probably may be left as a single function,
> but I would rather fallback to all this string matching logic only
> if the value doesn't happen to be a number, i.e. if ngx_atoi()
> fails.
>
> I also don't really see how you are going to distinguish constants
> returned from actual values specified by numbers.
>
> [...]
>
> > @@ -1223,6 +1282,17 @@ ngx_http_image_filter_merge_conf(ngx_conf_t
> > *cf, void *parent, void *child)
> >      ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size,
> >                                1 * 1024 * 1024);
> >
> > +    ngx_conf_merge_uint_value(conf->crop_offset_x,
> > prev->crop_offset_x, NGX_HTTP_IMAGE_OFFSET_CENTER);
> > +    ngx_conf_merge_uint_value(conf->crop_offset_y,
> > prev->crop_offset_y, NGX_HTTP_IMAGE_OFFSET_CENTER);
> > +
> > +    if (conf->oxcv == NULL) {
> > +        conf->oxcv = prev->oxcv;
> > +    }
> > +
> > +    if (conf->oycv == NULL) {
> > +        conf->oycv = prev->oycv;
> > +    }
>
> I believe I've already outlined in the previous review that this
> merge logic is wrong, as well as a style problem with lines longer
> than 80 chars here.
>
> [...]
>
> --
> Maxim Dounin
> http://nginx.com/support.html
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>



-- 
Regards, Ian Babrou
http://bobrik.name http://twitter.com/ibobrik skype:i.babrou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121204/9d5aa5df/attachment-0001.html>

From mdounin at mdounin.ru  Wed Dec  5 14:43:13 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 5 Dec 2012 18:43:13 +0400
Subject: image_filter enhancement
In-Reply-To: <CAG6Cn5brd19G5oi2r_5HGqCHWfNq46Q0R19PsXCJJyMwQAqYBQ@mail.gmail.com>
References: <CANWdNRDP_6+WHtCawX8KnoH4GFHHQeHC2tXY6-8gDctYq0Dnrg@mail.gmail.com>
 <20121106181601.GS40452@mdounin.ru>
 <CANWdNRDYuMDZOpcEwtsryfUpTBCsdB6VsSJfF+qUJV34y5j08g@mail.gmail.com>
 <20121114234732.GC40452@mdounin.ru>
 <CANWdNRB=bRCEjEBM+ksak8c6pEZw+VC1XfKBrPLEJ5zXv+B_=g@mail.gmail.com>
 <20121127160328.GZ40452@mdounin.ru>
 <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>
 <CANWdNRBXuGyFdo1NGUeHba2QnSYokMJpD61tW-VNcOo=TV-9gg@mail.gmail.com>
 <20121204130841.GI40452@mdounin.ru>
 <CAG6Cn5brd19G5oi2r_5HGqCHWfNq46Q0R19PsXCJJyMwQAqYBQ@mail.gmail.com>
Message-ID: <20121205144312.GN40452@mdounin.ru>

Hello!

On Tue, Dec 04, 2012 at 06:39:11PM +0400, Ruslan Khusnullin wrote:

> On Tue, Dec 4, 2012 at 5:08 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> >> -    n = ngx_atoi(value->data, value->len);
> >> +    if (ngx_strcmp(value->data, "center") == 0) {
> >> +        return NGX_HTTP_IMAGE_OFFSET_CENTER;
> ...
> > I'm not happy with this change as it degrades performance of other
> ...
> 
> Is it safe to do ngx_strcmp (ngx_str_t.data, "string")? As far as I
> know ngx_str_t.data may have no zero at the end.

In the code in question ngx_strcmp() is ok as long as it's used on 
image_filter_crop_offset values, as they are null-terminated due 
to zero flag set during complex value compilation (and that's why 
I missed it in my review...).

It's not safe for other values though, and as the function is used 
for all values - it needs fixing.

Thanks for noting this.

> I always do
> ngx_strncmp (ngx_str_t.data, "string", ngx_str_t.len). Does it make
> sense?

Correct test for non-null-terminated string is

    s.len == sizeof("string") - 1
    && ngx_strncmp(s.data, "string", sizeof("string") - 1)

With your code string "st" will result in match, while you 
probably don't want it to.

-- 
Maxim Dounin
http://nginx.com/support.html


From mdounin at mdounin.ru  Wed Dec  5 14:52:44 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 5 Dec 2012 18:52:44 +0400
Subject: image_filter enhancement
In-Reply-To: <CANWdNRATaVeEC7W0UVos9AdDM4Xq5BNRQUDea+UZkCuZzGL8gA@mail.gmail.com>
References: <CANWdNRDP_6+WHtCawX8KnoH4GFHHQeHC2tXY6-8gDctYq0Dnrg@mail.gmail.com>
 <20121106181601.GS40452@mdounin.ru>
 <CANWdNRDYuMDZOpcEwtsryfUpTBCsdB6VsSJfF+qUJV34y5j08g@mail.gmail.com>
 <20121114234732.GC40452@mdounin.ru>
 <CANWdNRB=bRCEjEBM+ksak8c6pEZw+VC1XfKBrPLEJ5zXv+B_=g@mail.gmail.com>
 <20121127160328.GZ40452@mdounin.ru>
 <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>
 <CANWdNRBXuGyFdo1NGUeHba2QnSYokMJpD61tW-VNcOo=TV-9gg@mail.gmail.com>
 <20121204130841.GI40452@mdounin.ru>
 <CANWdNRATaVeEC7W0UVos9AdDM4Xq5BNRQUDea+UZkCuZzGL8gA@mail.gmail.com>
Message-ID: <20121205145244.GO40452@mdounin.ru>

Hello!

On Tue, Dec 04, 2012 at 06:58:17PM +0400, ivan babrou wrote:

> >
> > Additionally, looking at the code I tend to think it's incorrect.
> > That is, it's pattern you've followed is incorrect, not your code.
> > E.g. the following config will result in $arg_q incorrectly used
> > for quality in /image/, instead of "80" explicitly set:
> >     image_filter_jpeg_quality $arg_q;
> >     location /image/ {
> >         image_filter crop $arg_w $arg_h;
> >         image_filter_jpeg_quality 80;
> >     }
> > This needs fixing.
> 
> 
> As I see other config variables are buggy too. Should I fix them too?

It would be fine to produce a patch which fixes existing config 
variables, and the patch to add crop offsets on top of it.

> Anyway, this is latest version of my patch where I fixed other issues:

I don't see at least some of previously expressed points 
addressed, so I assume you are still working on it.

Please also see other point rightfully noted by Ruslan Khusnullin - 
you can't use ngx_strcmp() in ngx_http_image_filter_get_value() as 
it's used not only on null-terminated strings.

-- 
Maxim Dounin
http://nginx.com/support.html


From ngx.eugaia at gmail.com  Wed Dec  5 16:54:00 2012
From: ngx.eugaia at gmail.com (Marcus Clyne)
Date: Wed, 05 Dec 2012 13:54:00 -0300
Subject: image_filter enhancement
In-Reply-To: <20121205145244.GO40452@mdounin.ru>
References: <CANWdNRDP_6+WHtCawX8KnoH4GFHHQeHC2tXY6-8gDctYq0Dnrg@mail.gmail.com>
 <20121106181601.GS40452@mdounin.ru>
 <CANWdNRDYuMDZOpcEwtsryfUpTBCsdB6VsSJfF+qUJV34y5j08g@mail.gmail.com>
 <20121114234732.GC40452@mdounin.ru>
 <CANWdNRB=bRCEjEBM+ksak8c6pEZw+VC1XfKBrPLEJ5zXv+B_=g@mail.gmail.com>
 <20121127160328.GZ40452@mdounin.ru>
 <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>
 <CANWdNRBXuGyFdo1NGUeHba2QnSYokMJpD61tW-VNcOo=TV-9gg@mail.gmail.com>
 <20121204130841.GI40452@mdounin.ru>
 <CANWdNRATaVeEC7W0UVos9AdDM4Xq5BNRQUDea+UZkCuZzGL8gA@mail.gmail.com>
 <20121205145244.GO40452@mdounin.ru>
Message-ID: <50BF7C28.8010704@gmail.com>

Hi,

El 05/12/12 11:52, Maxim Dounin escribi?:
> Please also see other point rightfully noted by Ruslan Khusnullin - 
> you can't use ngx_strcmp() in ngx_http_image_filter_get_value() as 
> it's used not only on null-terminated strings. 
If you check there's enough space to avoid buffer overruns, you could 
use the ngx_str[n]cmp() macros to save on function calls.

Marcus.


From ru at nginx.com  Thu Dec  6 21:22:58 2012
From: ru at nginx.com (ru at nginx.com)
Date: Thu,  6 Dec 2012 21:22:58 +0000
Subject: [nginx] svn commit:  r4947 - trunk/src/http/modules
Message-ID: <20121206212258.BCAB83F9C47@mail.nginx.com>

Author: ru
Date: 2012-12-06 21:22:58 +0000 (Thu, 06 Dec 2012)
New Revision: 4947
URL: http://trac.nginx.org/nginx/changeset/4947/nginx

Log:
Xslt: prevented infinite loop.

If XSLT transformation failed and error 500 was handled in the same
location, an infinite loop occured that exhausted the stack.


Modified:
   trunk/src/http/modules/ngx_http_xslt_filter_module.c

Modified: trunk/src/http/modules/ngx_http_xslt_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_xslt_filter_module.c	2012-12-03 16:00:26 UTC (rev 4946)
+++ trunk/src/http/modules/ngx_http_xslt_filter_module.c	2012-12-06 21:22:58 UTC (rev 4947)
@@ -307,7 +307,7 @@
     ctx->done = 1;
 
     if (b == NULL) {
-        return ngx_http_filter_finalize_request(r, NULL,
+        return ngx_http_filter_finalize_request(r, &ngx_http_xslt_filter_module,
                                                NGX_HTTP_INTERNAL_SERVER_ERROR);
     }
 
@@ -315,7 +315,7 @@
 
     if (cln == NULL) {
         ngx_free(b->pos);
-        return ngx_http_filter_finalize_request(r, NULL,
+        return ngx_http_filter_finalize_request(r, &ngx_http_xslt_filter_module,
                                                NGX_HTTP_INTERNAL_SERVER_ERROR);
     }
 


From ru at nginx.com  Thu Dec  6 23:03:53 2012
From: ru at nginx.com (ru at nginx.com)
Date: Thu,  6 Dec 2012 23:03:53 +0000
Subject: [nginx] svn commit:  r4948 - in trunk/src/http: . modules
Message-ID: <20121206230353.CE31A3F9F41@mail.nginx.com>

Author: ru
Date: 2012-12-06 23:03:53 +0000 (Thu, 06 Dec 2012)
New Revision: 4948
URL: http://trac.nginx.org/nginx/changeset/4948/nginx

Log:
Allow the complex value to be defined as an empty string.

This makes conversion from strings to complex values possible
without the loss of functionality.


Modified:
   trunk/src/http/modules/ngx_http_auth_basic_module.c
   trunk/src/http/modules/ngx_http_fastcgi_module.c
   trunk/src/http/modules/ngx_http_proxy_module.c
   trunk/src/http/modules/ngx_http_scgi_module.c
   trunk/src/http/modules/ngx_http_sub_filter_module.c
   trunk/src/http/modules/ngx_http_uwsgi_module.c
   trunk/src/http/ngx_http_core_module.c
   trunk/src/http/ngx_http_script.c

Modified: trunk/src/http/modules/ngx_http_auth_basic_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_auth_basic_module.c	2012-12-06 21:22:58 UTC (rev 4947)
+++ trunk/src/http/modules/ngx_http_auth_basic_module.c	2012-12-06 23:03:53 UTC (rev 4948)
@@ -117,7 +117,7 @@
 
     alcf = ngx_http_get_module_loc_conf(r, ngx_http_auth_basic_module);
 
-    if (alcf->realm.len == 0 || alcf->user_file.value.len == 0) {
+    if (alcf->realm.len == 0 || alcf->user_file.value.data == NULL) {
         return NGX_DECLINED;
     }
 
@@ -390,7 +390,7 @@
         conf->realm = prev->realm;
     }
 
-    if (conf->user_file.value.len == 0) {
+    if (conf->user_file.value.data == NULL) {
         conf->user_file = prev->user_file;
     }
 
@@ -456,7 +456,7 @@
     ngx_str_t                         *value;
     ngx_http_compile_complex_value_t   ccv;
 
-    if (alcf->user_file.value.len) {
+    if (alcf->user_file.value.data) {
         return "is duplicate";
     }
 

Modified: trunk/src/http/modules/ngx_http_fastcgi_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_fastcgi_module.c	2012-12-06 21:22:58 UTC (rev 4947)
+++ trunk/src/http/modules/ngx_http_fastcgi_module.c	2012-12-06 23:03:53 UTC (rev 4948)
@@ -3014,7 +3014,7 @@
 
     value = cf->args->elts;
 
-    if (flcf->cache_key.value.len) {
+    if (flcf->cache_key.value.data) {
         return "is duplicate";
     }
 

Modified: trunk/src/http/modules/ngx_http_proxy_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_proxy_module.c	2012-12-06 21:22:58 UTC (rev 4947)
+++ trunk/src/http/modules/ngx_http_proxy_module.c	2012-12-06 23:03:53 UTC (rev 4948)
@@ -837,7 +837,7 @@
         return NGX_ERROR;
     }
 
-    if (plcf->cache_key.value.len) {
+    if (plcf->cache_key.value.data) {
 
         if (ngx_http_complex_value(r, &plcf->cache_key, key) != NGX_OK) {
             return NGX_ERROR;
@@ -3651,7 +3651,7 @@
 
     value = cf->args->elts;
 
-    if (plcf->cache_key.value.len) {
+    if (plcf->cache_key.value.data) {
         return "is duplicate";
     }
 

Modified: trunk/src/http/modules/ngx_http_scgi_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_scgi_module.c	2012-12-06 21:22:58 UTC (rev 4947)
+++ trunk/src/http/modules/ngx_http_scgi_module.c	2012-12-06 23:03:53 UTC (rev 4948)
@@ -1771,7 +1771,7 @@
 
     value = cf->args->elts;
 
-    if (scf->cache_key.value.len) {
+    if (scf->cache_key.value.data) {
         return "is duplicate";
     }
 

Modified: trunk/src/http/modules/ngx_http_sub_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_sub_filter_module.c	2012-12-06 21:22:58 UTC (rev 4947)
+++ trunk/src/http/modules/ngx_http_sub_filter_module.c	2012-12-06 23:03:53 UTC (rev 4948)
@@ -628,7 +628,7 @@
     ngx_str_t                         *value;
     ngx_http_compile_complex_value_t   ccv;
 
-    if (slcf->match.len) {
+    if (slcf->match.data) {
         return "is duplicate";
     }
 
@@ -688,7 +688,7 @@
     ngx_conf_merge_value(conf->once, prev->once, 1);
     ngx_conf_merge_str_value(conf->match, prev->match, "");
 
-    if (conf->value.value.len == 0) {
+    if (conf->value.value.data == NULL) {
         conf->value = prev->value;
     }
 

Modified: trunk/src/http/modules/ngx_http_uwsgi_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_uwsgi_module.c	2012-12-06 21:22:58 UTC (rev 4947)
+++ trunk/src/http/modules/ngx_http_uwsgi_module.c	2012-12-06 23:03:53 UTC (rev 4948)
@@ -1807,7 +1807,7 @@
 
     value = cf->args->elts;
 
-    if (uwcf->cache_key.value.len) {
+    if (uwcf->cache_key.value.data) {
         return "is duplicate";
     }
 

Modified: trunk/src/http/ngx_http_core_module.c
===================================================================
--- trunk/src/http/ngx_http_core_module.c	2012-12-06 21:22:58 UTC (rev 4947)
+++ trunk/src/http/ngx_http_core_module.c	2012-12-06 23:03:53 UTC (rev 4948)
@@ -4593,7 +4593,7 @@
 
     ngx_str_null(&args);
 
-    if (cv.lengths == NULL && uri.data[0] == '/') {
+    if (cv.lengths == NULL && uri.len && uri.data[0] == '/') {
         p = (u_char *) ngx_strchr(uri.data, '?');
 
         if (p) {

Modified: trunk/src/http/ngx_http_script.c
===================================================================
--- trunk/src/http/ngx_http_script.c	2012-12-06 21:22:58 UTC (rev 4947)
+++ trunk/src/http/ngx_http_script.c	2012-12-06 23:03:53 UTC (rev 4948)
@@ -114,11 +114,6 @@
 
     v = ccv->value;
 
-    if (v->len == 0) {
-        ngx_conf_log_error(NGX_LOG_EMERG, ccv->cf, 0, "empty parameter");
-        return NGX_ERROR;
-    }
-
     nv = 0;
     nc = 0;
 
@@ -133,8 +128,9 @@
         }
     }
 
-    if (v->data[0] != '$' && (ccv->conf_prefix || ccv->root_prefix)) {
-
+    if ((v->len == 0 || v->data[0] != '$')
+        && (ccv->conf_prefix || ccv->root_prefix))
+    {
         if (ngx_conf_full_name(ccv->cf->cycle, v, ccv->conf_prefix) != NGX_OK) {
             return NGX_ERROR;
         }


From piotr at cloudflare.com  Fri Dec  7 01:14:11 2012
From: piotr at cloudflare.com (Piotr Sikora)
Date: Thu, 6 Dec 2012 17:14:11 -0800
Subject: [PATCH] Core: fix content handler and location configuration mismatch.
Message-ID: <CADMhe6cDf57_0wDvTt3i9JwuOGC8UTSwpUyToo8UgsE+HgT=zA@mail.gmail.com>

Core: fix content handler and location configuration mismatch.

When using exclusive content handlers (proxy_pass, etc.) and "if"
pseudo-locations, content handler was always set to the one from the
last "if" pseudo-location that evaluated to true or to the one from
the location block if "if" pseudo-locations didn't provide any.
At the same time, location configuration was updated for each "if"
pseudo-location that evaluated to true, which resulted in content
handler and location configuration mismatch.

For example, following configuration would result in SIGSEGV, because
"proxy_pass" content handler (set in the "if ($pass)" pseudo-location)
would be called with location configuration from the "if ($slow)"
pseudo-location which doesn't contain any upstream configuration.

    location / {
        set $pass  1;
        set $slow  1;

        if ($pass) {
            proxy_pass  http://127.0.0.1:8000;
        }

        if ($slow) {
            limit_rate  10k;
        }
    }

This patch fixes this issue by keeping track of and restoring location
configuration from which the content handler originated.

Signed-off-by: Piotr Sikora <piotr at cloudflare.com>
---
 src/http/ngx_http_core_module.c | 2 ++
 src/http/ngx_http_request.h     | 1 +
 2 files changed, 3 insertions(+)

diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
index faecadd..aec951c 100644
--- a/src/http/ngx_http_core_module.c
+++ b/src/http/ngx_http_core_module.c
@@ -1400,6 +1400,7 @@ ngx_http_core_content_phase(ngx_http_request_t *r,
     ngx_str_t  path;

     if (r->content_handler) {
+        r->loc_conf = r->content_handler_loc_conf;
         r->write_event_handler = ngx_http_request_empty_handler;
         ngx_http_finalize_request(r, r->content_handler(r));
         return NGX_OK;
@@ -1526,6 +1527,7 @@ ngx_http_update_location_config(ngx_http_request_t *r)

     if (clcf->handler) {
         r->content_handler = clcf->handler;
+        r->content_handler_loc_conf = r->loc_conf;
     }
 }

diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
index f234840..97c9607 100644
--- a/src/http/ngx_http_request.h
+++ b/src/http/ngx_http_request.h
@@ -358,6 +358,7 @@ struct ngx_http_request_s {
     void                            **main_conf;
     void                            **srv_conf;
     void                            **loc_conf;
+    void                            **content_handler_loc_conf;

     ngx_http_event_handler_pt         read_event_handler;
     ngx_http_event_handler_pt         write_event_handler;
-- 
1.8.0.1


From kindy61 at gmail.com  Fri Dec  7 09:01:50 2012
From: kindy61 at gmail.com (kindy)
Date: Fri, 7 Dec 2012 17:01:50 +0800
Subject: How about support "long string" in config
Message-ID: <CAPbYWPZqDNXWuQ8G8ceozsKqYTTWLgXad+2ScHp=9_foFM1FqA@mail.gmail.com>

Hi,

Currently nginx support only string in double quote and single quote,
how about add support for long string just like lua do (
http://www.lua.org/manual/5.1/manual.html#2.1 ).
then we can put everything we like in config file.

Also, we can support the long comment. and then we can comment block of
configure temperately.


Append the long string description here:

Strings can also be defined using a long format enclosed by *long brackets*.
We define an *opening long bracket of level n* as an opening square bracket
followed by *n* equal signs followed by another opening square bracket. So,
an opening long bracket of level 0 is written as [[, an opening long
bracket of level 1 is written as [=[, and so on. A *closing long bracket* is
defined similarly; for instance, a closing long bracket of level 4 is
written as ]====]. A long string starts with an opening long bracket of any
level and ends at the first closing long bracket of the same level.
Literals in this bracketed form can run for several lines, do not interpret
any escape sequences, and ignore long brackets of any other level. They can
contain anything except a closing bracket of the proper level.


-- 
- - - - - - - - - - - -
??(Kindy Lin)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121207/f92da0ac/attachment.html>

From mdounin at mdounin.ru  Fri Dec  7 16:27:45 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 7 Dec 2012 20:27:45 +0400
Subject: [PATCH] Core: fix content handler and location configuration
 mismatch.
In-Reply-To: <CADMhe6cDf57_0wDvTt3i9JwuOGC8UTSwpUyToo8UgsE+HgT=zA@mail.gmail.com>
References: <CADMhe6cDf57_0wDvTt3i9JwuOGC8UTSwpUyToo8UgsE+HgT=zA@mail.gmail.com>
Message-ID: <20121207162745.GB40452@mdounin.ru>

Hello!

On Thu, Dec 06, 2012 at 05:14:11PM -0800, Piotr Sikora wrote:

> Core: fix content handler and location configuration mismatch.
> 
> When using exclusive content handlers (proxy_pass, etc.) and "if"
> pseudo-locations, content handler was always set to the one from the
> last "if" pseudo-location that evaluated to true or to the one from
> the location block if "if" pseudo-locations didn't provide any.
> At the same time, location configuration was updated for each "if"
> pseudo-location that evaluated to true, which resulted in content
> handler and location configuration mismatch.
> 
> For example, following configuration would result in SIGSEGV, because
> "proxy_pass" content handler (set in the "if ($pass)" pseudo-location)
> would be called with location configuration from the "if ($slow)"
> pseudo-location which doesn't contain any upstream configuration.
> 
>     location / {
>         set $pass  1;
>         set $slow  1;
> 
>         if ($pass) {
>             proxy_pass  http://127.0.0.1:8000;
>         }
> 
>         if ($slow) {
>             limit_rate  10k;
>         }
>     }
> 
> This patch fixes this issue by keeping track of and restoring location
> configuration from which the content handler originated.

And what will happen with the configuration specified in the "if 
($slow) { ... }" as in the config above?

For limit_rate your config will work as it's saved in request by 
ngx_http_update_location_config(), but e.g.


    if ($slow) {
        add_header "X-Accel-Limit-Rate: 10k";
    }

will silently ignore if() match, which is wrong.

[...]

-- 
Maxim Dounin
http://nginx.com/support.html


From mdounin at mdounin.ru  Fri Dec  7 17:01:14 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 7 Dec 2012 21:01:14 +0400
Subject: How about support "long string" in config
In-Reply-To: <CAPbYWPZqDNXWuQ8G8ceozsKqYTTWLgXad+2ScHp=9_foFM1FqA@mail.gmail.com>
References: <CAPbYWPZqDNXWuQ8G8ceozsKqYTTWLgXad+2ScHp=9_foFM1FqA@mail.gmail.com>
Message-ID: <20121207170114.GC40452@mdounin.ru>

Hello!

On Fri, Dec 07, 2012 at 05:01:50PM +0800, kindy wrote:

> Hi,
> 
> Currently nginx support only string in double quote and single quote,
> how about add support for long string just like lua do (
> http://www.lua.org/manual/5.1/manual.html#2.1 ).
> then we can put everything we like in config file.
> 
> Also, we can support the long comment. and then we can comment block of
> configure temperately.
> 
> 
> Append the long string description here:
> 
> Strings can also be defined using a long format enclosed by *long brackets*.
> We define an *opening long bracket of level n* as an opening square bracket
> followed by *n* equal signs followed by another opening square bracket. So,
> an opening long bracket of level 0 is written as [[, an opening long
> bracket of level 1 is written as [=[, and so on. A *closing long bracket* is
> defined similarly; for instance, a closing long bracket of level 4 is
> written as ]====]. A long string starts with an opening long bracket of any
> level and ends at the first closing long bracket of the same level.
> Literals in this bracketed form can run for several lines, do not interpret
> any escape sequences, and ignore long brackets of any other level. They can
> contain anything except a closing bracket of the proper level.

I don't think we need it.

-- 
Maxim Dounin
http://nginx.com/support.html


From piotr at cloudflare.com  Fri Dec  7 20:24:49 2012
From: piotr at cloudflare.com (Piotr Sikora)
Date: Fri, 7 Dec 2012 12:24:49 -0800
Subject: [PATCH] Fix PCRE detection on OSX.
Message-ID: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>

Fix PCRE detection on OSX.

OSX ships with PCRE library in /usr, but without headers for it.

OSX default search paths includes both /usr and /usr/local, which
results in library (from /usr) and headers (from /usr/include)
mismatch when PCRE is installed via Homebrew and we try to compile
nginx without "-L/usr/local" parameter.

The solution is to skip the detection using default search paths
on OSX and let ./configure script detect PCRE installed in /usr/local
or other common prefixes.

Signed-off-by: Piotr Sikora <piotr at cloudflare.com>
---
 auto/lib/pcre/conf | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/auto/lib/pcre/conf b/auto/lib/pcre/conf
index 6a8c326..bc3bc56 100644
--- a/auto/lib/pcre/conf
+++ b/auto/lib/pcre/conf
@@ -101,7 +101,12 @@ else
         ngx_feature_test="pcre *re;
                           re = pcre_compile(NULL, 0, NULL, 0, NULL);
                           if (re == NULL) return 1"
-        . auto/feature
+
+        if [ "$NGX_SYSTEM" = "Darwin" ]; then
+            ngx_found=no
+        else
+            . auto/feature
+        fi

         if [ $ngx_found = no ]; then

-- 
1.8.0.1


From agentzh at gmail.com  Fri Dec  7 20:29:57 2012
From: agentzh at gmail.com (agentzh)
Date: Fri, 7 Dec 2012 12:29:57 -0800
Subject: How about support "long string" in config
In-Reply-To: <20121207170114.GC40452@mdounin.ru>
References: <CAPbYWPZqDNXWuQ8G8ceozsKqYTTWLgXad+2ScHp=9_foFM1FqA@mail.gmail.com>
 <20121207170114.GC40452@mdounin.ru>
Message-ID: <CAB4Tn6O4OwKXdffaBHPuxzdtXTDMF_3dS04Z8fxcysVk2nwTMg@mail.gmail.com>

Hello!

On Fri, Dec 7, 2012 at 9:01 AM, Maxim Dounin wrote:
>> Strings can also be defined using a long format enclosed by *long brackets*.
>> We define an *opening long bracket of level n* as an opening square bracket
>> followed by *n* equal signs followed by another opening square bracket. So,
>> an opening long bracket of level 0 is written as [[, an opening long
>> bracket of level 1 is written as [=[, and so on. A *closing long bracket* is
>> defined similarly; for instance, a closing long bracket of level 4 is
>> written as ]====]. A long string starts with an opening long bracket of any
>> level and ends at the first closing long bracket of the same level.
>> Literals in this bracketed form can run for several lines, do not interpret
>> any escape sequences, and ignore long brackets of any other level. They can
>> contain anything except a closing bracket of the proper level.
>
> I don't think we need it.
>

I believe this feature is great for

1.  inlining source code written in other languages (like Lua or
JavaScript) directly within nginx.conf and also for

2. inlining big chunk of text into directives like "echo" or "return"
to serve static contents directly from memory.

The current string syntax for nginx.conf is cumbersome when there're
many special characters like \ that need escaping.

So I think it'll be a big win for 3rd-party modules :)

Best regards,
-agentzh


From ru at nginx.com  Fri Dec  7 22:21:29 2012
From: ru at nginx.com (Ruslan Ermilov)
Date: Sat, 8 Dec 2012 02:21:29 +0400
Subject: [PATCH] Fix PCRE detection on OSX.
In-Reply-To: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>
References: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>
Message-ID: <20121207222129.GB30441@lo0.su>

On Fri, Dec 07, 2012 at 12:24:49PM -0800, Piotr Sikora wrote:
> Fix PCRE detection on OSX.
> 
> OSX ships with PCRE library in /usr, but without headers for it.
> 
> OSX default search paths includes both /usr and /usr/local, which
> results in library (from /usr) and headers (from /usr/include)
> mismatch when PCRE is installed via Homebrew and we try to compile
> nginx without "-L/usr/local" parameter.
> 
> The solution is to skip the detection using default search paths
> on OSX and let ./configure script detect PCRE installed in /usr/local
> or other common prefixes.

OS X is known to be broken in that respect.  That's why MacPorts
installs stuff into /opt/local by default, instead of /usr/local.

https://trac.macports.org/wiki/FAQ#defaultprefix
https://trac.macports.org/wiki/FAQ#usrlocal

But why should nginx care to "fix" this?

I think
    configure --with-cc-opt="-L/usr/local/lib -I/usr/local/include"
should help survive with broken build tools even if you install
your stuff into /usr/local.


From piotr at cloudflare.com  Fri Dec  7 22:52:49 2012
From: piotr at cloudflare.com (Piotr Sikora)
Date: Fri, 7 Dec 2012 14:52:49 -0800
Subject: [PATCH] Fix PCRE detection on OSX.
In-Reply-To: <20121207222129.GB30441@lo0.su>
References: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>
 <20121207222129.GB30441@lo0.su>
Message-ID: <CADMhe6c91o1J4Exm8pQhbdwUo_L1c24kGjAh5mF=DW6vS9gzRQ@mail.gmail.com>

Hey Ruslan,

> OS X is known to be broken in that respect.

Yes, it is, which makes this a known problem with an easy fix.

> That's why MacPorts
> installs stuff into /opt/local by default, instead of /usr/local.
>
> https://trac.macports.org/wiki/FAQ#defaultprefix
> https://trac.macports.org/wiki/FAQ#usrlocal

I know, I'm using MacPorts... But enough people was hit by this
problem that I've decided to "fix" it.

> But why should nginx care to "fix" this?
>
> I think
>     configure --with-cc-opt="-L/usr/local/lib -I/usr/local/include"
> should help survive with broken build tools even if you install
> your stuff into /usr/local.

Except that nginx's ./configure (called without additional options)
decides that it "found" PCRE and that it doesn't need any additional
arguments in order to compile properly.

Anyway, do you see anything wrong with this patch or is it simply
"it's not our problem" reasoning?

Best regards,
Piotr Sikora


From matthieu.tourne at gmail.com  Fri Dec  7 23:01:04 2012
From: matthieu.tourne at gmail.com (Matthieu Tourne)
Date: Fri, 7 Dec 2012 15:01:04 -0800
Subject: [PATCH] Fix PCRE detection on OSX.
In-Reply-To: <20121207222129.GB30441@lo0.su>
References: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>
 <20121207222129.GB30441@lo0.su>
Message-ID: <CAAGYmFdATb7xGBsfDPewYGi=afK2V4Zz=OwB1=s8WG53TZc49Q@mail.gmail.com>

On Fri, Dec 7, 2012 at 2:21 PM, Ruslan Ermilov <ru at nginx.com> wrote:

> On Fri, Dec 07, 2012 at 12:24:49PM -0800, Piotr Sikora wrote:
> > Fix PCRE detection on OSX.
> >
> > OSX ships with PCRE library in /usr, but without headers for it.
> >
> > OSX default search paths includes both /usr and /usr/local, which
> > results in library (from /usr) and headers (from /usr/include)
> > mismatch when PCRE is installed via Homebrew and we try to compile
> > nginx without "-L/usr/local" parameter.
> >
> > The solution is to skip the detection using default search paths
> > on OSX and let ./configure script detect PCRE installed in /usr/local
> > or other common prefixes.
>
> OS X is known to be broken in that respect.  That's why MacPorts
> installs stuff into /opt/local by default, instead of /usr/local.
>
> https://trac.macports.org/wiki/FAQ#defaultprefix
> https://trac.macports.org/wiki/FAQ#usrlocal
>
> But why should nginx care to "fix" this?


They don't, but there is already an exception line in the auto/conf for #
Macports, and adding a bit more just makes it easier for people using the
other concurrent (and newer) system of packages on Mac OS: Homebrew.

Actually it's worse than that.
When using homebrew the included pcre.h looks like it will support
PCRE_JIT, but the library used for the link doesn't export those symbols.
It results in a bad linking error, which might throw off some users that
have no idea why this is happening ..

I don't really have a way to fix Mac OS, or homebrew. But we can help
making sure Nginx will compile everywhere.

An alternative is to specifically look for PCRE_JIT functions, and test if
the linking will work (see example below), and either throw off an error at
./configure time, or add some exceptions.
And I believe this stays in the style of Nginx to handle "exotic" systems.

Regards,
Matthieu.



Config example for PCRE_JIT detection :

# Check version of PCRE to turn on PCRE_JIT

ngx_feature="PCRE_JIT capable PCRE library"
ngx_feature_name="NGX_PCRE"
ngx_feature_run=no
ngx_feature_incs="#include <pcre.h>"
ngx_feature_path=
ngx_feature_libs="-lpcre"
ngx_feature_test="
/* PCRE_JIT starts after 8.21 */
#if (PCRE_MAJOR > 8) || (PCRE_MAJOR == 8 && PCRE_MINOR >= 21)
    printf(\"%p\", pcre_free_study);
#endif
"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121207/5aed8ad8/attachment.html>

From info at tvdw.eu  Fri Dec  7 23:04:20 2012
From: info at tvdw.eu (Tom van der Woerdt)
Date: Sat, 08 Dec 2012 00:04:20 +0100
Subject: [PATCH] Fix PCRE detection on OSX.
In-Reply-To: <CADMhe6c91o1J4Exm8pQhbdwUo_L1c24kGjAh5mF=DW6vS9gzRQ@mail.gmail.com>
References: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>
 <20121207222129.GB30441@lo0.su>
 <CADMhe6c91o1J4Exm8pQhbdwUo_L1c24kGjAh5mF=DW6vS9gzRQ@mail.gmail.com>
Message-ID: <50C275F4.8070005@tvdw.eu>

I'm no nginx developer, but I have to agree with Piotr on this one. The 
./configure command is supposed to check for any system problems and 
determine the proper build configuration or error if that's not 
possible. Anything that could cause the `make` command to fail should be 
checked in the configuration script.

While it's obviously the OS at fault in this case, it is improper 
behavior for the configuration script to succeed if the actual 
compilation will fail.

That said, I believe that the patch is not ideal. Assuming that PCRE is 
not installed when building on Mac is bad behavior. Ideally the 
configuration script would check for the presence of the library. But 
until that is implemented (if ever), this patch is probably the best 
solution.

Tom


Op 12/7/12 11:52 PM, Piotr Sikora schreef:
> Hey Ruslan,
>
>> OS X is known to be broken in that respect.
> Yes, it is, which makes this a known problem with an easy fix.
>
>> That's why MacPorts
>> installs stuff into /opt/local by default, instead of /usr/local.
>>
>> https://trac.macports.org/wiki/FAQ#defaultprefix
>> https://trac.macports.org/wiki/FAQ#usrlocal
> I know, I'm using MacPorts... But enough people was hit by this
> problem that I've decided to "fix" it.
>
>> But why should nginx care to "fix" this?
>>
>> I think
>>      configure --with-cc-opt="-L/usr/local/lib -I/usr/local/include"
>> should help survive with broken build tools even if you install
>> your stuff into /usr/local.
> Except that nginx's ./configure (called without additional options)
> decides that it "found" PCRE and that it doesn't need any additional
> arguments in order to compile properly.
>
> Anyway, do you see anything wrong with this patch or is it simply
> "it's not our problem" reasoning?
>
> Best regards,
> Piotr Sikora
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel


From piotr at cloudflare.com  Fri Dec  7 23:15:51 2012
From: piotr at cloudflare.com (Piotr Sikora)
Date: Fri, 7 Dec 2012 15:15:51 -0800
Subject: [PATCH] Fix PCRE detection on OSX.
In-Reply-To: <50C275F4.8070005@tvdw.eu>
References: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>
 <20121207222129.GB30441@lo0.su>
 <CADMhe6c91o1J4Exm8pQhbdwUo_L1c24kGjAh5mF=DW6vS9gzRQ@mail.gmail.com>
 <50C275F4.8070005@tvdw.eu>
Message-ID: <CADMhe6cVh3=+zsqNp72+C2zOUX8jSH-G3TjOBdk_ec8RsYeHGw@mail.gmail.com>

Hey Tom,

> That said, I believe that the patch is not ideal. Assuming that PCRE is not
> installed when building on Mac is bad behavior. Ideally the configuration
> script would check for the presence of the library. But until that is
> implemented (if ever), this patch is probably the best solution.

You're misreading the patch - it just skips the check to see if PCRE
is available without any additional -L / -I options, it will still do
all the checks to see if PCRE is available in /usr/local, /opt/local
and all other common prefixes...

In short, it compiles nginx with PCRE on OSX, without requiring user
to do the "configure" work by hand.

Best regards,
Piotr Sikora


From lawrence.pit at gmail.com  Sat Dec  8 05:15:46 2012
From: lawrence.pit at gmail.com (Lawrence Pit)
Date: Sat, 08 Dec 2012 16:15:46 +1100
Subject: [PATCH] Feature: $msec and $start_msec http variables,
 useful for setting in proxy headers before sending request to backend
 servers so processing and queue times can be measured.
Message-ID: <3E3BD907-FBDD-43E4-96A3-266DCC918CB0@gmail.com>

>> Is there any objection to not output the dot character starting from 
>> the
>> 1.3 release?
>
> We absolutely don't want two different $msec variables, and $msec
> was documented to be holding a value in seconds with a milliseconds
> resolution for ages.

Okay, clear. In that case I'd like to submit this patch for the 1.3.9 
branch:

https://raw.github.com/gist/4238738/919e4b39c5a91d620f7dd005178b6562236cbc2a/nginx139_start_msec.patch

and this patch for the stable-1.2 branch:

https://raw.github.com/gist/4238742/ba715c7ffdd283cb3752c81f73894c949e7ad08f/nginx125_msec.patch


Cheers,
Lawrence

From ru at nginx.com  Sat Dec  8 05:25:38 2012
From: ru at nginx.com (Ruslan Ermilov)
Date: Sat, 8 Dec 2012 09:25:38 +0400
Subject: [PATCH] Fix PCRE detection on OSX.
In-Reply-To: <CADMhe6cVh3=+zsqNp72+C2zOUX8jSH-G3TjOBdk_ec8RsYeHGw@mail.gmail.com>
References: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>
 <20121207222129.GB30441@lo0.su>
 <CADMhe6c91o1J4Exm8pQhbdwUo_L1c24kGjAh5mF=DW6vS9gzRQ@mail.gmail.com>
 <50C275F4.8070005@tvdw.eu>
 <CADMhe6cVh3=+zsqNp72+C2zOUX8jSH-G3TjOBdk_ec8RsYeHGw@mail.gmail.com>
Message-ID: <20121208052538.GA36873@lo0.su>

On Fri, Dec 07, 2012 at 03:15:51PM -0800, Piotr Sikora wrote:
> > That said, I believe that the patch is not ideal. Assuming that PCRE is not
> > installed when building on Mac is bad behavior. Ideally the configuration
> > script would check for the presence of the library. But until that is
> > implemented (if ever), this patch is probably the best solution.
> 
> You're misreading the patch - it just skips the check to see if PCRE
> is available without any additional -L / -I options, it will still do
> all the checks to see if PCRE is available in /usr/local, /opt/local
> and all other common prefixes...

And what happens if Apple fixes it the other day by putting PCRE
headers (back?) under /usr/include?  How do we compile with the
version in /usr/include and /usr/lib then?

> In short, it compiles nginx with PCRE on OSX, without requiring user
> to do the "configure" work by hand.


From matthieu.tourne at gmail.com  Sat Dec  8 06:32:01 2012
From: matthieu.tourne at gmail.com (Matthieu Tourne)
Date: Fri, 7 Dec 2012 22:32:01 -0800
Subject: [PATCH] Fix PCRE detection on OSX.
In-Reply-To: <20121208052538.GA36873@lo0.su>
References: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>
 <20121207222129.GB30441@lo0.su>
 <CADMhe6c91o1J4Exm8pQhbdwUo_L1c24kGjAh5mF=DW6vS9gzRQ@mail.gmail.com>
 <50C275F4.8070005@tvdw.eu>
 <CADMhe6cVh3=+zsqNp72+C2zOUX8jSH-G3TjOBdk_ec8RsYeHGw@mail.gmail.com>
 <20121208052538.GA36873@lo0.su>
Message-ID: <CAAGYmFf0S7UTOgkpeF9eGRg1ssdt314QycgRFTSxdHFOmC6rmg@mail.gmail.com>

On Fri, Dec 7, 2012 at 9:25 PM, Ruslan Ermilov <ru at nginx.com> wrote:

> On Fri, Dec 07, 2012 at 03:15:51PM -0800, Piotr Sikora wrote:
> > > That said, I believe that the patch is not ideal. Assuming that PCRE
> is not
> > > installed when building on Mac is bad behavior. Ideally the
> configuration
> > > script would check for the presence of the library. But until that is
> > > implemented (if ever), this patch is probably the best solution.
> >
> > You're misreading the patch - it just skips the check to see if PCRE
> > is available without any additional -L / -I options, it will still do
> > all the checks to see if PCRE is available in /usr/local, /opt/local
> > and all other common prefixes...
>
> And what happens if Apple fixes it the other day by putting PCRE
> headers (back?) under /usr/include?  How do we compile with the
> version in /usr/include and /usr/lib then?


The version that I posted on this thread should guarantee that headers and
lib are in sync, or the configure itself will error.
Regards,
Matthieu.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121207/92d011fd/attachment.html>

From piotr at cloudflare.com  Sat Dec  8 07:01:18 2012
From: piotr at cloudflare.com (Piotr Sikora)
Date: Fri, 7 Dec 2012 23:01:18 -0800
Subject: [PATCH] Fix PCRE detection on OSX.
In-Reply-To: <20121208052538.GA36873@lo0.su>
References: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>
 <20121207222129.GB30441@lo0.su>
 <CADMhe6c91o1J4Exm8pQhbdwUo_L1c24kGjAh5mF=DW6vS9gzRQ@mail.gmail.com>
 <50C275F4.8070005@tvdw.eu>
 <CADMhe6cVh3=+zsqNp72+C2zOUX8jSH-G3TjOBdk_ec8RsYeHGw@mail.gmail.com>
 <20121208052538.GA36873@lo0.su>
Message-ID: <CADMhe6edXzrzvL56wPDaCVoL19MvvGEop4n6Z-LDG=BkYdX2Qw@mail.gmail.com>

Hey Ruslan,

> And what happens if Apple fixes it the other day by putting PCRE
> headers (back?) under /usr/include?  How do we compile with the
> version in /usr/include and /usr/lib then?

- if [ "$NGX_SYSTEM" = "Darwin" ]; then
+ if [ "$NGX_SYSTEM" = "Darwin" -a "$NGX_RELEASE" < "X.Y.Z" ]; then

where X.Y.Z is the fixed release... But this is really a question for
another day, because OSX ships with libpcre without pcre.h at least
from the OSX 10.6 release (so from 2009).

Best regards,
Piotr Sikora


From dave at daveb.net  Sat Dec  8 21:25:12 2012
From: dave at daveb.net (Dave Bailey)
Date: Sat, 8 Dec 2012 13:25:12 -0800
Subject: nginx + lua and GC latency
Message-ID: <CAJbhXfVPLm+AVTOxL1CtOrkDO__gpzt6BnL+rWsafrERJGEWEw@mail.gmail.com>

Hi,

I am thinking of embedding Lua into nginx for some things where I need a
little more flexibility than my current module implementations allow.
Before I invest the time into this, I was wondering if any of you have
looked into the LuaJit GC latency.  Mike Pall has a wiki page outlining his
plans for a new GC in LuaJIT 3.0:

http://wiki.luajit.org/New-Garbage-Collector

It sounds like he is not too thrilled with the performance of the current
(2.0) LuaJit GC, but I don't know if this means it's something I would need
to worry about, or if the latency of the current GC is fine.  Has anyone
done any profiling to measure this on nginx under "real" load (e.g. 1k-2k
req/s and 100-200+ Mbits/s bandwidth per worker, or somewhere in that
range)?  I like the flexibility that embedding LuaJit would give me, but
can't really afford GC pauses unless I know the latency is going to stay
below some reasonable value (maybe in the realm of 100 microseconds).

I am mainly interested in relatively simple Lua operations for now, such as
customized URL routing and other things that one could expect to be done
once per request.  Any info would be much appreciated.  Thanks-

-dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121208/901c2715/attachment.html>

From piotr at cloudflare.com  Sun Dec  9 07:45:17 2012
From: piotr at cloudflare.com (Piotr Sikora)
Date: Sat, 8 Dec 2012 23:45:17 -0800
Subject: [PATCH] Core: fix content handler and location configuration
 mismatch.
In-Reply-To: <20121207162745.GB40452@mdounin.ru>
References: <CADMhe6cDf57_0wDvTt3i9JwuOGC8UTSwpUyToo8UgsE+HgT=zA@mail.gmail.com>
 <20121207162745.GB40452@mdounin.ru>
Message-ID: <CADMhe6fM4gjYUNjHozy56oJp3KEi2WkfT1h_PQ8P3=kR1DdjqQ@mail.gmail.com>

Hey Maxim,

> And what will happen with the configuration specified in the "if
> ($slow) { ... }" as in the config above?

It will get ignored.

> For limit_rate your config will work as it's saved in request by
> ngx_http_update_location_config()

Correct.

> but e.g.
>
>
>     if ($slow) {
>         add_header "X-Accel-Limit-Rate: 10k";
>     }
>
> will silently ignore if() match, which is wrong.

So you're saying that ignoring if() match (by design and only location
configuration, rewrites and setting variables will work just fine) is
worse than crashing worker process? Sorry, but I'm not buying that.

Anyway, even without this patch, location configurations (other than
the last match) are always silently ignored when there is more than a
single if() match, e.g.:

    location / {
        set $pass  1;
        set $slow  1;

        proxy_pass  http://127.0.0.1:8000;

        if ($pass) {
            add_header "X-Passed" "True";
        }

        if ($slow) {
            add_header "X-Accel-Limit-Rate" "10k";
        }
    }

will silently ignore first if() match, which - quoting you - "is
wrong", so this isn't regression, just different behavior.

Alternatively, the current behavior can be retained by inheriting
content handler defined in the location block (if there is any).
Attached patch makes sure that the current behavior doesn't change,
but also takes care of the mentioned SIGSEGV case, in which case it
will use the location configuration from the last matched if()
pseudo-location that had content handler defined instead of
mismatching content handler and location configuration from different
if() pseudo-locations and crashing... Does this make more sense to
you?

Best regards,
Piotr Sikora


Core: fix content handler and location configuration mismatch.

When using exclusive content handlers (proxy_pass, etc.) and "if"
pseudo-locations, content handler was always set to the one from the
last "if" pseudo-location that evaluated to true or to the one from
the location block if "if" pseudo-locations didn't provide any.
At the same time, location configuration was updated for each "if"
pseudo-location that evaluated to true, which resulted in content
handler and location configuration mismatch.

For example, following configuration would result in SIGSEGV, because
"proxy_pass" content handler (set in the "if ($pass)" pseudo-location)
would be called with location configuration from the "if ($slow)"
pseudo-location which doesn't contain any upstream configuration.

    location / {
        set $pass  1;
        set $slow  1;

        if ($pass) {
            proxy_pass  http://127.0.0.1:8000;
        }

        if ($slow) {
            limit_rate  10k;
        }
    }

This patch fixes this issue by keeping track of and restoring location
configuration from which the content handler originated.

Signed-off-by: Piotr Sikora <piotr at cloudflare.com>
---
 src/http/modules/ngx_http_fastcgi_module.c | 2 +-
 src/http/modules/ngx_http_proxy_module.c   | 2 +-
 src/http/modules/ngx_http_scgi_module.c    | 2 +-
 src/http/modules/ngx_http_uwsgi_module.c   | 2 +-
 src/http/ngx_http_core_module.c            | 2 ++
 src/http/ngx_http_request.h                | 1 +
 6 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/src/http/modules/ngx_http_fastcgi_module.c
b/src/http/modules/ngx_http_fastcgi_module.c
index f1917e2..039e9b4 100644
--- a/src/http/modules/ngx_http_fastcgi_module.c
+++ b/src/http/modules/ngx_http_fastcgi_module.c
@@ -2414,7 +2414,7 @@ ngx_http_fastcgi_merge_loc_conf(ngx_conf_t *cf,
void *parent, void *child)

     if (conf->upstream.upstream || conf->fastcgi_lengths) {
         clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);
-        if (clcf->handler == NULL && clcf->lmt_excpt) {
+        if (clcf->handler == NULL) {
             clcf->handler = ngx_http_fastcgi_handler;
         }
     }
diff --git a/src/http/modules/ngx_http_proxy_module.c
b/src/http/modules/ngx_http_proxy_module.c
index 541c59f..6c65970 100644
--- a/src/http/modules/ngx_http_proxy_module.c
+++ b/src/http/modules/ngx_http_proxy_module.c
@@ -2768,7 +2768,7 @@ ngx_http_proxy_merge_loc_conf(ngx_conf_t *cf,
void *parent, void *child)

     if (conf->upstream.upstream || conf->proxy_lengths) {
         clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);
-        if (clcf->handler == NULL && clcf->lmt_excpt) {
+        if (clcf->handler == NULL) {
             clcf->handler = ngx_http_proxy_handler;
             conf->location = prev->location;
         }
diff --git a/src/http/modules/ngx_http_scgi_module.c
b/src/http/modules/ngx_http_scgi_module.c
index 5a3af92..5fdefc0 100644
--- a/src/http/modules/ngx_http_scgi_module.c
+++ b/src/http/modules/ngx_http_scgi_module.c
@@ -1358,7 +1358,7 @@ ngx_http_scgi_merge_loc_conf(ngx_conf_t *cf,
void *parent, void *child)

     if (conf->upstream.upstream || conf->scgi_lengths) {
         clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);
-        if (clcf->handler == NULL && clcf->lmt_excpt) {
+        if (clcf->handler == NULL) {
             clcf->handler = ngx_http_scgi_handler;
         }
     }
diff --git a/src/http/modules/ngx_http_uwsgi_module.c
b/src/http/modules/ngx_http_uwsgi_module.c
index 0c6414c..f2ce278 100644
--- a/src/http/modules/ngx_http_uwsgi_module.c
+++ b/src/http/modules/ngx_http_uwsgi_module.c
@@ -1397,7 +1397,7 @@ ngx_http_uwsgi_merge_loc_conf(ngx_conf_t *cf,
void *parent, void *child)

     if (conf->upstream.upstream || conf->uwsgi_lengths) {
         clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);
-        if (clcf->handler == NULL && clcf->lmt_excpt) {
+        if (clcf->handler == NULL) {
             clcf->handler = ngx_http_uwsgi_handler;
         }
     }
diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
index a7db74b..a9213d2 100644
--- a/src/http/ngx_http_core_module.c
+++ b/src/http/ngx_http_core_module.c
@@ -1400,6 +1400,7 @@ ngx_http_core_content_phase(ngx_http_request_t *r,
     ngx_str_t  path;

     if (r->content_handler) {
+        r->loc_conf = r->content_handler_loc_conf;
         r->write_event_handler = ngx_http_request_empty_handler;
         ngx_http_finalize_request(r, r->content_handler(r));
         return NGX_OK;
@@ -1526,6 +1527,7 @@ ngx_http_update_location_config(ngx_http_request_t *r)

     if (clcf->handler) {
         r->content_handler = clcf->handler;
+        r->content_handler_loc_conf = r->loc_conf;
     }
 }

diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
index f234840..97c9607 100644
--- a/src/http/ngx_http_request.h
+++ b/src/http/ngx_http_request.h
@@ -358,6 +358,7 @@ struct ngx_http_request_s {
     void                            **main_conf;
     void                            **srv_conf;
     void                            **loc_conf;
+    void                            **content_handler_loc_conf;

     ngx_http_event_handler_pt         read_event_handler;
     ngx_http_event_handler_pt         write_event_handler;
-- 
1.8.0.1


From piotr at cloudflare.com  Sun Dec  9 08:29:21 2012
From: piotr at cloudflare.com (Piotr Sikora)
Date: Sun, 9 Dec 2012 00:29:21 -0800
Subject: [PATCH] Core: fix content handler and location configuration
 mismatch.
In-Reply-To: <CADMhe6fM4gjYUNjHozy56oJp3KEi2WkfT1h_PQ8P3=kR1DdjqQ@mail.gmail.com>
References: <CADMhe6cDf57_0wDvTt3i9JwuOGC8UTSwpUyToo8UgsE+HgT=zA@mail.gmail.com>
 <20121207162745.GB40452@mdounin.ru>
 <CADMhe6fM4gjYUNjHozy56oJp3KEi2WkfT1h_PQ8P3=kR1DdjqQ@mail.gmail.com>
Message-ID: <CADMhe6dxW0qv=opOW1fhrhc5F38qRDeG2uj-j+FFHjycbAZXAQ@mail.gmail.com>

Hey Maxim,
the patch I've just sent also fixes the problem reported by agentzh in June:
http://mailman.nginx.org/pipermail/nginx-devel/2012-June/002374.html

Best regards,
Piotr Sikora


From mdounin at mdounin.ru  Mon Dec 10 01:03:48 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 10 Dec 2012 05:03:48 +0400
Subject: [PATCH] Core: fix content handler and location configuration
 mismatch.
In-Reply-To: <CADMhe6fM4gjYUNjHozy56oJp3KEi2WkfT1h_PQ8P3=kR1DdjqQ@mail.gmail.com>
References: <CADMhe6cDf57_0wDvTt3i9JwuOGC8UTSwpUyToo8UgsE+HgT=zA@mail.gmail.com>
 <20121207162745.GB40452@mdounin.ru>
 <CADMhe6fM4gjYUNjHozy56oJp3KEi2WkfT1h_PQ8P3=kR1DdjqQ@mail.gmail.com>
Message-ID: <20121210010348.GH40452@mdounin.ru>

Hello!

On Sat, Dec 08, 2012 at 11:45:17PM -0800, Piotr Sikora wrote:

> Hey Maxim,
> 
> > And what will happen with the configuration specified in the "if
> > ($slow) { ... }" as in the config above?
> 
> It will get ignored.
> 
> > For limit_rate your config will work as it's saved in request by
> > ngx_http_update_location_config()
> 
> Correct.
> 
> > but e.g.
> >
> >
> >     if ($slow) {
> >         add_header "X-Accel-Limit-Rate: 10k";
> >     }
> >
> > will silently ignore if() match, which is wrong.
> 
> So you're saying that ignoring if() match (by design and only location
> configuration, rewrites and setting variables will work just fine) is
> worse than crashing worker process? Sorry, but I'm not buying that.
> 
> Anyway, even without this patch, location configurations (other than
> the last match) are always silently ignored when there is more than a
> single if() match, e.g.:
> 
>     location / {
>         set $pass  1;
>         set $slow  1;
> 
>         proxy_pass  http://127.0.0.1:8000;
> 
>         if ($pass) {
>             add_header "X-Passed" "True";
>         }
> 
>         if ($slow) {
>             add_header "X-Accel-Limit-Rate" "10k";
>         }
>     }
> 
> will silently ignore first if() match, which - quoting you - "is
> wrong", so this isn't regression, just different behavior.

Ignoring non-rewrite-directives in all-but-last matched if() 
is known (and event somewhat documented) behaviour.  While it's 
indeed counter-intuitive (and that's why I specifically outlined it 
as of one the examples of the "if" directive wierdness on the 
http://wiki.nginx.org/IfIsEvil page), it is well defined.

Changing the behaviour to silently ignore arbitrary matched if() 
block is certainly a regression, even compared to a SIGSEGV - 
which is bad, but logged and clearly indicates something is wrong 
(where "something" is unsupported configuration using if() in this 
case).

> Alternatively, the current behavior can be retained by inheriting
> content handler defined in the location block (if there is any).
> Attached patch makes sure that the current behavior doesn't change,
> but also takes care of the mentioned SIGSEGV case, in which case it
> will use the location configuration from the last matched if()
> pseudo-location that had content handler defined instead of
> mismatching content handler and location configuration from different
> if() pseudo-locations and crashing... Does this make more sense to
> you?

No, as it breaks number of configs which rely on content handlers 
not being inherited, e.g.

   location / {
       proxy_pass http://backend;

       location ~ \.jpg$ {
           # static
       }
   }

(Your patch also makes things bad by introducing inheritance for 
some content handlers but not others, but that's another story 
anyway.)

Aproach I would prefer to use to resolve this is already outlined 
at the IfIsEvil page: I would like to keep declarative config and 
imprative rewrite module directives separate, that is, disable all 
non-rewrite-module directives inside if()'s.  That's the way we 
are slowly moving along, in particular, via introducing variables 
support in various places.

If you can't wait and want to fix this particular SIGSIGV, here 
are some possible aproaches I will be happy enough with:

1) Just log error and return 500 if configuration incosistency 
detected instead of dereferencing a NULL pointer.

2) Postpone ngx_http_update_location_config() till the end of the 
script code execution, thus making configs like

    if ($true) { proxy_pass ... }
    if ($true) { add_header 2 ... }

consistent with

    if ($true) { add_header 1 ... }
    if ($true) { add_header 2 ... }

via ignoring anything but the last matched if() block.  This 
aproach needs careful investigation though, as I'm not really sure 
the won't be any undesired side effects (in particular, some 
limit_rate uses will no longer work, but probably we can live with 
it).

3) Introduce inheritance for content handlers into if blocks only 
(similar to limit_except blocks? likely needs cleanup too though), 
and always overwrite r->content_handler in 
ngx_http_update_location_config().  Net effect should be similar 
to (2), and much like (2) it also neeeds careful investigation.

-- 
Maxim Dounin
http://nginx.com/support.html


From mdounin at mdounin.ru  Mon Dec 10 02:37:24 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 10 Dec 2012 06:37:24 +0400
Subject: [PATCH] Fix PCRE detection on OSX.
In-Reply-To: <CADMhe6edXzrzvL56wPDaCVoL19MvvGEop4n6Z-LDG=BkYdX2Qw@mail.gmail.com>
References: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>
 <20121207222129.GB30441@lo0.su>
 <CADMhe6c91o1J4Exm8pQhbdwUo_L1c24kGjAh5mF=DW6vS9gzRQ@mail.gmail.com>
 <50C275F4.8070005@tvdw.eu>
 <CADMhe6cVh3=+zsqNp72+C2zOUX8jSH-G3TjOBdk_ec8RsYeHGw@mail.gmail.com>
 <20121208052538.GA36873@lo0.su>
 <CADMhe6edXzrzvL56wPDaCVoL19MvvGEop4n6Z-LDG=BkYdX2Qw@mail.gmail.com>
Message-ID: <20121210023724.GI40452@mdounin.ru>

Hello!

On Fri, Dec 07, 2012 at 11:01:18PM -0800, Piotr Sikora wrote:

> Hey Ruslan,
> 
> > And what happens if Apple fixes it the other day by putting PCRE
> > headers (back?) under /usr/include?  How do we compile with the
> > version in /usr/include and /usr/lib then?
> 
> - if [ "$NGX_SYSTEM" = "Darwin" ]; then
> + if [ "$NGX_SYSTEM" = "Darwin" -a "$NGX_RELEASE" < "X.Y.Z" ]; then
> 
> where X.Y.Z is the fixed release... But this is really a question for
> another day, because OSX ships with libpcre without pcre.h at least
> from the OSX 10.6 release (so from 2009).

The problem with Mac OS X is not with libpcre withou pcre.h, but with 
broken toolchain which prefers headers from /usr/local/include but 
libraries from /usr/lib by default.

Here is a test:

#!/bin/sh -e

# in /usr/ {include, lib}:
# bad foo.h, good libfoo.a

# in /usr/local/ {include, lib}
# good foo.h, bad libfoo.a

CC=cc
AR=ar

echo '#error "broken foo.h in /usr/include"' > foo.h
install foo.h /usr/include/

echo 'char *foo() { return "broken libfoo.a from /usr/lib"; }' > foo.c
${CC} -c -o foo.o foo.c
${AR} -rs libfoo.a foo.o
install libfoo.a /usr/lib/

echo '#warning "good foo.h in /usr/local/include"' > foo.h
echo 'char *foo();' >> foo.h
install foo.h /usr/local/include/

echo 'char *foo() { return "good libfoo.a from /usr/local/"; }' > foo.c
${CC} -c -o foo.o foo.c
${AR} -rs libfoo.a foo.o
install libfoo.a /usr/local/lib/

cat << END > test.c

#include <foo.h>
#include <stdio.h>

int
main(void) {
    printf("%s\n", foo());
}

END

${CC} -o test -l foo test.c
./test

On Mac OS X 10.7.5 with Xcode 4.5.2 it results in:

# ./test.sh
In file included from test.c:2:
/usr/local/include/foo.h:1:2: warning: "good foo.h in /usr/local/include" [-W#warnings]
#warning "good foo.h in /usr/local/include"
 ^
1 warning generated.
broken libfoo.a from /usr/lib

That is, header file from /usr/local/include was used, but library 
from /usr/lib.  This is the clear incosistency in build toolchain 
which will result in various build problems till fixed.

Easy fix is to avoid using /usr/local/ till the problem is fixed.  
Alternatively, one may try to provide appropriate compiler 
arguments to fix the toolchain.  Another alternative would be to 
switch to a different toolchain (e.g. MacPort's provided one).

I don't think nginx should try to do anything with it as it's 
indeed not our problem.  But if we are going to, it probably 
should be something better that just not searching for PCRE in 
default compiler search paths, as it e.g. breaks compilation with 
search paths explicitly configured by a user via ./configure 
arguments.

-- 
Maxim Dounin
http://nginx.com/support.html


From dirk.feytons at gmail.com  Mon Dec 10 09:00:34 2012
From: dirk.feytons at gmail.com (Dirk Feytons)
Date: Mon, 10 Dec 2012 10:00:34 +0100
Subject: nginx + lua and GC latency
In-Reply-To: <CAJbhXfVPLm+AVTOxL1CtOrkDO__gpzt6BnL+rWsafrERJGEWEw@mail.gmail.com>
References: <CAJbhXfVPLm+AVTOxL1CtOrkDO__gpzt6BnL+rWsafrERJGEWEw@mail.gmail.com>
Message-ID: <CAEdzpzeHUWNHcgTNZEHQRhCvnQsckDqGOqJa+vwnX9e+UpwsKg@mail.gmail.com>

Are you talking about http://wiki.nginx.org/HttpLuaModule ?
>From what I've heard about ngx_lua people are using it in large production
environments and I haven't heard them complain about GC pauses (at least
not publicly on the mailing list). But that doesn't mean that Mike is
wrong. The Lua GC is known to not scale that well when the number of
objects increases. Whether you hit those limits can only be determined by
actually trying :)


Dirk F.


On Sat, Dec 8, 2012 at 10:25 PM, Dave Bailey <dave at daveb.net> wrote:

> Hi,
>
> I am thinking of embedding Lua into nginx for some things where I need a
> little more flexibility than my current module implementations allow.
> Before I invest the time into this, I was wondering if any of you have
> looked into the LuaJit GC latency.  Mike Pall has a wiki page outlining his
> plans for a new GC in LuaJIT 3.0:
>
> http://wiki.luajit.org/New-Garbage-Collector
>
> It sounds like he is not too thrilled with the performance of the current
> (2.0) LuaJit GC, but I don't know if this means it's something I would need
> to worry about, or if the latency of the current GC is fine.  Has anyone
> done any profiling to measure this on nginx under "real" load (e.g. 1k-2k
> req/s and 100-200+ Mbits/s bandwidth per worker, or somewhere in that
> range)?  I like the flexibility that embedding LuaJit would give me, but
> can't really afford GC pauses unless I know the latency is going to stay
> below some reasonable value (maybe in the realm of 100 microseconds).
>
> I am mainly interested in relatively simple Lua operations for now, such
> as customized URL routing and other things that one could expect to be done
> once per request.  Any info would be much appreciated.  Thanks-
>
> -dave
>
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121210/21584f61/attachment.html>

From ru at nginx.com  Mon Dec 10 10:31:42 2012
From: ru at nginx.com (Ruslan Ermilov)
Date: Mon, 10 Dec 2012 14:31:42 +0400
Subject: [PATCH] Fix PCRE detection on OSX.
In-Reply-To: <20121210023724.GI40452@mdounin.ru>
References: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>
 <20121207222129.GB30441@lo0.su>
 <CADMhe6c91o1J4Exm8pQhbdwUo_L1c24kGjAh5mF=DW6vS9gzRQ@mail.gmail.com>
 <50C275F4.8070005@tvdw.eu>
 <CADMhe6cVh3=+zsqNp72+C2zOUX8jSH-G3TjOBdk_ec8RsYeHGw@mail.gmail.com>
 <20121208052538.GA36873@lo0.su>
 <CADMhe6edXzrzvL56wPDaCVoL19MvvGEop4n6Z-LDG=BkYdX2Qw@mail.gmail.com>
 <20121210023724.GI40452@mdounin.ru>
Message-ID: <20121210103142.GA71576@lo0.su>

On Mon, Dec 10, 2012 at 06:37:24AM +0400, Maxim Dounin wrote:
> The problem with Mac OS X is not with libpcre withou pcre.h, but with 
> broken toolchain which prefers headers from /usr/local/include but 
> libraries from /usr/lib by default.
[...]
> That is, header file from /usr/local/include was used, but library 
> from /usr/lib.  This is the clear incosistency in build toolchain 
> which will result in various build problems till fixed.

> Easy fix is to avoid using /usr/local/ till the problem is fixed.  
> Alternatively, one may try to provide appropriate compiler 
> arguments to fix the toolchain.  Another alternative would be to 
> switch to a different toolchain (e.g. MacPort's provided one).

Actually there are two problems.

The problem with broken toolchain isn't unique to OS X.  I've tried
it on at least Gentoo, CentOS, Debian, FreeBSD ports compilers, and
they all (mis)behave similar to OS X.

The problem with the broken toolchain is quite easy to fix, adding
"-isystem /usr/include" will result in /usr/include preferred over
/usr/local/lib (works on gcc, llvm-gcc and clang).  Depending on
the actual compiler used, the output might look like this:

: $ cc -isystem /usr/include -v -c test.c
[...]
: #include "..." search starts here:
: #include <...> search starts here:
:  /usr/include
:  /usr/llvm-gcc-4.2/bin/../lib/gcc/i686-apple-darwin11/4.2.1/include
:  /usr/local/include
:  /Developer/usr/llvm-gcc-4.2/lib/gcc/i686-apple-darwin11/4.2.1/include
:  /System/Library/Frameworks (framework directory)
:  /Library/Frameworks (framework directory)
: End of search list.

The second problem is missing /usr/include/pcre.h on OS X.

Fixing the first problem doesn't help to fix the second: the missing
/usr/include/pcre.h in OS X will still result in nginx (or any other
software using the default compiler and linker with default flags)
attempting to use /usr/local/include/pcre.h and /usr/lib/libpcre,
even if the compiler and linker are consistent and prefer
/usr/{include,lib}.

So I tent to repeat: avoid using /usr/local if at all possible.
(Or at least don't install libraries into /usr/local that also
exist in /usr/{include,lib}.)

A correct fix seems to be to exclude /usr/local/{include,lib} from
the default search paths (like FreeBSD does), but I'm not aware of
an easy way to do it.  And again, this is not unique to only OS X,
nor is nginx specific.

An alternative approach that prefers /usr/local/{include,lib} over
/usr/{include,lib} (as simple as -L/usr/local/lib) seems like an
overkill to me just to work around a misinstalled PCRE on OS X.

> I don't think nginx should try to do anything with it as it's 
> indeed not our problem.  But if we are going to, it probably 
> should be something better that just not searching for PCRE in 
> default compiler search paths, as it e.g. breaks compilation with 
> search paths explicitly configured by a user via ./configure 
> arguments.

Also note that the default feature test in auto/lib/pcre/conf
is not about "PCRE library in /usr/include", it's about the
"PCRE library in default locations", whatever it means, and
depends on the toolchain used.

Disabling the default feature test on OS X, as suggested, is bad
because while it adds a workaround for one of the possible setups,
it breaks the others.

Imagine there is a packaging system that installs stuff into
/my/prefix and ships with the toolchain that looks for headers
and libraries in /my/prefix/{include,lib}.  This system also
ships with the nginx port that configures nginx with the
--with-cc= argument to point to the compiler from the
toolchain (or sets up $PATH in environment accordingly).

A correct workaround should ensure that we don't try to use libpcre
from /usr/lib, because we know it's misinstalled, but shouldn't
otherwise pessimize other valid cases.  Since we don't know how
exactly the toolchain used is configured, we can merely delay the
default PCRE test on OS X:

%%%
Index: auto/lib/pcre/conf
===================================================================
--- auto/lib/pcre/conf	(revision 4948)
+++ auto/lib/pcre/conf	(working copy)
@@ -101,7 +101,17 @@
         ngx_feature_test="pcre *re;
                           re = pcre_compile(NULL, 0, NULL, 0, NULL);
                           if (re == NULL) return 1"
-        . auto/feature
+
+        if [ "$NGX_SYSTEM" = "Darwin" ]; then
+            # Apple provides libpcre in /usr/lib but no /usr/include/pcre.h.
+            # This may interfere with libpcre installed into /usr/local if
+            # system compiler is used.  To work around this, we delay the
+            # default test until after we tested /usr/local.
+            ngx_found=no
+
+        else
+            . auto/feature
+        fi
 
         if [ $ngx_found = no ]; then
 
@@ -119,6 +129,14 @@
             . auto/feature
         fi
 
+        if [ $ngx_found = no -a "$NGX_SYSTEM" = "Darwin" ]; then
+            ngx_feature="PCRE library"
+            ngx_feature_path=
+            ngx_feature_libs="-lpcre"
+
+            . auto/feature
+        fi
+
         if [ $ngx_found = no ]; then
 
             # RedHat RPM, Solaris package
%%%


From ru at nginx.com  Mon Dec 10 13:11:11 2012
From: ru at nginx.com (ru at nginx.com)
Date: Mon, 10 Dec 2012 13:11:11 +0000
Subject: [nginx] svn commit:  r4949 - trunk/src/http/modules
Message-ID: <20121210131111.F339E3FA0D4@mail.nginx.com>

Author: ru
Date: 2012-12-10 13:11:11 +0000 (Mon, 10 Dec 2012)
New Revision: 4949
URL: http://trac.nginx.org/nginx/changeset/4949/nginx

Log:
The "auth_basic" directive gained support of variables.


Modified:
   trunk/src/http/modules/ngx_http_auth_basic_module.c

Modified: trunk/src/http/modules/ngx_http_auth_basic_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_auth_basic_module.c	2012-12-06 23:03:53 UTC (rev 4948)
+++ trunk/src/http/modules/ngx_http_auth_basic_module.c	2012-12-10 13:11:11 UTC (rev 4949)
@@ -20,8 +20,8 @@
 
 
 typedef struct {
-    ngx_str_t                 realm;
-    ngx_http_complex_value_t  user_file;
+    ngx_http_complex_value_t  *realm;
+    ngx_http_complex_value_t   user_file;
 } ngx_http_auth_basic_loc_conf_t;
 
 
@@ -35,22 +35,19 @@
 static char *ngx_http_auth_basic_merge_loc_conf(ngx_conf_t *cf,
     void *parent, void *child);
 static ngx_int_t ngx_http_auth_basic_init(ngx_conf_t *cf);
-static char *ngx_http_auth_basic(ngx_conf_t *cf, void *post, void *data);
 static char *ngx_http_auth_basic_user_file(ngx_conf_t *cf, ngx_command_t *cmd,
     void *conf);
 
 
-static ngx_conf_post_handler_pt  ngx_http_auth_basic_p = ngx_http_auth_basic;
-
 static ngx_command_t  ngx_http_auth_basic_commands[] = {
 
     { ngx_string("auth_basic"),
       NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LMT_CONF
                         |NGX_CONF_TAKE1,
-      ngx_conf_set_str_slot,
+      ngx_http_set_complex_value_slot,
       NGX_HTTP_LOC_CONF_OFFSET,
       offsetof(ngx_http_auth_basic_loc_conf_t, realm),
-      &ngx_http_auth_basic_p },
+      NULL },
 
     { ngx_string("auth_basic_user_file"),
       NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LMT_CONF
@@ -103,7 +100,7 @@
     ngx_fd_t                         fd;
     ngx_int_t                        rc;
     ngx_err_t                        err;
-    ngx_str_t                        pwd, user_file;
+    ngx_str_t                        pwd, realm, user_file;
     ngx_uint_t                       i, level, login, left, passwd;
     ngx_file_t                       file;
     ngx_http_auth_basic_ctx_t       *ctx;
@@ -117,15 +114,23 @@
 
     alcf = ngx_http_get_module_loc_conf(r, ngx_http_auth_basic_module);
 
-    if (alcf->realm.len == 0 || alcf->user_file.value.data == NULL) {
+    if (alcf->realm == NULL || alcf->user_file.value.data == NULL) {
         return NGX_DECLINED;
     }
 
+    if (ngx_http_complex_value(r, alcf->realm, &realm) != NGX_OK) {
+        return NGX_ERROR;
+    }
+
+    if (realm.len == 3 && ngx_strncmp(realm.data, "off", 3) == 0) {
+        return NGX_DECLINED;
+    }
+
     ctx = ngx_http_get_module_ctx(r, ngx_http_auth_basic_module);
 
     if (ctx) {
         return ngx_http_auth_basic_crypt_handler(r, ctx, &ctx->passwd,
-                                                 &alcf->realm);
+                                                 &realm);
     }
 
     rc = ngx_http_auth_basic_user(r);
@@ -135,7 +140,7 @@
         ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                       "no user/password was provided for basic authentication");
 
-        return ngx_http_auth_basic_set_realm(r, &alcf->realm);
+        return ngx_http_auth_basic_set_realm(r, &realm);
     }
 
     if (rc == NGX_ERROR) {
@@ -233,7 +238,7 @@
                     pwd.data = &buf[passwd];
 
                     return ngx_http_auth_basic_crypt_handler(r, NULL, &pwd,
-                                                             &alcf->realm);
+                                                             &realm);
                 }
 
                 break;
@@ -271,14 +276,14 @@
 
         ngx_cpystrn(pwd.data, &buf[passwd], pwd.len + 1);
 
-        return ngx_http_auth_basic_crypt_handler(r, NULL, &pwd, &alcf->realm);
+        return ngx_http_auth_basic_crypt_handler(r, NULL, &pwd, &realm);
     }
 
     ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                   "user \"%V\" was not found in \"%V\"",
                   &r->headers_in.user, &user_file);
 
-    return ngx_http_auth_basic_set_realm(r, &alcf->realm);
+    return ngx_http_auth_basic_set_realm(r, &realm);
 }
 
 
@@ -344,14 +349,29 @@
 static ngx_int_t
 ngx_http_auth_basic_set_realm(ngx_http_request_t *r, ngx_str_t *realm)
 {
+    size_t   len;
+    u_char  *basic, *p;
+
     r->headers_out.www_authenticate = ngx_list_push(&r->headers_out.headers);
     if (r->headers_out.www_authenticate == NULL) {
         return NGX_HTTP_INTERNAL_SERVER_ERROR;
     }
 
+    len = sizeof("Basic realm=\"\"") - 1 + realm->len;
+
+    basic = ngx_pnalloc(r->pool, len);
+    if (basic == NULL) {
+        return NGX_HTTP_INTERNAL_SERVER_ERROR;
+    }
+
+    p = ngx_cpymem(basic, "Basic realm=\"", sizeof("Basic realm=\"") - 1);
+    p = ngx_cpymem(p, realm->data, realm->len);
+    *p = '"';
+
     r->headers_out.www_authenticate->hash = 1;
     ngx_str_set(&r->headers_out.www_authenticate->key, "WWW-Authenticate");
-    r->headers_out.www_authenticate->value = *realm;
+    r->headers_out.www_authenticate->value.data = basic;
+    r->headers_out.www_authenticate->value.len = len;
 
     return NGX_HTTP_UNAUTHORIZED;
 }
@@ -386,7 +406,7 @@
     ngx_http_auth_basic_loc_conf_t  *prev = parent;
     ngx_http_auth_basic_loc_conf_t  *conf = child;
 
-    if (conf->realm.data == NULL) {
+    if (conf->realm == NULL) {
         conf->realm = prev->realm;
     }
 
@@ -418,37 +438,6 @@
 
 
 static char *
-ngx_http_auth_basic(ngx_conf_t *cf, void *post, void *data)
-{
-    ngx_str_t  *realm = data;
-
-    size_t   len;
-    u_char  *basic, *p;
-
-    if (ngx_strcmp(realm->data, "off") == 0) {
-        ngx_str_set(realm, "");
-        return NGX_CONF_OK;
-    }
-
-    len = sizeof("Basic realm=\"") - 1 + realm->len + 1;
-
-    basic = ngx_pnalloc(cf->pool, len);
-    if (basic == NULL) {
-        return NGX_CONF_ERROR;
-    }
-
-    p = ngx_cpymem(basic, "Basic realm=\"", sizeof("Basic realm=\"") - 1);
-    p = ngx_cpymem(p, realm->data, realm->len);
-    *p = '"';
-
-    realm->len = len;
-    realm->data = basic;
-
-    return NGX_CONF_OK;
-}
-
-
-static char *
 ngx_http_auth_basic_user_file(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
 {
     ngx_http_auth_basic_loc_conf_t *alcf = conf;


From mdounin at mdounin.ru  Mon Dec 10 14:50:00 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 10 Dec 2012 18:50:00 +0400
Subject: [PATCH] Fix PCRE detection on OSX.
In-Reply-To: <20121210103142.GA71576@lo0.su>
References: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>
 <20121207222129.GB30441@lo0.su>
 <CADMhe6c91o1J4Exm8pQhbdwUo_L1c24kGjAh5mF=DW6vS9gzRQ@mail.gmail.com>
 <50C275F4.8070005@tvdw.eu>
 <CADMhe6cVh3=+zsqNp72+C2zOUX8jSH-G3TjOBdk_ec8RsYeHGw@mail.gmail.com>
 <20121208052538.GA36873@lo0.su>
 <CADMhe6edXzrzvL56wPDaCVoL19MvvGEop4n6Z-LDG=BkYdX2Qw@mail.gmail.com>
 <20121210023724.GI40452@mdounin.ru> <20121210103142.GA71576@lo0.su>
Message-ID: <20121210145000.GL40452@mdounin.ru>

Hello!

On Mon, Dec 10, 2012 at 02:31:42PM +0400, Ruslan Ermilov wrote:

[...]

> Fixing the first problem doesn't help to fix the second: the missing
> /usr/include/pcre.h in OS X will still result in nginx (or any other
> software using the default compiler and linker with default flags)
> attempting to use /usr/local/include/pcre.h and /usr/lib/libpcre,
> even if the compiler and linker are consistent and prefer
> /usr/{include,lib}.

Consisten behaviour would be either to prefer /usr/local/{include,lib} 
or to don't search them for anything by default.  This way both problems 
observed on Mac OS X would be resolved.

> So I tent to repeat: avoid using /usr/local if at all possible.
> (Or at least don't install libraries into /usr/local that also
> exist in /usr/{include,lib}.)
> 
> A correct fix seems to be to exclude /usr/local/{include,lib} from
> the default search paths (like FreeBSD does), but I'm not aware of
> an easy way to do it.  And again, this is not unique to only OS X,
> nor is nginx specific.

Just a side note: it looks like --with-local-prefix=/nonexistent 
during gcc configure is a good way to do it.

Another side note: unfortunately, I was wrong claiming MacPorts 
toolchain is good.  It has the same problem too, but with MacPorts 
installed in /opt/local it uses /opt/local/include as headers 
search path.

[...]

> A correct workaround should ensure that we don't try to use libpcre
> from /usr/lib, because we know it's misinstalled, but shouldn't
> otherwise pessimize other valid cases.  Since we don't know how
> exactly the toolchain used is configured, we can merely delay the
> default PCRE test on OS X:

I don't like the idea of delayed test either, for a reasons 
similar to ones you don't like removing default search paths 
testing: one should be able to supply appropriate search paths 
which will be tested before we'll try to guess anything.

As a workaround for a particular build problem with PCRE on 
Mac OS X something as simple as

--- a/auto/lib/pcre/conf	Mon Nov 26 18:01:49 2012 +0000
+++ b/auto/lib/pcre/conf	Mon Dec 10 18:21:53 2012 +0400
@@ -172,6 +172,7 @@ else
             ngx_feature="PCRE JIT support"
             ngx_feature_name="NGX_HAVE_PCRE_JIT"
             ngx_feature_test="int jit = 0;
+                              pcre_free_study(NULL);
                               pcre_config(PCRE_CONFIG_JIT, &jit);
                               if (jit != 1) return 1;"
             . auto/feature

probably will be good enough.  It will prevent build failures 
by checking if the code we are going to compile will compile.

(Yes, I understand that it won't resolve the root cause, i.e. the 
library vs. headers mismatch.  And yes, I understand that it will 
make PCRE JIT unavailable in some cases.  I don't think we care 
though.)

-- 
Maxim Dounin
http://nginx.com/support.html


From ru at nginx.com  Mon Dec 10 15:07:28 2012
From: ru at nginx.com (Ruslan Ermilov)
Date: Mon, 10 Dec 2012 19:07:28 +0400
Subject: [PATCH] Fix PCRE detection on OSX.
In-Reply-To: <20121210145000.GL40452@mdounin.ru>
References: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>
 <20121207222129.GB30441@lo0.su>
 <CADMhe6c91o1J4Exm8pQhbdwUo_L1c24kGjAh5mF=DW6vS9gzRQ@mail.gmail.com>
 <50C275F4.8070005@tvdw.eu>
 <CADMhe6cVh3=+zsqNp72+C2zOUX8jSH-G3TjOBdk_ec8RsYeHGw@mail.gmail.com>
 <20121208052538.GA36873@lo0.su>
 <CADMhe6edXzrzvL56wPDaCVoL19MvvGEop4n6Z-LDG=BkYdX2Qw@mail.gmail.com>
 <20121210023724.GI40452@mdounin.ru> <20121210103142.GA71576@lo0.su>
 <20121210145000.GL40452@mdounin.ru>
Message-ID: <20121210150728.GC82873@lo0.su>

On Mon, Dec 10, 2012 at 06:50:00PM +0400, Maxim Dounin wrote:
> Hello!
> 
> On Mon, Dec 10, 2012 at 02:31:42PM +0400, Ruslan Ermilov wrote:
> 
> [...]
> 
> > Fixing the first problem doesn't help to fix the second: the missing
> > /usr/include/pcre.h in OS X will still result in nginx (or any other
> > software using the default compiler and linker with default flags)
> > attempting to use /usr/local/include/pcre.h and /usr/lib/libpcre,
> > even if the compiler and linker are consistent and prefer
> > /usr/{include,lib}.
> 
> Consisten behaviour would be either to prefer /usr/local/{include,lib} 
> or to don't search them for anything by default.  This way both problems 
> observed on Mac OS X would be resolved.
> 
> > So I tent to repeat: avoid using /usr/local if at all possible.
> > (Or at least don't install libraries into /usr/local that also
> > exist in /usr/{include,lib}.)
> > 
> > A correct fix seems to be to exclude /usr/local/{include,lib} from
> > the default search paths (like FreeBSD does), but I'm not aware of
> > an easy way to do it.  And again, this is not unique to only OS X,
> > nor is nginx specific.
> 
> Just a side note: it looks like --with-local-prefix=/nonexistent 
> during gcc configure is a good way to do it.
> 
> Another side note: unfortunately, I was wrong claiming MacPorts 
> toolchain is good.  It has the same problem too, but with MacPorts 
> installed in /opt/local it uses /opt/local/include as headers 
> search path.
> 
> [...]
> 
> > A correct workaround should ensure that we don't try to use libpcre
> > from /usr/lib, because we know it's misinstalled, but shouldn't
> > otherwise pessimize other valid cases.  Since we don't know how
> > exactly the toolchain used is configured, we can merely delay the
> > default PCRE test on OS X:
> 
> I don't like the idea of delayed test either, for a reasons 
> similar to ones you don't like removing default search paths 
> testing: one should be able to supply appropriate search paths 
> which will be tested before we'll try to guess anything.
> 
> As a workaround for a particular build problem with PCRE on 
> Mac OS X something as simple as
> 
> --- a/auto/lib/pcre/conf	Mon Nov 26 18:01:49 2012 +0000
> +++ b/auto/lib/pcre/conf	Mon Dec 10 18:21:53 2012 +0400
> @@ -172,6 +172,7 @@ else
>              ngx_feature="PCRE JIT support"
>              ngx_feature_name="NGX_HAVE_PCRE_JIT"
>              ngx_feature_test="int jit = 0;
> +                              pcre_free_study(NULL);
>                                pcre_config(PCRE_CONFIG_JIT, &jit);
>                                if (jit != 1) return 1;"
>              . auto/feature
> 
> probably will be good enough.  It will prevent build failures 
> by checking if the code we are going to compile will compile.
> 
> (Yes, I understand that it won't resolve the root cause, i.e. the 
> library vs. headers mismatch.  And yes, I understand that it will 
> make PCRE JIT unavailable in some cases.  I don't think we care 
> though.)

I think we should commit this patch because it's correct.  :)

It will also fix the problem for those who can't fix it yourself,
with the small price by those who install stuff under /usr/local.


From mdounin at mdounin.ru  Mon Dec 10 15:43:41 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon, 10 Dec 2012 15:43:41 +0000
Subject: [nginx] svn commit: r4950 - in branches/stable-1.2/src: core
 http/modules/perl
Message-ID: <20121210154342.15B5A3F9C45@mail.nginx.com>

Author: mdounin
Date: 2012-12-10 15:43:41 +0000 (Mon, 10 Dec 2012)
New Revision: 4950
URL: http://trac.nginx.org/nginx/changeset/4950/nginx

Log:
Version bump.


Modified:
   branches/stable-1.2/src/core/nginx.h
   branches/stable-1.2/src/http/modules/perl/nginx.pm

Modified: branches/stable-1.2/src/core/nginx.h
===================================================================
--- branches/stable-1.2/src/core/nginx.h	2012-12-10 13:11:11 UTC (rev 4949)
+++ branches/stable-1.2/src/core/nginx.h	2012-12-10 15:43:41 UTC (rev 4950)
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1002005
-#define NGINX_VERSION      "1.2.5"
+#define nginx_version      1002006
+#define NGINX_VERSION      "1.2.6"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #define NGINX_VAR          "NGINX"

Modified: branches/stable-1.2/src/http/modules/perl/nginx.pm
===================================================================
--- branches/stable-1.2/src/http/modules/perl/nginx.pm	2012-12-10 13:11:11 UTC (rev 4949)
+++ branches/stable-1.2/src/http/modules/perl/nginx.pm	2012-12-10 15:43:41 UTC (rev 4950)
@@ -50,7 +50,7 @@
     HTTP_INSUFFICIENT_STORAGE
 );
 
-our $VERSION = '1.2.5';
+our $VERSION = '1.2.6';
 
 require XSLoader;
 XSLoader::load('nginx', $VERSION);


From mdounin at mdounin.ru  Mon Dec 10 16:00:55 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon, 10 Dec 2012 16:00:55 +0000
Subject: [nginx] svn commit:  r4951 - in branches/stable-1.2: . src/os/unix
Message-ID: <20121210160055.A3A1E3FA128@mail.nginx.com>

Author: mdounin
Date: 2012-12-10 16:00:55 +0000 (Mon, 10 Dec 2012)
New Revision: 4951
URL: http://trac.nginx.org/nginx/changeset/4951/nginx

Log:
Merge of r4913: fixed CPU affinity on respawn of dead workers.

Worker processes are now made aware of their sequential number needed
to select CPU affinity mask.  This replaces a workaround from r4865.


Modified:
   branches/stable-1.2/
   branches/stable-1.2/src/os/unix/ngx_process_cycle.c

Index: branches/stable-1.2
===================================================================
--- branches/stable-1.2	2012-12-10 15:43:41 UTC (rev 4950)
+++ branches/stable-1.2	2012-12-10 16:00:55 UTC (rev 4951)

Property changes on: branches/stable-1.2
___________________________________________________________________
Modified: svn:mergeinfo
## -1 +1 ##
-/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896
+/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913
\ No newline at end of property
Modified: branches/stable-1.2/src/os/unix/ngx_process_cycle.c
===================================================================
--- branches/stable-1.2/src/os/unix/ngx_process_cycle.c	2012-12-10 15:43:41 UTC (rev 4950)
+++ branches/stable-1.2/src/os/unix/ngx_process_cycle.c	2012-12-10 16:00:55 UTC (rev 4951)
@@ -20,7 +20,7 @@
 static ngx_uint_t ngx_reap_children(ngx_cycle_t *cycle);
 static void ngx_master_process_exit(ngx_cycle_t *cycle);
 static void ngx_worker_process_cycle(ngx_cycle_t *cycle, void *data);
-static void ngx_worker_process_init(ngx_cycle_t *cycle, ngx_uint_t priority);
+static void ngx_worker_process_init(ngx_cycle_t *cycle, ngx_int_t worker);
 static void ngx_worker_process_exit(ngx_cycle_t *cycle);
 static void ngx_channel_handler(ngx_event_t *ev);
 #if (NGX_THREADS)
@@ -62,7 +62,6 @@
 #endif
 
 
-uint64_t       cpu_affinity;
 static u_char  master_process[] = "master process";
 
 
@@ -360,19 +359,15 @@
 
     for (i = 0; i < n; i++) {
 
-        cpu_affinity = ngx_get_cpu_affinity(i);
+        ngx_spawn_process(cycle, ngx_worker_process_cycle,
+                          (void *) (intptr_t) i, "worker process", type);
 
-        ngx_spawn_process(cycle, ngx_worker_process_cycle, NULL,
-                          "worker process", type);
-
         ch.pid = ngx_processes[ngx_process_slot].pid;
         ch.slot = ngx_process_slot;
         ch.fd = ngx_processes[ngx_process_slot].channel[0];
 
         ngx_pass_open_channel(cycle, &ch);
     }
-
-    cpu_affinity = 0;
 }
 
 
@@ -726,12 +721,14 @@
 static void
 ngx_worker_process_cycle(ngx_cycle_t *cycle, void *data)
 {
+    ngx_int_t worker = (intptr_t) data;
+
     ngx_uint_t         i;
     ngx_connection_t  *c;
 
     ngx_process = NGX_PROCESS_WORKER;
 
-    ngx_worker_process_init(cycle, 1);
+    ngx_worker_process_init(cycle, worker);
 
     ngx_setproctitle("worker process");
 
@@ -837,9 +834,10 @@
 
 
 static void
-ngx_worker_process_init(ngx_cycle_t *cycle, ngx_uint_t priority)
+ngx_worker_process_init(ngx_cycle_t *cycle, ngx_int_t worker)
 {
     sigset_t          set;
+    uint64_t          cpu_affinity;
     ngx_int_t         n;
     ngx_uint_t        i;
     struct rlimit     rlmt;
@@ -853,7 +851,7 @@
 
     ccf = (ngx_core_conf_t *) ngx_get_conf(cycle->conf_ctx, ngx_core_module);
 
-    if (priority && ccf->priority != 0) {
+    if (worker >= 0 && ccf->priority != 0) {
         if (setpriority(PRIO_PROCESS, 0, ccf->priority) == -1) {
             ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno,
                           "setpriority(%d) failed", ccf->priority);
@@ -917,8 +915,12 @@
         }
     }
 
-    if (cpu_affinity) {
-        ngx_setaffinity(cpu_affinity, cycle->log);
+    if (worker >= 0) {
+        cpu_affinity = ngx_get_cpu_affinity(worker);
+
+        if (cpu_affinity) {
+            ngx_setaffinity(cpu_affinity, cycle->log);
+        }
     }
 
 #if (NGX_HAVE_PR_SET_DUMPABLE)
@@ -1298,7 +1300,7 @@
 
     ngx_process = NGX_PROCESS_HELPER;
 
-    ngx_worker_process_init(cycle, 0);
+    ngx_worker_process_init(cycle, -1);
 
     ngx_close_listening_sockets(cycle);
 


From mdounin at mdounin.ru  Mon Dec 10 16:03:56 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon, 10 Dec 2012 16:03:56 +0000
Subject: [nginx] svn commit:  r4952 - in branches/stable-1.2: . src/http
Message-ID: <20121210160356.8DA6D3F9FAE@mail.nginx.com>

Author: mdounin
Date: 2012-12-10 16:03:56 +0000 (Mon, 10 Dec 2012)
New Revision: 4952
URL: http://trac.nginx.org/nginx/changeset/4952/nginx

Log:
Merge of r4914: variables $request_time and $msec.

Log module counterparts are preserved for efficiency.


Modified:
   branches/stable-1.2/
   branches/stable-1.2/src/http/ngx_http_variables.c

Index: branches/stable-1.2
===================================================================
--- branches/stable-1.2	2012-12-10 16:00:55 UTC (rev 4951)
+++ branches/stable-1.2	2012-12-10 16:03:56 UTC (rev 4952)

Property changes on: branches/stable-1.2
___________________________________________________________________
Modified: svn:mergeinfo
## -1 +1 ##
-/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913
+/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913-4914
\ No newline at end of property
Modified: branches/stable-1.2/src/http/ngx_http_variables.c
===================================================================
--- branches/stable-1.2/src/http/ngx_http_variables.c	2012-12-10 16:00:55 UTC (rev 4951)
+++ branches/stable-1.2/src/http/ngx_http_variables.c	2012-12-10 16:03:56 UTC (rev 4952)
@@ -79,6 +79,8 @@
     ngx_http_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_http_variable_request_body_file(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_http_variable_request_time(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_http_variable_status(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
 
@@ -108,6 +110,8 @@
     ngx_http_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_http_variable_pid(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_http_variable_msec(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data);
 
 /*
  * TODO:
@@ -237,6 +241,9 @@
       ngx_http_variable_request_body_file,
       0, 0, 0 },
 
+    { ngx_string("request_time"), NULL, ngx_http_variable_request_time,
+      0, NGX_HTTP_VAR_NOCACHEABLE, 0 },
+
     { ngx_string("status"), NULL,
       ngx_http_variable_status, 0,
       NGX_HTTP_VAR_NOCACHEABLE, 0 },
@@ -285,6 +292,9 @@
     { ngx_string("pid"), NULL, ngx_http_variable_pid,
       0, 0, 0 },
 
+    { ngx_string("msec"), NULL, ngx_http_variable_msec,
+      0, NGX_HTTP_VAR_NOCACHEABLE, 0 },
+
 #if (NGX_HAVE_TCP_INFO)
     { ngx_string("tcpinfo_rtt"), NULL, ngx_http_variable_tcpinfo,
       0, NGX_HTTP_VAR_NOCACHEABLE, 0 },
@@ -1825,6 +1835,35 @@
 
 
 static ngx_int_t
+ngx_http_variable_request_time(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data)
+{
+    u_char          *p;
+    ngx_time_t      *tp;
+    ngx_msec_int_t   ms;
+
+    p = ngx_pnalloc(r->pool, NGX_TIME_T_LEN + 4);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
+
+    tp = ngx_timeofday();
+
+    ms = (ngx_msec_int_t)
+             ((tp->sec - r->start_sec) * 1000 + (tp->msec - r->start_msec));
+    ms = ngx_max(ms, 0);
+
+    v->len = ngx_sprintf(p, "%T.%03M", ms / 1000, ms % 1000) - p;
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+    v->data = p;
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
 ngx_http_variable_connection(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data)
 {
@@ -1915,6 +1954,30 @@
 }
 
 
+static ngx_int_t
+ngx_http_variable_msec(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data)
+{
+    u_char      *p;
+    ngx_time_t  *tp;
+
+    p = ngx_pnalloc(r->pool, NGX_TIME_T_LEN + 4);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
+
+    tp = ngx_timeofday();
+
+    v->len = ngx_sprintf(p, "%T.%03M", tp->sec, tp->msec) - p;
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+    v->data = p;
+
+    return NGX_OK;
+}
+
+
 void *
 ngx_http_map_find(ngx_http_request_t *r, ngx_http_map_t *map, ngx_str_t *match)
 {


From jefftk at google.com  Mon Dec 10 16:28:44 2012
From: jefftk at google.com (Jeff Kaufman)
Date: Mon, 10 Dec 2012 11:28:44 -0500
Subject: content handler priority
Message-ID: <CAMJ6YUtP=mY_wRFJ-VX+QuXTK4J8mpQxEKoyo5HB3+gBp7X7dg@mail.gmail.com>

Can a module add a content handler that runs before any other content handler?

For example, if someone has:

   location / {
      proxy_pass ...;
   }

or:

   location ~ \.(js|css|png|jpg|gif|swf|ico|pdf|mov|fla|zip|rar)$ {
        try_files $uri =404;
    }

then the content handler I set with:

    ngx_http_handler_pt* h = static_cast<ngx_http_handler_pt*>(
        ngx_array_push(&cmcf->phases[NGX_HTTP_CONTENT_PHASE].handlers));
    if (h == NULL) {
      return NGX_ERROR;
    }
    *h = my_content_handler;

won't run.  I can work around this by adding location blocks like:

    location ~ "\.pagespeed\.[a-z]{2}\.[^.]{10}\.[^.]+" {
       # let ngx_pagespeed's content handler run
    }
    location ~ "^/ngx_pagespeed_static/" {
       # let ngx_pagespeed's content handler run
    }

but is this something I can make the ngx_pagespeed module do by
default, without configuration?

Is it safe for me to modify
cmcf->phases[NGX_HTTP_CONTENT_PHASE].handlers by inserting my handler
at the beginning instead of appending it?  Will that make it run
first?

Motivation: ngx_pagespeed (a mod_pagespeed port) has two cases where
it needs to be sure it's controlling the handler.  One is where the
request is for anything in the top level ngx_pagespeed_static
directory, like
"http://example.com/ngx_pagespeed_static/js_defer.0.js" and the other
is for any path like
"http://example.com/foo/bar/Puzzle.jpg.pagespeed.ce.91_WewrLtP.jpg".
These are dynamically generated, and any other handler won't know how
to produce them.  My content handler already knows how to return
NGX_DECLINED for unrelated requests but it doesn't get invoked if
something else claims the content handling first.

Jeff


From mdounin at mdounin.ru  Mon Dec 10 16:35:32 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon, 10 Dec 2012 16:35:32 +0000
Subject: [nginx] svn commit:  r4953 - in branches/stable-1.2: . src/http
Message-ID: <20121210163532.EF6C23F9C3E@mail.nginx.com>

Author: mdounin
Date: 2012-12-10 16:35:32 +0000 (Mon, 10 Dec 2012)
New Revision: 4953
URL: http://trac.nginx.org/nginx/changeset/4953/nginx

Log:
Merge of r4915, r4916, r4917: upstream minor fixes.

*) Upstream: honor the "down" flag for a single server.

   If an upstream block was defined with the only server marked as
   "down", e.g. 
        
       upstream u {
           server 127.0.0.1:8080 down;
       }

   an attempt was made to contact the server despite the "down" flag.
   It is believed that immediate 502 response is better in such a
   case, and it's also consistent with what is currently done in case
   of multiple servers all marked as "down".

*) Upstream: better detection of connect() failures with kqueue.

   Pending EOF might be reported on both read and write events, whichever
   comes first, so check both of them.

   Patch by Yichun Zhang (agentzh), slightly modified.


Modified:
   branches/stable-1.2/
   branches/stable-1.2/src/http/ngx_http_upstream.c
   branches/stable-1.2/src/http/ngx_http_upstream_round_robin.c

Index: branches/stable-1.2
===================================================================
--- branches/stable-1.2	2012-12-10 16:03:56 UTC (rev 4952)
+++ branches/stable-1.2	2012-12-10 16:35:32 UTC (rev 4953)

Property changes on: branches/stable-1.2
___________________________________________________________________
Modified: svn:mergeinfo
## -1 +1 ##
-/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913-4914
+/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913-4917
\ No newline at end of property
Modified: branches/stable-1.2/src/http/ngx_http_upstream.c
===================================================================
--- branches/stable-1.2/src/http/ngx_http_upstream.c	2012-12-10 16:03:56 UTC (rev 4952)
+++ branches/stable-1.2/src/http/ngx_http_upstream.c	2012-12-10 16:35:32 UTC (rev 4953)
@@ -1809,9 +1809,16 @@
 #if (NGX_HAVE_KQUEUE)
 
     if (ngx_event_flags & NGX_USE_KQUEUE_EVENT)  {
-        if (c->write->pending_eof) {
+        if (c->write->pending_eof || c->read->pending_eof) {
+            if (c->write->pending_eof) {
+                err = c->write->kq_errno;
+
+            } else {
+                err = c->read->kq_errno;
+            }
+
             c->log->action = "connecting to upstream";
-            (void) ngx_connection_error(c, c->write->kq_errno,
+            (void) ngx_connection_error(c, err,
                                     "kevent() reported that connect() failed");
             return NGX_ERROR;
         }

Modified: branches/stable-1.2/src/http/ngx_http_upstream_round_robin.c
===================================================================
--- branches/stable-1.2/src/http/ngx_http_upstream_round_robin.c	2012-12-10 16:03:56 UTC (rev 4952)
+++ branches/stable-1.2/src/http/ngx_http_upstream_round_robin.c	2012-12-10 16:35:32 UTC (rev 4953)
@@ -430,6 +430,10 @@
     if (rrp->peers->single) {
         peer = &rrp->peers->peer[0];
 
+        if (peer->down) {
+            goto failed;
+        }
+
     } else {
 
         /* there are several peers */


From mdounin at mdounin.ru  Mon Dec 10 17:46:52 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon, 10 Dec 2012 17:46:52 +0000
Subject: [nginx] svn commit: r4954 - in branches/stable-1.2: . src/core
 src/os/unix
Message-ID: <20121210174652.5CDD23F9FD7@mail.nginx.com>

Author: mdounin
Date: 2012-12-10 17:46:51 +0000 (Mon, 10 Dec 2012)
New Revision: 4954
URL: http://trac.nginx.org/nginx/changeset/4954/nginx

Log:
Merge of r4918: cache manager startup with many listen sockets.

Fixed failure to start cache manager and cache loader processes
if there were more than 512 listening sockets in configuration.


Modified:
   branches/stable-1.2/
   branches/stable-1.2/src/core/ngx_connection.c
   branches/stable-1.2/src/os/unix/ngx_process_cycle.c

Index: branches/stable-1.2
===================================================================
--- branches/stable-1.2	2012-12-10 16:35:32 UTC (rev 4953)
+++ branches/stable-1.2	2012-12-10 17:46:51 UTC (rev 4954)

Property changes on: branches/stable-1.2
___________________________________________________________________
Modified: svn:mergeinfo
## -1 +1 ##
-/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913-4917
+/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913-4918
\ No newline at end of property
Modified: branches/stable-1.2/src/core/ngx_connection.c
===================================================================
--- branches/stable-1.2/src/core/ngx_connection.c	2012-12-10 16:35:32 UTC (rev 4953)
+++ branches/stable-1.2/src/core/ngx_connection.c	2012-12-10 17:46:51 UTC (rev 4954)
@@ -749,6 +749,8 @@
 
         ls[i].fd = (ngx_socket_t) -1;
     }
+
+    cycle->listening.nelts = 0;
 }
 
 

Modified: branches/stable-1.2/src/os/unix/ngx_process_cycle.c
===================================================================
--- branches/stable-1.2/src/os/unix/ngx_process_cycle.c	2012-12-10 16:35:32 UTC (rev 4953)
+++ branches/stable-1.2/src/os/unix/ngx_process_cycle.c	2012-12-10 17:46:51 UTC (rev 4954)
@@ -1296,14 +1296,19 @@
     void         *ident[4];
     ngx_event_t   ev;
 
+    /*
+     * Set correct process type since closing listening Unix domain socket
+     * in a master process also removes the Unix domain socket file.
+     */
+    ngx_process = NGX_PROCESS_HELPER;
+
+    ngx_close_listening_sockets(cycle);
+
+    /* Set a moderate number of connections for a helper process. */
     cycle->connection_n = 512;
 
-    ngx_process = NGX_PROCESS_HELPER;
-
     ngx_worker_process_init(cycle, -1);
 
-    ngx_close_listening_sockets(cycle);
-
     ngx_memzero(&ev, sizeof(ngx_event_t));
     ev.handler = ctx->handler;
     ev.data = ident;


From mdounin at mdounin.ru  Mon Dec 10 17:51:10 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon, 10 Dec 2012 17:51:10 +0000
Subject: [nginx] svn commit: r4955 - in branches/stable-1.2: . src/http/modules
Message-ID: <20121210175110.5D4463F9F01@mail.nginx.com>

Author: mdounin
Date: 2012-12-10 17:51:10 +0000 (Mon, 10 Dec 2012)
New Revision: 4955
URL: http://trac.nginx.org/nginx/changeset/4955/nginx

Log:
Merge of r4919: fixed segfault on PUT in dav module.

Dav: fixed segfault on PUT if body was already read (ticket #238).

If request body reading happens with different options it's possible
that there will be no r->request_body->temp_file available (or even
no r->request_body available if body was discarded).  Return internal
server error in this case instead of committing suicide by dereferencing
a null pointer.


Modified:
   branches/stable-1.2/
   branches/stable-1.2/src/http/modules/ngx_http_dav_module.c

Index: branches/stable-1.2
===================================================================
--- branches/stable-1.2	2012-12-10 17:46:51 UTC (rev 4954)
+++ branches/stable-1.2	2012-12-10 17:51:10 UTC (rev 4955)

Property changes on: branches/stable-1.2
___________________________________________________________________
Modified: svn:mergeinfo
## -1 +1 ##
-/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913-4918
+/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913-4919
\ No newline at end of property
Modified: branches/stable-1.2/src/http/modules/ngx_http_dav_module.c
===================================================================
--- branches/stable-1.2/src/http/modules/ngx_http_dav_module.c	2012-12-10 17:46:51 UTC (rev 4954)
+++ branches/stable-1.2/src/http/modules/ngx_http_dav_module.c	2012-12-10 17:51:10 UTC (rev 4955)
@@ -209,6 +209,11 @@
     ngx_ext_rename_file_t     ext;
     ngx_http_dav_loc_conf_t  *dlcf;
 
+    if (r->request_body == NULL || r->request_body->temp_file == NULL) {
+        ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
+        return;
+    }
+
     ngx_http_map_uri_to_path(r, &path, &root, 0);
 
     path.len--;


From mdounin at mdounin.ru  Mon Dec 10 18:04:54 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon, 10 Dec 2012 18:04:54 +0000
Subject: [nginx] svn commit:  r4956 - in branches/stable-1.2: . src/os/unix
Message-ID: <20121210180454.90F0E3F9C6B@mail.nginx.com>

Author: mdounin
Date: 2012-12-10 18:04:54 +0000 (Mon, 10 Dec 2012)
New Revision: 4956
URL: http://trac.nginx.org/nginx/changeset/4956/nginx

Log:
Merge of r4920, r4939: ngx_write_chain_to_file() fixes.

*) Core: added debug logging of writev() in ngx_write_chain_to_file().

*) Core: fixed ngx_write_chain_to_file() with IOV_MAX reached.

   Catched by dav_chunked.t on Solaris.  In released versions this might
   potentially result in corruption of complex protocol responses if they
   were written to disk and there were more distinct buffers than IOV_MAX
   in a single write.


Modified:
   branches/stable-1.2/
   branches/stable-1.2/src/os/unix/ngx_files.c

Index: branches/stable-1.2
===================================================================
--- branches/stable-1.2	2012-12-10 17:51:10 UTC (rev 4955)
+++ branches/stable-1.2	2012-12-10 18:04:54 UTC (rev 4956)

Property changes on: branches/stable-1.2
___________________________________________________________________
Modified: svn:mergeinfo
## -1 +1 ##
-/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913-4919
+/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913-4920,4939
\ No newline at end of property
Modified: branches/stable-1.2/src/os/unix/ngx_files.c
===================================================================
--- branches/stable-1.2/src/os/unix/ngx_files.c	2012-12-10 17:51:10 UTC (rev 4955)
+++ branches/stable-1.2/src/os/unix/ngx_files.c	2012-12-10 18:04:54 UTC (rev 4956)
@@ -241,8 +241,12 @@
             return NGX_ERROR;
         }
 
+        ngx_log_debug2(NGX_LOG_DEBUG_CORE, file->log, 0,
+                       "writev: %d, %z", file->fd, n);
+
         file->sys_offset += n;
         file->offset += n;
+        offset += n;
         total += n;
 
     } while (cl);


From mdounin at mdounin.ru  Mon Dec 10 18:17:32 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon, 10 Dec 2012 18:17:32 +0000
Subject: [nginx] svn commit:  r4957 - in branches/stable-1.2: . src/core
Message-ID: <20121210181732.796DD3F9E2A@mail.nginx.com>

Author: mdounin
Date: 2012-12-10 18:17:32 +0000 (Mon, 10 Dec 2012)
New Revision: 4957
URL: http://trac.nginx.org/nginx/changeset/4957/nginx

Log:
Merge of r4933, r4933: shared memory fixes.

*) Fixed location of debug message in ngx_shmtx_lock().

*) Core: don't reuse shared memory zone that changed ownership (ticket #210).

   nginx doesn't allow the same shared memory zone to be used for different
   purposes, but failed to check this on reconfiguration.  If a shared memory
   zone was used for another purpose in the new configuration, nginx attempted
   to reuse it and crashed.


Modified:
   branches/stable-1.2/
   branches/stable-1.2/src/core/ngx_cycle.c
   branches/stable-1.2/src/core/ngx_shmtx.c

Index: branches/stable-1.2
===================================================================
--- branches/stable-1.2	2012-12-10 18:04:54 UTC (rev 4956)
+++ branches/stable-1.2	2012-12-10 18:17:32 UTC (rev 4957)

Property changes on: branches/stable-1.2
___________________________________________________________________
Modified: svn:mergeinfo
## -1 +1 ##
-/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913-4920,4939
+/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913-4920,4933-4934,4939
\ No newline at end of property
Modified: branches/stable-1.2/src/core/ngx_cycle.c
===================================================================
--- branches/stable-1.2/src/core/ngx_cycle.c	2012-12-10 18:04:54 UTC (rev 4956)
+++ branches/stable-1.2/src/core/ngx_cycle.c	2012-12-10 18:17:32 UTC (rev 4957)
@@ -447,7 +447,9 @@
                 continue;
             }
 
-            if (shm_zone[i].shm.size == oshm_zone[n].shm.size) {
+            if (shm_zone[i].tag == oshm_zone[n].tag
+                && shm_zone[i].shm.size == oshm_zone[n].shm.size)
+            {
                 shm_zone[i].shm.addr = oshm_zone[n].shm.addr;
 
                 if (shm_zone[i].init(&shm_zone[i], oshm_zone[n].data)

Modified: branches/stable-1.2/src/core/ngx_shmtx.c
===================================================================
--- branches/stable-1.2/src/core/ngx_shmtx.c	2012-12-10 18:04:54 UTC (rev 4956)
+++ branches/stable-1.2/src/core/ngx_shmtx.c	2012-12-10 18:17:32 UTC (rev 4957)
@@ -117,11 +117,11 @@
                                   "sem_wait() failed while waiting on shmtx");
                     break;
                 }
-
-                ngx_log_debug0(NGX_LOG_DEBUG_CORE, ngx_cycle->log, 0,
-                               "shmtx awoke");
             }
 
+            ngx_log_debug0(NGX_LOG_DEBUG_CORE, ngx_cycle->log, 0,
+                           "shmtx awoke");
+
             continue;
         }
 


From piotr at cloudflare.com  Mon Dec 10 20:16:00 2012
From: piotr at cloudflare.com (Piotr Sikora)
Date: Mon, 10 Dec 2012 12:16:00 -0800
Subject: content handler priority
In-Reply-To: <CAMJ6YUtP=mY_wRFJ-VX+QuXTK4J8mpQxEKoyo5HB3+gBp7X7dg@mail.gmail.com>
References: <CAMJ6YUtP=mY_wRFJ-VX+QuXTK4J8mpQxEKoyo5HB3+gBp7X7dg@mail.gmail.com>
Message-ID: <CADMhe6fRkQxTqU-_mfYT3X8juq1pk_5hwEyFGDbZ_A-5+DpNXQ@mail.gmail.com>

Hi Jeff,

> Can a module add a content handler that runs before any other content handler?

Yes, with a bit of trickery... You need to add your content handler
(wrapper) that does its job and then calls the original content
handler. To make this work with all the inheritance going around, you
need to make sure to replace original content handler in the location
configurations merge phase. Also, make sure to properly test various
cases with "if" pseudo-locations.

See https://github.com/FRiCKLE/ngx_cache_purge/commit/822a28c for a
working example.

> For example, if someone has:
>
>    location / {
>       proxy_pass ...;
>    }
>
> or:
>
>    location ~ \.(js|css|png|jpg|gif|swf|ico|pdf|mov|fla|zip|rar)$ {
>         try_files $uri =404;
>     }
>
> then the content handler I set with:
>
>     ngx_http_handler_pt* h = static_cast<ngx_http_handler_pt*>(
>         ngx_array_push(&cmcf->phases[NGX_HTTP_CONTENT_PHASE].handlers));
>     if (h == NULL) {
>       return NGX_ERROR;
>     }
>     *h = my_content_handler;
>
> won't run.

That's because content phase handlers are executed only if there is no
"exclusive" content handler set (clcf->handler).

> Is it safe for me to modify
> cmcf->phases[NGX_HTTP_CONTENT_PHASE].handlers by inserting my handler
> at the beginning instead of appending it?  Will that make it run
> first?

It's safe, but nginx won't execute it anyway (for your use case).

Best regards,
Piotr Sikora


From roysterking6 at gmail.com  Tue Dec 11 07:49:16 2012
From: roysterking6 at gmail.com (King Royster)
Date: Tue, 11 Dec 2012 02:49:16 -0500
Subject: No subject
Message-ID: <CAKd-PFAz1yEUDc97eF1AQuBeUZOcWi0kip0fsg=vJbgNfOeRUg@mail.gmail.com>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121211/b20550cf/attachment-0001.html>

From piotr at cloudflare.com  Tue Dec 11 08:19:25 2012
From: piotr at cloudflare.com (Piotr Sikora)
Date: Tue, 11 Dec 2012 00:19:25 -0800
Subject: [PATCH] Fix PCRE detection on OSX.
In-Reply-To: <20121210150728.GC82873@lo0.su>
References: <CADMhe6faJEb07SPzz2bJD13qr+G-dGY1wg99R_QN0Rm6ejjk1Q@mail.gmail.com>
 <20121207222129.GB30441@lo0.su>
 <CADMhe6c91o1J4Exm8pQhbdwUo_L1c24kGjAh5mF=DW6vS9gzRQ@mail.gmail.com>
 <50C275F4.8070005@tvdw.eu>
 <CADMhe6cVh3=+zsqNp72+C2zOUX8jSH-G3TjOBdk_ec8RsYeHGw@mail.gmail.com>
 <20121208052538.GA36873@lo0.su>
 <CADMhe6edXzrzvL56wPDaCVoL19MvvGEop4n6Z-LDG=BkYdX2Qw@mail.gmail.com>
 <20121210023724.GI40452@mdounin.ru> <20121210103142.GA71576@lo0.su>
 <20121210145000.GL40452@mdounin.ru> <20121210150728.GC82873@lo0.su>
Message-ID: <CADMhe6fOyww4skE5wjy0jTBF5VTudhoTKKTDxRFsLvyVOsJ5fg@mail.gmail.com>

Hey guys,
thanks for going into details why this happens, but to cut this
discussion short, I'll just focus on the patches.

> Index: auto/lib/pcre/conf
> ===================================================================
> --- auto/lib/pcre/conf  (revision 4948)
> +++ auto/lib/pcre/conf  (working copy)
> @@ -101,7 +101,17 @@
>          ngx_feature_test="pcre *re;
>                            re = pcre_compile(NULL, 0, NULL, 0, NULL);
>                            if (re == NULL) return 1"
> -        . auto/feature
> +
> +        if [ "$NGX_SYSTEM" = "Darwin" ]; then
> +            # Apple provides libpcre in /usr/lib but no /usr/include/pcre.h.
> +            # This may interfere with libpcre installed into /usr/local if
> +            # system compiler is used.  To work around this, we delay the
> +            # default test until after we tested /usr/local.
> +            ngx_found=no
> +
> +        else
> +            . auto/feature
> +        fi
>
>          if [ $ngx_found = no ]; then
>
> @@ -119,6 +129,14 @@
>              . auto/feature
>          fi
>
> +        if [ $ngx_found = no -a "$NGX_SYSTEM" = "Darwin" ]; then
> +            ngx_feature="PCRE library"
> +            ngx_feature_path=
> +            ngx_feature_libs="-lpcre"
> +
> +            . auto/feature
> +        fi
> +
>          if [ $ngx_found = no ]; then
>
>              # RedHat RPM, Solaris package

I like this, it makes much more sense to defer the check instead of
completely skipping it.

> --- a/auto/lib/pcre/conf        Mon Nov 26 18:01:49 2012 +0000
> +++ b/auto/lib/pcre/conf        Mon Dec 10 18:21:53 2012 +0400
> @@ -172,6 +172,7 @@ else
>              ngx_feature="PCRE JIT support"
>              ngx_feature_name="NGX_HAVE_PCRE_JIT"
>              ngx_feature_test="int jit = 0;
> +                              pcre_free_study(NULL);
>                                pcre_config(PCRE_CONFIG_JIT, &jit);
>                                if (jit != 1) return 1;"
>              . auto/feature
>
> probably will be good enough.  It will prevent build failures
> by checking if the code we are going to compile will compile.
>
> (Yes, I understand that it won't resolve the root cause, i.e. the
> library vs. headers mismatch.  And yes, I understand that it will
> make PCRE JIT unavailable in some cases.  I don't think we care
> though.)

While I agree that this is a better test for PCRE JIT presence and
that it should be included regardless of the OSX issue, I don't think
that this is solution to the problem at hand, because all it does is
hide the issue from the end user... Putting JIT capabilities aside, I
also dislike the fact that we might be throwing away almost 3 years of
PCRE fixes just because we cannot be bothered to link against newer
library that's available outside of the default search path... I
definitely prefer the solution with postponed check for the default
search path on OSX.

Alternatively, we could first try looking for the PCRE library with
JIT support and fallback to looking for the PCRE library without one:

diff --git a/auto/lib/pcre/conf b/auto/lib/pcre/conf
index 6a8c326..4f75018 100644
--- a/auto/lib/pcre/conf
+++ b/auto/lib/pcre/conf
@@ -92,21 +92,101 @@ else

         PCRE=NO

-        ngx_feature="PCRE library"
-        ngx_feature_name="NGX_PCRE"
+        ngx_feature="PCRE library with JIT support"
+        ngx_feature_name="NGX_HAVE_PCRE_JIT"
         ngx_feature_run=no
         ngx_feature_incs="#include <pcre.h>"
         ngx_feature_path=
         ngx_feature_libs="-lpcre"
         ngx_feature_test="pcre *re;
+                          int jit = 0;
                           re = pcre_compile(NULL, 0, NULL, 0, NULL);
-                          if (re == NULL) return 1"
+                          if (re == NULL) return 1;
+                          pcre_free_study(NULL);
+                          pcre_config(PCRE_CONFIG_JIT, &jit);
+                          if (jit != 1) return 1;"
         . auto/feature

         if [ $ngx_found = no ]; then

             # FreeBSD port

+            ngx_feature="PCRE library with JIT support in /usr/local/"
+            ngx_feature_path="/usr/local/include"
+
+            if [ $NGX_RPATH = YES ]; then
+                ngx_feature_libs="-R/usr/local/lib -L/usr/local/lib -lpcre"
+            else
+                ngx_feature_libs="-L/usr/local/lib -lpcre"
+            fi
+
+            . auto/feature
+        fi
+
+        if [ $ngx_found = no ]; then
+
+            # RedHat RPM, Solaris package
+
+            ngx_feature="PCRE library with JIT support in /usr/include/pcre/"
+            ngx_feature_path="/usr/include/pcre"
+            ngx_feature_libs="-lpcre"
+
+            . auto/feature
+        fi
+
+        if [ $ngx_found = no ]; then
+
+            # NetBSD port
+
+            ngx_feature="PCRE library with JIT support in /usr/pkg/"
+            ngx_feature_path="/usr/pkg/include"
+
+            if [ $NGX_RPATH = YES ]; then
+                ngx_feature_libs="-R/usr/pkg/lib -L/usr/pkg/lib -lpcre"
+            else
+                ngx_feature_libs="-L/usr/pkg/lib -lpcre"
+            fi
+
+            . auto/feature
+        fi
+
+        if [ $ngx_found = no ]; then
+
+            # MacPorts
+
+            ngx_feature="PCRE library with JIT support in /opt/local/"
+            ngx_feature_path="/opt/local/include"
+
+            if [ $NGX_RPATH = YES ]; then
+                ngx_feature_libs="-R/opt/local/lib -L/opt/local/lib -lpcre"
+            else
+                ngx_feature_libs="-L/opt/local/lib -lpcre"
+            fi
+
+            . auto/feature
+        fi
+
+        if [ $ngx_found = yes ]; then
+            PCRE_JIT=YES
+        fi
+
+        if [ $ngx_found = no ]; then
+
+            ngx_feature="PCRE library"
+            ngx_feature_name="NGX_PCRE"
+            ngx_feature_path=
+            ngx_feature_libs="-lpcre"
+            ngx_feature_test="pcre *re;
+                              re = pcre_compile(NULL, 0, NULL, 0, NULL);
+                              if (re == NULL) return 1"
+
+            . auto/feature
+        fi
+
+        if [ $ngx_found = no ]; then
+
+            # FreeBSD port
+
             ngx_feature="PCRE library in /usr/local/"
             ngx_feature_path="/usr/local/include"

@@ -167,19 +247,6 @@ else
             CORE_LIBS="$CORE_LIBS $ngx_feature_libs"
             PCRE=YES
         fi
-
-        if [ $PCRE = YES ]; then
-            ngx_feature="PCRE JIT support"
-            ngx_feature_name="NGX_HAVE_PCRE_JIT"
-            ngx_feature_test="int jit = 0;
-                              pcre_config(PCRE_CONFIG_JIT, &jit);
-                              if (jit != 1) return 1;"
-            . auto/feature
-
-            if [ $ngx_found = yes ]; then
-                PCRE_JIT=YES
-            fi
-        fi
     fi

     if [ $PCRE != YES ]; then

but honestly, it seems like a bit of an overkill.

Best regards,
Piotr Sikora


From piotr at cloudflare.com  Tue Dec 11 09:12:30 2012
From: piotr at cloudflare.com (Piotr Sikora)
Date: Tue, 11 Dec 2012 01:12:30 -0800
Subject: [PATCH] Core: fix content handler and location configuration
 mismatch.
In-Reply-To: <20121210010348.GH40452@mdounin.ru>
References: <CADMhe6cDf57_0wDvTt3i9JwuOGC8UTSwpUyToo8UgsE+HgT=zA@mail.gmail.com>
 <20121207162745.GB40452@mdounin.ru>
 <CADMhe6fM4gjYUNjHozy56oJp3KEi2WkfT1h_PQ8P3=kR1DdjqQ@mail.gmail.com>
 <20121210010348.GH40452@mdounin.ru>
Message-ID: <CADMhe6favQL6RdUKgCi_mEs3Ju0934QNmE0guqC8iOR1QVWgmQ@mail.gmail.com>

Hey Maxim,

> Aproach I would prefer to use to resolve this is already outlined
> at the IfIsEvil page: I would like to keep declarative config and
> imprative rewrite module directives separate, that is, disable all
> non-rewrite-module directives inside if()'s.  That's the way we
> are slowly moving along, in particular, via introducing variables
> support in various places.

I agree with you 100% on that :)

Having said that, removing "if" support from that many directives will
definitely break a lot of setups, so I didn't even bother with working
on a patch that would do that...

> If you can't wait and want to fix this particular SIGSIGV, here
> are some possible aproaches I will be happy enough with:

I can wait, this isn't an issue for any of my existing code or
configurations... I've just noticed the mismatch while working on
ngx_cache_purge-2.0 and I found it terribly wrong, so I wanted to get
it fixed ASAP.

> 1) Just log error and return 500 if configuration incosistency
> detected instead of dereferencing a NULL pointer.

Attached, hopefully this meets your requirements ;)
Feel free to change the error message.

Best regards,
Piotr Sikora


Core: detect content handler and location configuration mismatch.

When using exclusive content handlers (proxy_pass, etc.) and "if"
pseudo-locations, content handler is always set to the one from the
last "if" pseudo-location that evaluates to true or to the one from
the location block if "if" pseudo-locations don't provide any.
At the same time, location configuration is updated for each "if"
pseudo-location that evaluates to true, which results in content
handler and location configuration mismatch.

For example, following configuration would result in SIGSEGV, because
"proxy_pass" content handler (set in the "if ($pass)" pseudo-location)
would be called with location configuration from the "if ($slow)"
pseudo-location which doesn't contain any upstream configuration.

    location / {
        set $pass  1;
        set $slow  1;

        if ($pass) {
            proxy_pass  http://127.0.0.1:8000;
        }

        if ($slow) {
            limit_rate  10k;
        }
    }

This patch detects this mismatch by keeping track of the location
configuration from which the content handler originated and returns
"500 Internal Server Error" response instead of dereferencing NULL
pointer and crashing worker process.

Signed-off-by: Piotr Sikora <piotr at cloudflare.com>
---
 src/http/ngx_http_core_module.c | 9 +++++++++
 src/http/ngx_http_request.h     | 1 +
 2 files changed, 10 insertions(+)

diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
index a7db74b..b48175b 100644
--- a/src/http/ngx_http_core_module.c
+++ b/src/http/ngx_http_core_module.c
@@ -1400,6 +1400,14 @@ ngx_http_core_content_phase(ngx_http_request_t *r,
     ngx_str_t  path;

     if (r->content_handler) {
+        if (r->loc_conf != r->content_handler_loc_conf) {
+            ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                          "content handler and configuration mismatch");
+
+            ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
+            return NGX_OK;
+        }
+
         r->write_event_handler = ngx_http_request_empty_handler;
         ngx_http_finalize_request(r, r->content_handler(r));
         return NGX_OK;
@@ -1526,6 +1534,7 @@ ngx_http_update_location_config(ngx_http_request_t *r)

     if (clcf->handler) {
         r->content_handler = clcf->handler;
+        r->content_handler_loc_conf = r->loc_conf;
     }
 }

diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
index f234840..97c9607 100644
--- a/src/http/ngx_http_request.h
+++ b/src/http/ngx_http_request.h
@@ -358,6 +358,7 @@ struct ngx_http_request_s {
     void                            **main_conf;
     void                            **srv_conf;
     void                            **loc_conf;
+    void                            **content_handler_loc_conf;

     ngx_http_event_handler_pt         read_event_handler;
     ngx_http_event_handler_pt         write_event_handler;


From mdounin at mdounin.ru  Tue Dec 11 13:18:50 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Tue, 11 Dec 2012 13:18:50 +0000
Subject: [nginx] svn commit:  r4958 - in branches/stable-1.2: . src/http
Message-ID: <20121211131850.F37B13F9C4E@mail.nginx.com>

Author: mdounin
Date: 2012-12-11 13:18:50 +0000 (Tue, 11 Dec 2012)
New Revision: 4958
URL: http://trac.nginx.org/nginx/changeset/4958/nginx

Log:
Merge of r4921, r4922, r4923, r4924, r4925: request body fixes.

*) Request body: fixed "501 Not Implemented" error handling.

   It is not about "Method" but a generic message, and is expected to be used
   e.g. if specified Transfer-Encoding is not supported.  Fixed message to
   match RFC 2616.

   Additionally, disable keepalive on such errors as we won't be able to read
   request body correctly if we don't understand Transfer-Encoding used.

*) Request body: $request_body variable generalization.

   The $request_body variable was assuming there can't be more than two
   buffers.  While this is currently true due to request body reading
   implementation details, this is not a good thing to depend on and may
   change in the future.

*) Request body: code duplication reduced, no functional changes.

   The r->request_body_in_file_only with empty body case is now handled in
   ngx_http_write_request_body().

*) Request body: fixed socket leak on errors.

   The r->main->count reference counter was always incremented in
   ngx_http_read_client_request_body(), while it is only needs to be
   incremented on positive returns.

*) Request body: properly handle events while discarding body.

   An attempt to call ngx_handle_read_event() before actually reading
   data from a socket might result in read event being disabled, which is
   wrong.  Catched by body.t test on Solaris.


Modified:
   branches/stable-1.2/
   branches/stable-1.2/src/http/ngx_http_header_filter_module.c
   branches/stable-1.2/src/http/ngx_http_request_body.c
   branches/stable-1.2/src/http/ngx_http_special_response.c
   branches/stable-1.2/src/http/ngx_http_variables.c

Index: branches/stable-1.2
===================================================================
--- branches/stable-1.2	2012-12-10 18:17:32 UTC (rev 4957)
+++ branches/stable-1.2	2012-12-11 13:18:50 UTC (rev 4958)

Property changes on: branches/stable-1.2
___________________________________________________________________
Modified: svn:mergeinfo
## -1 +1 ##
-/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913-4920,4933-4934,4939
+/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913-4925,4933-4934,4939
\ No newline at end of property
Modified: branches/stable-1.2/src/http/ngx_http_header_filter_module.c
===================================================================
--- branches/stable-1.2/src/http/ngx_http_header_filter_module.c	2012-12-10 18:17:32 UTC (rev 4957)
+++ branches/stable-1.2/src/http/ngx_http_header_filter_module.c	2012-12-11 13:18:50 UTC (rev 4958)
@@ -112,7 +112,7 @@
 #define NGX_HTTP_OFF_5XX   (NGX_HTTP_LAST_4XX - 400 + NGX_HTTP_OFF_4XX)
 
     ngx_string("500 Internal Server Error"),
-    ngx_string("501 Method Not Implemented"),
+    ngx_string("501 Not Implemented"),
     ngx_string("502 Bad Gateway"),
     ngx_string("503 Service Temporarily Unavailable"),
     ngx_string("504 Gateway Time-out"),

Modified: branches/stable-1.2/src/http/ngx_http_request_body.c
===================================================================
--- branches/stable-1.2/src/http/ngx_http_request_body.c	2012-12-10 18:17:32 UTC (rev 4957)
+++ branches/stable-1.2/src/http/ngx_http_request_body.c	2012-12-11 13:18:50 UTC (rev 4958)
@@ -31,9 +31,9 @@
 {
     size_t                     preread;
     ssize_t                    size;
+    ngx_int_t                  rc;
     ngx_buf_t                 *b;
     ngx_chain_t               *cl, **next;
-    ngx_temp_file_t           *tf;
     ngx_http_request_body_t   *rb;
     ngx_http_core_loc_conf_t  *clcf;
 
@@ -45,12 +45,14 @@
     }
 
     if (ngx_http_test_expect(r) != NGX_OK) {
-        return NGX_HTTP_INTERNAL_SERVER_ERROR;
+        rc = NGX_HTTP_INTERNAL_SERVER_ERROR;
+        goto done;
     }
 
     rb = ngx_pcalloc(r->pool, sizeof(ngx_http_request_body_t));
     if (rb == NULL) {
-        return NGX_HTTP_INTERNAL_SERVER_ERROR;
+        rc = NGX_HTTP_INTERNAL_SERVER_ERROR;
+        goto done;
     }
 
     r->request_body = rb;
@@ -65,32 +67,10 @@
     if (r->headers_in.content_length_n == 0) {
 
         if (r->request_body_in_file_only) {
-            tf = ngx_pcalloc(r->pool, sizeof(ngx_temp_file_t));
-            if (tf == NULL) {
-                return NGX_HTTP_INTERNAL_SERVER_ERROR;
+            if (ngx_http_write_request_body(r, NULL) != NGX_OK) {
+                rc = NGX_HTTP_INTERNAL_SERVER_ERROR;
+                goto done;
             }
-
-            tf->file.fd = NGX_INVALID_FILE;
-            tf->file.log = r->connection->log;
-            tf->path = clcf->client_body_temp_path;
-            tf->pool = r->pool;
-            tf->warn = "a client request body is buffered to a temporary file";
-            tf->log_level = r->request_body_file_log_level;
-            tf->persistent = r->request_body_in_persistent_file;
-            tf->clean = r->request_body_in_clean_file;
-
-            if (r->request_body_file_group_access) {
-                tf->access = 0660;
-            }
-
-            rb->temp_file = tf;
-
-            if (ngx_create_temp_file(&tf->file, tf->path, tf->pool,
-                                     tf->persistent, tf->clean, tf->access)
-                != NGX_OK)
-            {
-                return NGX_HTTP_INTERNAL_SERVER_ERROR;
-            }
         }
 
         post_handler(r);
@@ -119,7 +99,8 @@
 
         b = ngx_calloc_buf(r->pool);
         if (b == NULL) {
-            return NGX_HTTP_INTERNAL_SERVER_ERROR;
+            rc = NGX_HTTP_INTERNAL_SERVER_ERROR;
+            goto done;
         }
 
         b->temporary = 1;
@@ -130,7 +111,8 @@
 
         rb->bufs = ngx_alloc_chain_link(r->pool);
         if (rb->bufs == NULL) {
-            return NGX_HTTP_INTERNAL_SERVER_ERROR;
+            rc = NGX_HTTP_INTERNAL_SERVER_ERROR;
+            goto done;
         }
 
         rb->bufs->buf = b;
@@ -148,7 +130,8 @@
 
             if (r->request_body_in_file_only) {
                 if (ngx_http_write_request_body(r, rb->bufs) != NGX_OK) {
-                    return NGX_HTTP_INTERNAL_SERVER_ERROR;
+                    rc = NGX_HTTP_INTERNAL_SERVER_ERROR;
+                    goto done;
                 }
             }
 
@@ -175,7 +158,8 @@
 
             r->read_event_handler = ngx_http_read_client_request_body_handler;
 
-            return ngx_http_do_read_client_request_body(r);
+            rc = ngx_http_do_read_client_request_body(r);
+            goto done;
         }
 
         next = &rb->bufs->next;
@@ -205,12 +189,14 @@
 
     rb->buf = ngx_create_temp_buf(r->pool, size);
     if (rb->buf == NULL) {
-        return NGX_HTTP_INTERNAL_SERVER_ERROR;
+        rc = NGX_HTTP_INTERNAL_SERVER_ERROR;
+        goto done;
     }
 
     cl = ngx_alloc_chain_link(r->pool);
     if (cl == NULL) {
-        return NGX_HTTP_INTERNAL_SERVER_ERROR;
+        rc = NGX_HTTP_INTERNAL_SERVER_ERROR;
+        goto done;
     }
 
     cl->buf = rb->buf;
@@ -235,7 +221,15 @@
 
     r->read_event_handler = ngx_http_read_client_request_body_handler;
 
-    return ngx_http_do_read_client_request_body(r);
+    rc = ngx_http_do_read_client_request_body(r);
+
+done:
+
+    if (rc >= NGX_HTTP_SPECIAL_RESPONSE) {
+        r->main->count--;
+    }
+
+    return rc;
 }
 
 
@@ -419,6 +413,19 @@
         }
 
         rb->temp_file = tf;
+
+        if (body == NULL) {
+            /* empty body with r->request_body_in_file_only */
+
+            if (ngx_create_temp_file(&tf->file, tf->path, tf->pool,
+                                     tf->persistent, tf->clean, tf->access)
+                != NGX_OK)
+            {
+                return NGX_ERROR;
+            }
+
+            return NGX_OK;
+        }
     }
 
     n = ngx_write_chain_to_temp_file(rb->temp_file, body);
@@ -475,20 +482,22 @@
         }
     }
 
+    if (ngx_http_read_discarded_request_body(r) == NGX_OK) {
+        r->lingering_close = 0;
+        return NGX_OK;
+    }
+
+    /* == NGX_AGAIN */
+
     r->read_event_handler = ngx_http_discarded_request_body_handler;
 
     if (ngx_handle_read_event(rev, 0) != NGX_OK) {
         return NGX_HTTP_INTERNAL_SERVER_ERROR;
     }
 
-    if (ngx_http_read_discarded_request_body(r) == NGX_OK) {
-        r->lingering_close = 0;
+    r->count++;
+    r->discard_body = 1;
 
-    } else {
-        r->count++;
-        r->discard_body = 1;
-    }
-
     return NGX_OK;
 }
 

Modified: branches/stable-1.2/src/http/ngx_http_special_response.c
===================================================================
--- branches/stable-1.2/src/http/ngx_http_special_response.c	2012-12-10 18:17:32 UTC (rev 4957)
+++ branches/stable-1.2/src/http/ngx_http_special_response.c	2012-12-11 13:18:50 UTC (rev 4958)
@@ -260,9 +260,9 @@
 
 static char ngx_http_error_501_page[] =
 "<html>" CRLF
-"<head><title>501 Method Not Implemented</title></head>" CRLF
+"<head><title>501 Not Implemented</title></head>" CRLF
 "<body bgcolor=\"white\">" CRLF
-"<center><h1>501 Method Not Implemented</h1></center>" CRLF
+"<center><h1>501 Not Implemented</h1></center>" CRLF
 ;
 
 
@@ -384,6 +384,7 @@
             case NGX_HTTPS_CERT_ERROR:
             case NGX_HTTPS_NO_CERT:
             case NGX_HTTP_INTERNAL_SERVER_ERROR:
+            case NGX_HTTP_NOT_IMPLEMENTED:
                 r->keepalive = 0;
         }
     }

Modified: branches/stable-1.2/src/http/ngx_http_variables.c
===================================================================
--- branches/stable-1.2/src/http/ngx_http_variables.c	2012-12-10 18:17:32 UTC (rev 4957)
+++ branches/stable-1.2/src/http/ngx_http_variables.c	2012-12-11 13:18:50 UTC (rev 4958)
@@ -1767,7 +1767,7 @@
 {
     u_char       *p;
     size_t        len;
-    ngx_buf_t    *buf, *next;
+    ngx_buf_t    *buf;
     ngx_chain_t  *cl;
 
     if (r->request_body == NULL
@@ -1792,18 +1792,26 @@
         return NGX_OK;
     }
 
-    next = cl->next->buf;
-    len = (buf->last - buf->pos) + (next->last - next->pos);
+    len = buf->last - buf->pos;
+    cl = cl->next;
 
+    for ( /* void */ ; cl; cl = cl->next) {
+        buf = cl->buf;
+        len += buf->last - buf->pos;
+    }
+
     p = ngx_pnalloc(r->pool, len);
     if (p == NULL) {
         return NGX_ERROR;
     }
 
     v->data = p;
+    cl = r->request_body->bufs;
 
-    p = ngx_cpymem(p, buf->pos, buf->last - buf->pos);
-    ngx_memcpy(p, next->pos, next->last - next->pos);
+    for ( /* void */ ; cl; cl = cl->next) {
+        buf = cl->buf;
+        p = ngx_cpymem(p, buf->pos, buf->last - buf->pos);
+    }
 
     v->len = len;
     v->valid = 1;


From mdounin at mdounin.ru  Tue Dec 11 14:24:23 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Tue, 11 Dec 2012 14:24:23 +0000
Subject: [nginx] svn commit:  r4959 - branches/stable-1.2/docs/xml/nginx
Message-ID: <20121211142423.86CA53F9C4B@mail.nginx.com>

Author: mdounin
Date: 2012-12-11 14:24:23 +0000 (Tue, 11 Dec 2012)
New Revision: 4959
URL: http://trac.nginx.org/nginx/changeset/4959/nginx

Log:
nginx-1.2.6-RELEASE


Modified:
   branches/stable-1.2/docs/xml/nginx/changes.xml

Modified: branches/stable-1.2/docs/xml/nginx/changes.xml
===================================================================
--- branches/stable-1.2/docs/xml/nginx/changes.xml	2012-12-11 13:18:50 UTC (rev 4958)
+++ branches/stable-1.2/docs/xml/nginx/changes.xml	2012-12-11 14:24:23 UTC (rev 4959)
@@ -5,6 +5,42 @@
 <change_log title="nginx">
 
 
+<changes ver="1.2.6" date="11.12.2012">
+
+<change type="feature">
+<para lang="ru">
+?????????? $request_time ? $msec
+?????? ????? ???????????? ?? ?????? ? ????????? log_format.
+</para>
+<para lang="en">
+the $request_time and $msec variables
+can now be used not only in the "log_format" directive.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+cache manager ? cache loader ????? ?? ???????????,
+???? ?????????????? ????? 512 listen-???????.
+</para>
+<para lang="en">
+cache manager and cache loader processes might not be able to start
+if more than 512 listen sockets were used.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ?????? ngx_http_dav_module.
+</para>
+<para lang="en">
+in the ngx_http_dav_module.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.2.5" date="13.11.2012">
 
 <change type="feature">


From mdounin at mdounin.ru  Tue Dec 11 14:26:09 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Tue, 11 Dec 2012 14:26:09 +0000
Subject: [nginx] svn commit:  r4960 - in tags: . release-1.2.6
Message-ID: <20121211142609.648543F9C11@mail.nginx.com>

Author: mdounin
Date: 2012-12-11 14:26:07 +0000 (Tue, 11 Dec 2012)
New Revision: 4960
URL: http://trac.nginx.org/nginx/changeset/4960/nginx

Log:
release-1.2.6 tag


Added:
   tags/release-1.2.6/

Index: tags/release-1.2.6
===================================================================
--- branches/stable-1.2	2012-12-11 14:24:23 UTC (rev 4959)
+++ tags/release-1.2.6	2012-12-11 14:26:07 UTC (rev 4960)

Property changes on: tags/release-1.2.6
___________________________________________________________________
Added: svn:ignore
## -0,0 +1,14 ##
+access.log
+client_body_temp
+fastcgi_temp
+proxy_temp
+scgi_temp
+uwsgi_temp
+GNUmakefile
+Makefile
+makefile
+nginx
+nginx.conf
+nginx-*.tar.gz
+objs*
+tmp
Added: svn:mergeinfo
## -0,0 +1 ##
+/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844,4865-4872,4885-4887,4890-4896,4913-4925,4933-4934,4939
\ No newline at end of property


From brian at akins.org  Wed Dec 12 01:35:17 2012
From: brian at akins.org (Brian Akins)
Date: Tue, 11 Dec 2012 20:35:17 -0500
Subject: nginx + lua and GC latency
In-Reply-To: <CAJbhXfVPLm+AVTOxL1CtOrkDO__gpzt6BnL+rWsafrERJGEWEw@mail.gmail.com>
References: <CAJbhXfVPLm+AVTOxL1CtOrkDO__gpzt6BnL+rWsafrERJGEWEw@mail.gmail.com>
Message-ID: <8D3616C4-63EC-4531-B7CA-A1948B6C47E0@akins.org>


On Dec 8, 2012, at 4:25 PM, Dave Bailey <dave at daveb.net> wrote:

> .  Has anyone done any profiling to measure this on nginx under "real" load (e.g. 1k-2k req/s and 100-200+ Mbits/s bandwidth per worker, or somewhere in that range)? 

I do tens of thousands of req/sec and can fill a 10Gb interface with hundreds of thousands of concurrent users while being a heavy user of Lua on a single server. 

In short, don't worry about it.

--bakins

From wangzhiming1757 at gmail.com  Wed Dec 12 06:55:52 2012
From: wangzhiming1757 at gmail.com (=?GB2312?B?zfXWvsP3?=)
Date: Wed, 12 Dec 2012 14:55:52 +0800
Subject: fresh to nginx
Message-ID: <CANA-K5Ughn0OD9UuvTs0hwubxS4ed4Pia3yqi6kvHQhiBMm0rw@mail.gmail.com>

hi all,

I am a fresh man to nginx, who can tell me how to read the nginx source
code in general, i just trace the source from main() entry, if this is the
right  manner.

and any related documents are welcomed.

brgs/ thanks
Zhiming
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121212/6d34e891/attachment.html>

From matthieu.tourne at gmail.com  Wed Dec 12 07:32:46 2012
From: matthieu.tourne at gmail.com (Matthieu Tourne)
Date: Tue, 11 Dec 2012 23:32:46 -0800
Subject: fresh to nginx
In-Reply-To: <CANA-K5Ughn0OD9UuvTs0hwubxS4ed4Pia3yqi6kvHQhiBMm0rw@mail.gmail.com>
References: <CANA-K5Ughn0OD9UuvTs0hwubxS4ed4Pia3yqi6kvHQhiBMm0rw@mail.gmail.com>
Message-ID: <CAAGYmFeYo-vUu5pKKyVV50wDJ1skgLihQMKQTy4yMWhNsZr_+A@mail.gmail.com>

Hi Zhiming!

On Tue, Dec 11, 2012 at 10:55 PM, ??? <wangzhiming1757 at gmail.com> wrote:

> hi all,
>
> I am a fresh man to nginx, who can tell me how to read the nginx source
> code in general, i just trace the source from main() entry, if this is the
> right  manner.
>
> and any related documents are welcomed.
>

There are some great guides on the web to get started :

Evan Miller Guide To Nginx Module Development [1] and advanced topics [2]
A bit old, but definitely not dated if you want to learn how it works
inside.

Nginx Guts [3] also has quite a few articles on the core of Nginx.

If you're interested in adding new features, you can find many Nginx [4] or
3rd-party modules [5],
they all add new extensions and capabilities to the core of Nginx.

As for the main() I think it parses the command line options, reads the
configuration and spawn the worker processes.
Then each worker process will start accepting and processing HTTP requests.

Hope that helps,

Matthieu.


[1] http://www.evanmiller.org/nginx-modules-guide.html
[2] http://www.evanmiller.org/nginx-modules-guide-advanced.html
[3] http://www.nginxguts.com/2011/01/phases/
[4] http://wiki.nginx.org/Modules
[5] http://wiki.nginx.org/3rdPartyModules
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121211/77d06b5a/attachment.html>

From wangzhiming1757 at gmail.com  Wed Dec 12 08:09:25 2012
From: wangzhiming1757 at gmail.com (=?GB2312?B?zfXWvsP3?=)
Date: Wed, 12 Dec 2012 16:09:25 +0800
Subject: fresh to nginx
In-Reply-To: <CAAGYmFeYo-vUu5pKKyVV50wDJ1skgLihQMKQTy4yMWhNsZr_+A@mail.gmail.com>
References: <CANA-K5Ughn0OD9UuvTs0hwubxS4ed4Pia3yqi6kvHQhiBMm0rw@mail.gmail.com>
 <CAAGYmFeYo-vUu5pKKyVV50wDJ1skgLihQMKQTy4yMWhNsZr_+A@mail.gmail.com>
Message-ID: <CANA-K5XgrbzhkJfBY2_WYRdjBpKEZb8aFkKm_k9Hrcf9ES9=5A@mail.gmail.com>

hi Matthieu,

Thanks for your suggestion and guideline, and now i am analysising the
Nginx memory pool, it is a little more complicated, maybe i should take
more time on it, hope enjoy it.

brgs\ thanks
Zhiming


2012/12/12 Matthieu Tourne <matthieu.tourne at gmail.com>

>
> Hi Zhiming!
>
> On Tue, Dec 11, 2012 at 10:55 PM, ??? <wangzhiming1757 at gmail.com> wrote:
>
>> hi all,
>>
>> I am a fresh man to nginx, who can tell me how to read the nginx source
>> code in general, i just trace the source from main() entry, if this is the
>> right  manner.
>>
>> and any related documents are welcomed.
>>
>
> There are some great guides on the web to get started :
>
> Evan Miller Guide To Nginx Module Development [1] and advanced topics [2]
> A bit old, but definitely not dated if you want to learn how it works
> inside.
>
> Nginx Guts [3] also has quite a few articles on the core of Nginx.
>
> If you're interested in adding new features, you can find many Nginx [4]
> or 3rd-party modules [5],
> they all add new extensions and capabilities to the core of Nginx.
>
> As for the main() I think it parses the command line options, reads the
> configuration and spawn the worker processes.
>  Then each worker process will start accepting and processing HTTP
> requests.
>
> Hope that helps,
>
> Matthieu.
>
>
> [1] http://www.evanmiller.org/nginx-modules-guide.html
> [2] http://www.evanmiller.org/nginx-modules-guide-advanced.html
> [3] http://www.nginxguts.com/2011/01/phases/
> [4] http://wiki.nginx.org/Modules
> [5] http://wiki.nginx.org/3rdPartyModules
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121212/923dd07d/attachment-0001.html>

From mdounin at mdounin.ru  Wed Dec 12 11:04:48 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 12 Dec 2012 15:04:48 +0400
Subject: [PATCH] Fix PCRE detection on OSX.
In-Reply-To: <CADMhe6fOyww4skE5wjy0jTBF5VTudhoTKKTDxRFsLvyVOsJ5fg@mail.gmail.com>
References: <CADMhe6c91o1J4Exm8pQhbdwUo_L1c24kGjAh5mF=DW6vS9gzRQ@mail.gmail.com>
 <50C275F4.8070005@tvdw.eu>
 <CADMhe6cVh3=+zsqNp72+C2zOUX8jSH-G3TjOBdk_ec8RsYeHGw@mail.gmail.com>
 <20121208052538.GA36873@lo0.su>
 <CADMhe6edXzrzvL56wPDaCVoL19MvvGEop4n6Z-LDG=BkYdX2Qw@mail.gmail.com>
 <20121210023724.GI40452@mdounin.ru> <20121210103142.GA71576@lo0.su>
 <20121210145000.GL40452@mdounin.ru> <20121210150728.GC82873@lo0.su>
 <CADMhe6fOyww4skE5wjy0jTBF5VTudhoTKKTDxRFsLvyVOsJ5fg@mail.gmail.com>
Message-ID: <20121212110448.GL40452@mdounin.ru>

Hello!

On Tue, Dec 11, 2012 at 12:19:25AM -0800, Piotr Sikora wrote:

[...]

> > +        if [ $ngx_found = no -a "$NGX_SYSTEM" = "Darwin" ]; then
> > +            ngx_feature="PCRE library"
> > +            ngx_feature_path=
> > +            ngx_feature_libs="-lpcre"
> > +
> > +            . auto/feature
> > +        fi
> > +
> >          if [ $ngx_found = no ]; then
> >
> >              # RedHat RPM, Solaris package
> 
> I like this, it makes much more sense to defer the check instead of
> completely skipping it.

Consider I don't want /usr/local/include to be used at all, e.g. 
because I have some incompatible headers there.  Right now I can 
do this with correct compiler/linker arguments, e.g. like this:

./configure \
    --with-cc-opt="-nostdlibinc -I/usr/include -I/path/to/pcre/include" \
    --with-ld-opt="-L/path/to/pcre/lib"

With the patch it will no longer be possible.  (The problem is 
actually similar to one with the check completely skipped, but 
manifests itself in a bit more exotic cases.)

(Moreover, the patch doesn't resolve MacPorts toolchain case as it 
checks default include paths before the /opt/local/.)

> > --- a/auto/lib/pcre/conf        Mon Nov 26 18:01:49 2012 +0000
> > +++ b/auto/lib/pcre/conf        Mon Dec 10 18:21:53 2012 +0400
> > @@ -172,6 +172,7 @@ else
> >              ngx_feature="PCRE JIT support"
> >              ngx_feature_name="NGX_HAVE_PCRE_JIT"
> >              ngx_feature_test="int jit = 0;
> > +                              pcre_free_study(NULL);
> >                                pcre_config(PCRE_CONFIG_JIT, &jit);
> >                                if (jit != 1) return 1;"
> >              . auto/feature
> >
> > probably will be good enough.  It will prevent build failures
> > by checking if the code we are going to compile will compile.
> >
> > (Yes, I understand that it won't resolve the root cause, i.e. the
> > library vs. headers mismatch.  And yes, I understand that it will
> > make PCRE JIT unavailable in some cases.  I don't think we care
> > though.)
> 
> While I agree that this is a better test for PCRE JIT presence and
> that it should be included regardless of the OSX issue, I don't think
> that this is solution to the problem at hand, because all it does is
> hide the issue from the end user... Putting JIT capabilities aside, I
> also dislike the fact that we might be throwing away almost 3 years of
> PCRE fixes just because we cannot be bothered to link against newer
> library that's available outside of the default search path... I
> definitely prefer the solution with postponed check for the default
> search path on OSX.

The problem is that the issue isn't specific to PCRE, hence it 
can't be fixed by a patch on auto/lib/pcre/conf.  It's about 
toolchain, and all libraries are equally affected.  It's just 
happened to be visible with PCRE due to interface change and 
resulting build failures.  So all patches on auto/lib/pcre/conf is 
about hiding the issue from the end user, nothing more.

Correct fix would be to add -L/usr/local/lib in such cases (or 
-L/opt/local/lib in case of MacPorts toolchain with MacPorts 
installed into /opt/local), but I can't think of an easy way to 
correctly detect if we need it.

> Alternatively, we could first try looking for the PCRE library with
> JIT support and fallback to looking for the PCRE library without one:

[...]

> but honestly, it seems like a bit of an overkill.

Looks like an overkill for me, too.  I'll commit PCRE JIT test 
modifications for now.

-- 
Maxim Dounin
http://nginx.com/support.html


From mdounin at mdounin.ru  Wed Dec 12 13:50:07 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Wed, 12 Dec 2012 13:50:07 +0000
Subject: [nginx] svn commit:  r4961 - trunk/auto/lib/pcre
Message-ID: <20121212135007.C40C53F9E74@mail.nginx.com>

Author: mdounin
Date: 2012-12-12 13:50:07 +0000 (Wed, 12 Dec 2012)
New Revision: 4961
URL: http://trac.nginx.org/nginx/changeset/4961/nginx

Log:
Configure: better check for PCRE JIT.

On Mac OS X system toolchain by default prefers include files
from /usr/local/include, but libraries from /usr/lib.  This might result in
various problems, in particular the one outlined below.

If the PCRE library is installed into /usr/local/, this results in pcre.h
being used from /usr/local/include (with PCRE_CONFIG_JIT defined), but
libpcre from /usr/lib (as shipped with the OS, without pcre_free_study()
symbol).  As a result build fails as we use pcre_free_study() function
if we try to compile with PCRE JIT support.

Obvious workaround is to the root cause is to ask compiler to prefer
library from /usr/local/lib via ./configure --with-ld-opt="-L/usr/local/lib".
On the other hand, in any case it would be good to check if the function
we are going to use is available, hence the change.

See thread here for details:
http://mailman.nginx.org/pipermail/nginx-devel/2012-December/003074.html

Prodded by Piotr Sikora.


Modified:
   trunk/auto/lib/pcre/conf

Modified: trunk/auto/lib/pcre/conf
===================================================================
--- trunk/auto/lib/pcre/conf	2012-12-11 14:26:07 UTC (rev 4960)
+++ trunk/auto/lib/pcre/conf	2012-12-12 13:50:07 UTC (rev 4961)
@@ -172,6 +172,7 @@
             ngx_feature="PCRE JIT support"
             ngx_feature_name="NGX_HAVE_PCRE_JIT"
             ngx_feature_test="int jit = 0;
+                              pcre_free_study(NULL);
                               pcre_config(PCRE_CONFIG_JIT, &jit);
                               if (jit != 1) return 1;"
             . auto/feature


From vbart at nginx.com  Wed Dec 12 14:48:48 2012
From: vbart at nginx.com (vbart at nginx.com)
Date: Wed, 12 Dec 2012 14:48:48 +0000
Subject: [nginx] svn commit:  r4962 - trunk/src/http
Message-ID: <20121212144848.DEC0C3F9FA5@mail.nginx.com>

Author: vbart
Date: 2012-12-12 14:48:48 +0000 (Wed, 12 Dec 2012)
New Revision: 4962
URL: http://trac.nginx.org/nginx/changeset/4962/nginx

Log:
Limit rate: fixed integer overflow in limit calculation (ticket #256).

Patch by Alexey Antropov.



Modified:
   trunk/src/http/ngx_http_write_filter_module.c

Modified: trunk/src/http/ngx_http_write_filter_module.c
===================================================================
--- trunk/src/http/ngx_http_write_filter_module.c	2012-12-12 13:50:07 UTC (rev 4961)
+++ trunk/src/http/ngx_http_write_filter_module.c	2012-12-12 14:48:48 UTC (rev 4962)
@@ -207,7 +207,7 @@
     }
 
     if (r->limit_rate) {
-        limit = r->limit_rate * (ngx_time() - r->start_sec + 1)
+        limit = (off_t) r->limit_rate * (ngx_time() - r->start_sec + 1)
                 - (c->sent - clcf->limit_rate_after);
 
         if (limit <= 0) {


From mdounin at mdounin.ru  Wed Dec 12 15:10:55 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 12 Dec 2012 19:10:55 +0400
Subject: [PATCH] Core: fix content handler and location configuration
 mismatch.
In-Reply-To: <CADMhe6favQL6RdUKgCi_mEs3Ju0934QNmE0guqC8iOR1QVWgmQ@mail.gmail.com>
References: <CADMhe6cDf57_0wDvTt3i9JwuOGC8UTSwpUyToo8UgsE+HgT=zA@mail.gmail.com>
 <20121207162745.GB40452@mdounin.ru>
 <CADMhe6fM4gjYUNjHozy56oJp3KEi2WkfT1h_PQ8P3=kR1DdjqQ@mail.gmail.com>
 <20121210010348.GH40452@mdounin.ru>
 <CADMhe6favQL6RdUKgCi_mEs3Ju0934QNmE0guqC8iOR1QVWgmQ@mail.gmail.com>
Message-ID: <20121212151055.GO40452@mdounin.ru>

Hello!

On Tue, Dec 11, 2012 at 01:12:30AM -0800, Piotr Sikora wrote:

[...]

> --- a/src/http/ngx_http_core_module.c
> +++ b/src/http/ngx_http_core_module.c
> @@ -1400,6 +1400,14 @@ ngx_http_core_content_phase(ngx_http_request_t *r,
>      ngx_str_t  path;
> 
>      if (r->content_handler) {
> +        if (r->loc_conf != r->content_handler_loc_conf) {
> +            ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> +                          "content handler and configuration mismatch");
> +
> +            ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
> +            return NGX_OK;
> +        }
> +
>          r->write_event_handler = ngx_http_request_empty_handler;
>          ngx_http_finalize_request(r, r->content_handler(r));
>          return NGX_OK;
> @@ -1526,6 +1534,7 @@ ngx_http_update_location_config(ngx_http_request_t *r)
> 
>      if (clcf->handler) {
>          r->content_handler = clcf->handler;
> +        r->content_handler_loc_conf = r->loc_conf;
>      }
>  }

As far as I can tell, this will break configs like

    location / {
        proxy_pass http://backend1;

        set $true 1;
        if ($true) {
            proxy_pass http://backend2;
        }

        if ($true) {
            # nothing
        }
    }

which currently works fine.  I mean something like this instead:

--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -636,6 +636,14 @@ ngx_http_upstream_init_request(ngx_http_
 
 found:
 
+    if (uscf == NULL) {
+        ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0,
+                      "no upstream configuration");
+        ngx_http_upstream_finalize_request(r, u,
+                                           NGX_HTTP_INTERNAL_SERVER_ERROR);
+        return;
+    }
+
     if (uscf->peer.init(r, uscf) != NGX_OK) {
         ngx_http_upstream_finalize_request(r, u,
                                            NGX_HTTP_INTERNAL_SERVER_ERROR);

-- 
Maxim Dounin
http://nginx.com/support.html


From toli at webforge.bg  Thu Dec 13 13:24:18 2012
From: toli at webforge.bg (Anatoli Marinov)
Date: Thu, 13 Dec 2012 15:24:18 +0200
Subject: configuration reload
In-Reply-To: <20121125105534.GD40452@mdounin.ru>
References: <50AF9C11.2020206@webforge.bg> <20121125105534.GD40452@mdounin.ru>
Message-ID: <50C9D702.6050706@webforge.bg>

What do you exactly mean with "dynamic configuration"?
I cannot find anything about it ...

On 11/25/2012 12:55 PM, Maxim Dounin wrote:
> Hello!
>
> On Fri, Nov 23, 2012 at 05:53:53PM +0200, Anatoli Marinov wrote:
>
>> *Hello colleagues,
>> We have some troubles when nginx have to reload its configuration.
>> *There is a*control panel and every user which has account may
>> change its own settings. For example it may add new domain name or
>> change other option. After that new config files are created and
>> they are sent to the servers.
>> When there is a new config file master process receives HUP signal
>> and it loads this new configuration. After the configuration is
>> loaded the master process starts new workers and waits for old
>> workers to complete its responses with the end users.  Usually there
>> are several slow connections and old workers can stay alive for a
>> long time.
>> If there are several new configs for small period of time the number
>> of workers may become very high and servers' load also gets high
>> values. In general this is not good for the service.
> That's why it's generally not recommended to do automated
> configuration reloads.  If you do automated reloads - you have to
> implement some safeguards to prevent situation of too many worker
> processes shutting down by either preventing further reloads till
> some workers exit, or killing oldest shutting down workers if
> another reload is strictly required.
>
>> Do you have any idea how this issue could be solved? Is there
>> different way to set new config file without a proc*ess *of new
>> workers creation?
> Current nginx configuration reload code uses new worker processes
> to apply new configuration.  Reimplementing this with some other
> mechanism is certainly possible, but wouldn't be trivial (and I
> suspect might result in other problems).  I would recommend you to
> focus on dynamic configuration of aspects you want to be
> user-configurable without changing main nginx configuration.
>


From mdounin at mdounin.ru  Thu Dec 13 13:45:39 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 13 Dec 2012 13:45:39 +0000
Subject: [nginx] svn commit:  r4963 - trunk/src/http/modules
Message-ID: <20121213134539.A7C403F9FA5@mail.nginx.com>

Author: mdounin
Date: 2012-12-13 13:45:39 +0000 (Thu, 13 Dec 2012)
New Revision: 4963
URL: http://trac.nginx.org/nginx/changeset/4963/nginx

Log:
Proxy: better error message about unexpected data.

Requested by Igor Sysoev.


Modified:
   trunk/src/http/modules/ngx_http_proxy_module.c

Modified: trunk/src/http/modules/ngx_http_proxy_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_proxy_module.c	2012-12-12 14:48:48 UTC (rev 4962)
+++ trunk/src/http/modules/ngx_http_proxy_module.c	2012-12-13 13:45:39 UTC (rev 4963)
@@ -1614,7 +1614,8 @@
         p->upstream_done = 1;
 
         ngx_log_error(NGX_LOG_WARN, r->connection->log, 0,
-                      "upstream sent too much data");
+                      "upstream sent more data than specified in "
+                      "\"Content-Length\" header");
     }
 
     return NGX_OK;


From ru at nginx.com  Thu Dec 13 15:05:19 2012
From: ru at nginx.com (ru at nginx.com)
Date: Thu, 13 Dec 2012 15:05:19 +0000
Subject: [nginx] svn commit:  r4964 - in trunk/src/http/modules: . perl
Message-ID: <20121213150519.B6AEC3FA034@mail.nginx.com>

Author: ru
Date: 2012-12-13 15:05:19 +0000 (Thu, 13 Dec 2012)
New Revision: 4964
URL: http://trac.nginx.org/nginx/changeset/4964/nginx

Log:
Fixed variable syntax checking in "set", "geo", "limit_conn_zone",
and "perl_set" directives.


Modified:
   trunk/src/http/modules/ngx_http_geo_module.c
   trunk/src/http/modules/ngx_http_limit_conn_module.c
   trunk/src/http/modules/ngx_http_rewrite_module.c
   trunk/src/http/modules/perl/ngx_http_perl_module.c

Modified: trunk/src/http/modules/ngx_http_geo_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_geo_module.c	2012-12-13 13:45:39 UTC (rev 4963)
+++ trunk/src/http/modules/ngx_http_geo_module.c	2012-12-13 15:05:19 UTC (rev 4964)
@@ -322,6 +322,13 @@
     }
 
     name = value[1];
+
+    if (name.len < 2 || name.data[0] != '$') {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "invalid variable name \"%V\"", &name);
+        return NGX_CONF_ERROR;
+    }
+
     name.len--;
     name.data++;
 

Modified: trunk/src/http/modules/ngx_http_limit_conn_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_limit_conn_module.c	2012-12-13 13:45:39 UTC (rev 4963)
+++ trunk/src/http/modules/ngx_http_limit_conn_module.c	2012-12-13 15:05:19 UTC (rev 4964)
@@ -540,7 +540,7 @@
             continue;
         }
 
-        if (value[i].data[0] == '$') {
+        if (value[i].len > 1 && value[i].data[0] == '$') {
 
             value[i].len--;
             value[i].data++;
@@ -613,7 +613,7 @@
 
     value = cf->args->elts;
 
-    if (value[2].data[0] != '$') {
+    if (value[2].len < 2 || value[2].data[0] != '$') {
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "invalid variable name \"%V\"", &value[2]);
         return NGX_CONF_ERROR;

Modified: trunk/src/http/modules/ngx_http_rewrite_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_rewrite_module.c	2012-12-13 13:45:39 UTC (rev 4963)
+++ trunk/src/http/modules/ngx_http_rewrite_module.c	2012-12-13 15:05:19 UTC (rev 4964)
@@ -908,7 +908,7 @@
 
     value = cf->args->elts;
 
-    if (value[1].data[0] != '$') {
+    if (value[1].len < 2 || value[1].data[0] != '$') {
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "invalid variable name \"%V\"", &value[1]);
         return NGX_CONF_ERROR;

Modified: trunk/src/http/modules/perl/ngx_http_perl_module.c
===================================================================
--- trunk/src/http/modules/perl/ngx_http_perl_module.c	2012-12-13 13:45:39 UTC (rev 4963)
+++ trunk/src/http/modules/perl/ngx_http_perl_module.c	2012-12-13 15:05:19 UTC (rev 4964)
@@ -968,7 +968,7 @@
 
     value = cf->args->elts;
 
-    if (value[1].data[0] != '$') {
+    if (value[1].len < 2 || value[1].data[0] != '$') {
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "invalid variable name \"%V\"", &value[1]);
         return NGX_CONF_ERROR;


From mdounin at mdounin.ru  Thu Dec 13 15:35:54 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 13 Dec 2012 19:35:54 +0400
Subject: configuration reload
In-Reply-To: <50C9D702.6050706@webforge.bg>
References: <50AF9C11.2020206@webforge.bg> <20121125105534.GD40452@mdounin.ru>
 <50C9D702.6050706@webforge.bg>
Message-ID: <20121213153554.GB40452@mdounin.ru>

Hello!

On Thu, Dec 13, 2012 at 03:24:18PM +0200, Anatoli Marinov wrote:

> What do you exactly mean with "dynamic configuration"?
> I cannot find anything about it ...

It's up to you how to do it.  Various examples include:

1) Some functionality may be configured dynamically with files on 
filesystem and checking them with try_files / if.  E.g. this is 
how people typically implement "maintenance" page.

2) The auth_basic module uses data from auth_basic_user_file, 
which is expected to be dynamically changed.  Currently it's just 
re-read on each request.

[...]

-- 
Maxim Dounin
http://nginx.com/support.html


From mdounin at mdounin.ru  Thu Dec 13 16:06:00 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 13 Dec 2012 16:06:00 +0000
Subject: [nginx] svn commit:  r4965 - trunk/src/http
Message-ID: <20121213160600.629C73F9C0F@mail.nginx.com>

Author: mdounin
Date: 2012-12-13 16:05:59 +0000 (Thu, 13 Dec 2012)
New Revision: 4965
URL: http://trac.nginx.org/nginx/changeset/4965/nginx

Log:
Upstream: fixed SIGSEGV with the "if" directive.

Configuration like

    location / {
        set $true 1;

        if ($true) {
            proxy_pass http://backend;
        }

        if ($true) {
            # nothing
        }
    }

resulted in segmentation fault due to NULL pointer dereference as the
upstream configuration wasn't initialized in an implicit location created
by the last if(), but the r->content_handler was set due to first if().

Instead of committing a suicide by dereferencing a NULL pointer, return
500 (Internal Server Error) in such cases, i.e. if uscf is NULL.  Better
fix would be to avoid such cases by fixing the "if" directive handling,
but it's out of scope of this patch.

Prodded by Piotr Sikora.


Modified:
   trunk/src/http/ngx_http_upstream.c

Modified: trunk/src/http/ngx_http_upstream.c
===================================================================
--- trunk/src/http/ngx_http_upstream.c	2012-12-13 15:05:19 UTC (rev 4964)
+++ trunk/src/http/ngx_http_upstream.c	2012-12-13 16:05:59 UTC (rev 4965)
@@ -636,6 +636,14 @@
 
 found:
 
+    if (uscf == NULL) {
+        ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0,
+                      "no upstream configuration");
+        ngx_http_upstream_finalize_request(r, u,
+                                           NGX_HTTP_INTERNAL_SERVER_ERROR);
+        return;
+    }
+
     if (uscf->peer.init(r, uscf) != NGX_OK) {
         ngx_http_upstream_finalize_request(r, u,
                                            NGX_HTTP_INTERNAL_SERVER_ERROR);


From crk_world at yahoo.com.cn  Fri Dec 14 03:39:24 2012
From: crk_world at yahoo.com.cn (chen cw)
Date: Fri, 14 Dec 2012 11:39:24 +0800
Subject: fresh to nginx
In-Reply-To: <CANA-K5XgrbzhkJfBY2_WYRdjBpKEZb8aFkKm_k9Hrcf9ES9=5A@mail.gmail.com>
References: <CANA-K5Ughn0OD9UuvTs0hwubxS4ed4Pia3yqi6kvHQhiBMm0rw@mail.gmail.com>
 <CAAGYmFeYo-vUu5pKKyVV50wDJ1skgLihQMKQTy4yMWhNsZr_+A@mail.gmail.com>
 <CANA-K5XgrbzhkJfBY2_WYRdjBpKEZb8aFkKm_k9Hrcf9ES9=5A@mail.gmail.com>
Message-ID: <CAKPCrs1k97SZ-16uMt0ZBVe74oRyrr6se66F1afyzA5Ff3tJFA@mail.gmail.com>

I recommend this article to you

http://tengine.taobao.org/book/chapter_2.html#ngx-pool-t-100

as you are analysising nginx pool




On Wed, Dec 12, 2012 at 4:09 PM, ??? <wangzhiming1757 at gmail.com> wrote:

> hi Matthieu,
>
> Thanks for your suggestion and guideline, and now i am analysising the
> Nginx memory pool, it is a little more complicated, maybe i should take
> more time on it, hope enjoy it.
>
> brgs\ thanks
> Zhiming
>
>
> 2012/12/12 Matthieu Tourne <matthieu.tourne at gmail.com>
>
>>
>> Hi Zhiming!
>>
>> On Tue, Dec 11, 2012 at 10:55 PM, ??? <wangzhiming1757 at gmail.com> wrote:
>>
>>> hi all,
>>>
>>> I am a fresh man to nginx, who can tell me how to read the nginx source
>>> code in general, i just trace the source from main() entry, if this is the
>>> right  manner.
>>>
>>> and any related documents are welcomed.
>>>
>>
>> There are some great guides on the web to get started :
>>
>> Evan Miller Guide To Nginx Module Development [1] and advanced topics [2]
>> A bit old, but definitely not dated if you want to learn how it works
>> inside.
>>
>> Nginx Guts [3] also has quite a few articles on the core of Nginx.
>>
>> If you're interested in adding new features, you can find many Nginx [4]
>> or 3rd-party modules [5],
>> they all add new extensions and capabilities to the core of Nginx.
>>
>> As for the main() I think it parses the command line options, reads the
>> configuration and spawn the worker processes.
>>  Then each worker process will start accepting and processing HTTP
>> requests.
>>
>> Hope that helps,
>>
>> Matthieu.
>>
>>
>> [1] http://www.evanmiller.org/nginx-modules-guide.html
>> [2] http://www.evanmiller.org/nginx-modules-guide-advanced.html
>> [3] http://www.nginxguts.com/2011/01/phases/
>> [4] http://wiki.nginx.org/Modules
>> [5] http://wiki.nginx.org/3rdPartyModules
>>
>> _______________________________________________
>> nginx-devel mailing list
>> nginx-devel at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>>
>
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>



-- 

--

Charles Chen

Software Engineer

Server Platforms Team at Taobao.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121214/70b4b8c2/attachment.html>

From wangzhiming1757 at gmail.com  Fri Dec 14 03:51:56 2012
From: wangzhiming1757 at gmail.com (=?GB2312?B?zfXWvsP3?=)
Date: Fri, 14 Dec 2012 11:51:56 +0800
Subject: fresh to nginx
In-Reply-To: <CAKPCrs1k97SZ-16uMt0ZBVe74oRyrr6se66F1afyzA5Ff3tJFA@mail.gmail.com>
References: <CANA-K5Ughn0OD9UuvTs0hwubxS4ed4Pia3yqi6kvHQhiBMm0rw@mail.gmail.com>
 <CAAGYmFeYo-vUu5pKKyVV50wDJ1skgLihQMKQTy4yMWhNsZr_+A@mail.gmail.com>
 <CANA-K5XgrbzhkJfBY2_WYRdjBpKEZb8aFkKm_k9Hrcf9ES9=5A@mail.gmail.com>
 <CAKPCrs1k97SZ-16uMt0ZBVe74oRyrr6se66F1afyzA5Ff3tJFA@mail.gmail.com>
Message-ID: <CANA-K5UxOH9fNdisT=vKG7y6tuysA3sHSn=4T8HoF3k2FD6+uA@mail.gmail.com>

Really Thanks, it is the right i m finding.


2012/12/14 chen cw <crk_world at yahoo.com.cn>

> I recommend this article to you
>
> http://tengine.taobao.org/book/chapter_2.html#ngx-pool-t-100
>
> as you are analysising nginx pool
>
>
>
>
> On Wed, Dec 12, 2012 at 4:09 PM, ??? <wangzhiming1757 at gmail.com> wrote:
>
>> hi Matthieu,
>>
>> Thanks for your suggestion and guideline, and now i am analysising the
>> Nginx memory pool, it is a little more complicated, maybe i should take
>> more time on it, hope enjoy it.
>>
>> brgs\ thanks
>> Zhiming
>>
>>
>> 2012/12/12 Matthieu Tourne <matthieu.tourne at gmail.com>
>>
>>>
>>> Hi Zhiming!
>>>
>>> On Tue, Dec 11, 2012 at 10:55 PM, ??? <wangzhiming1757 at gmail.com> wrote:
>>>
>>>> hi all,
>>>>
>>>> I am a fresh man to nginx, who can tell me how to read the nginx source
>>>> code in general, i just trace the source from main() entry, if this is the
>>>> right  manner.
>>>>
>>>> and any related documents are welcomed.
>>>>
>>>
>>> There are some great guides on the web to get started :
>>>
>>> Evan Miller Guide To Nginx Module Development [1] and advanced topics [2]
>>> A bit old, but definitely not dated if you want to learn how it works
>>> inside.
>>>
>>> Nginx Guts [3] also has quite a few articles on the core of Nginx.
>>>
>>> If you're interested in adding new features, you can find many Nginx [4]
>>> or 3rd-party modules [5],
>>> they all add new extensions and capabilities to the core of Nginx.
>>>
>>> As for the main() I think it parses the command line options, reads the
>>> configuration and spawn the worker processes.
>>>  Then each worker process will start accepting and processing HTTP
>>> requests.
>>>
>>> Hope that helps,
>>>
>>> Matthieu.
>>>
>>>
>>> [1] http://www.evanmiller.org/nginx-modules-guide.html
>>> [2] http://www.evanmiller.org/nginx-modules-guide-advanced.html
>>> [3] http://www.nginxguts.com/2011/01/phases/
>>> [4] http://wiki.nginx.org/Modules
>>> [5] http://wiki.nginx.org/3rdPartyModules
>>>
>>> _______________________________________________
>>> nginx-devel mailing list
>>> nginx-devel at nginx.org
>>> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>>>
>>
>>
>> _______________________________________________
>> nginx-devel mailing list
>> nginx-devel at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>>
>
>
>
> --
>
> --
>
> Charles Chen
>
> Software Engineer
>
> Server Platforms Team at Taobao.com
>
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121214/d6f448e7/attachment-0001.html>

From vbart at nginx.com  Fri Dec 14 15:17:58 2012
From: vbart at nginx.com (vbart at nginx.com)
Date: Fri, 14 Dec 2012 15:17:58 +0000
Subject: [nginx] svn commit:  r4966 - trunk/src/http/modules
Message-ID: <20121214151758.9BA223FA016@mail.nginx.com>

Author: vbart
Date: 2012-12-14 15:17:58 +0000 (Fri, 14 Dec 2012)
New Revision: 4966
URL: http://trac.nginx.org/nginx/changeset/4966/nginx

Log:
Gzip: fixed zlib memLevel adjusting.

An incorrect memLevel (lower than 1) might be passed to deflateInit2() if the
"gzip_hash" directive is set to a value less than the value of "gzip_window"
directive. This resulted in "deflateInit2() failed: -2" alert and an empty
reply.



Modified:
   trunk/src/http/modules/ngx_http_gzip_filter_module.c

Modified: trunk/src/http/modules/ngx_http_gzip_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_gzip_filter_module.c	2012-12-13 16:05:59 UTC (rev 4965)
+++ trunk/src/http/modules/ngx_http_gzip_filter_module.c	2012-12-14 15:17:58 UTC (rev 4966)
@@ -498,6 +498,10 @@
             wbits--;
             memlevel--;
         }
+
+        if (memlevel < 1) {
+            memlevel = 1;
+        }
     }
 
     ctx->wbits = wbits;


From vbart at nginx.com  Fri Dec 14 15:24:25 2012
From: vbart at nginx.com (vbart at nginx.com)
Date: Fri, 14 Dec 2012 15:24:25 +0000
Subject: [nginx] svn commit:  r4967 - trunk/src/core
Message-ID: <20121214152425.1E0FB3FA0C6@mail.nginx.com>

Author: vbart
Date: 2012-12-14 15:24:24 +0000 (Fri, 14 Dec 2012)
New Revision: 4967
URL: http://trac.nginx.org/nginx/changeset/4967/nginx

Log:
Fixed handling of ngx_write_fd() and ngx_read_fd() errors.

The ngx_write_fd() and ngx_read_fd() functions return -1 in case of error,
so the incorrect comparison with NGX_FILE_ERROR (which is 0 on windows
platforms) might result in inaccurate error message in the error log.

Also the ngx_errno global variable is being set only if the returned value
is -1.



Modified:
   trunk/src/core/ngx_conf_file.c
   trunk/src/core/ngx_cycle.c
   trunk/src/core/ngx_file.c

Modified: trunk/src/core/ngx_conf_file.c
===================================================================
--- trunk/src/core/ngx_conf_file.c	2012-12-14 15:17:58 UTC (rev 4966)
+++ trunk/src/core/ngx_conf_file.c	2012-12-14 15:24:24 UTC (rev 4967)
@@ -983,7 +983,7 @@
 
         n = ngx_write_fd(file[i].fd, file[i].buffer, len);
 
-        if (n == NGX_FILE_ERROR) {
+        if (n == -1) {
             ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno,
                           ngx_write_fd_n " to \"%s\" failed",
                           file[i].name.data);

Modified: trunk/src/core/ngx_cycle.c
===================================================================
--- trunk/src/core/ngx_cycle.c	2012-12-14 15:17:58 UTC (rev 4966)
+++ trunk/src/core/ngx_cycle.c	2012-12-14 15:24:24 UTC (rev 4967)
@@ -1145,7 +1145,7 @@
 
             n = ngx_write_fd(file[i].fd, file[i].buffer, len);
 
-            if (n == NGX_FILE_ERROR) {
+            if (n == -1) {
                 ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno,
                               ngx_write_fd_n " to \"%s\" failed",
                               file[i].name.data);

Modified: trunk/src/core/ngx_file.c
===================================================================
--- trunk/src/core/ngx_file.c	2012-12-14 15:17:58 UTC (rev 4966)
+++ trunk/src/core/ngx_file.c	2012-12-14 15:24:24 UTC (rev 4967)
@@ -732,14 +732,14 @@
 
         n = ngx_read_fd(fd, buf, len);
 
-        if (n == NGX_FILE_ERROR) {
+        if (n == -1) {
             ngx_log_error(NGX_LOG_ALERT, cf->log, ngx_errno,
                           ngx_read_fd_n " \"%s\" failed", from);
             goto failed;
         }
 
         if ((size_t) n != len) {
-            ngx_log_error(NGX_LOG_ALERT, cf->log, ngx_errno,
+            ngx_log_error(NGX_LOG_ALERT, cf->log, 0,
                           ngx_read_fd_n " has read only %z of %uz from %s",
                           n, size, from);
             goto failed;
@@ -747,14 +747,14 @@
 
         n = ngx_write_fd(nfd, buf, len);
 
-        if (n == NGX_FILE_ERROR) {
+        if (n == -1) {
             ngx_log_error(NGX_LOG_ALERT, cf->log, ngx_errno,
                           ngx_write_fd_n " \"%s\" failed", to);
             goto failed;
         }
 
         if ((size_t) n != len) {
-            ngx_log_error(NGX_LOG_ALERT, cf->log, ngx_errno,
+            ngx_log_error(NGX_LOG_ALERT, cf->log, 0,
                           ngx_write_fd_n " has written only %z of %uz to %s",
                           n, size, to);
             goto failed;


From ru at nginx.com  Fri Dec 14 19:35:38 2012
From: ru at nginx.com (ru at nginx.com)
Date: Fri, 14 Dec 2012 19:35:38 +0000
Subject: [nginx] svn commit:  r4968 - trunk/src/http/modules
Message-ID: <20121214193539.2AC9C3FA135@mail.nginx.com>

Author: ru
Date: 2012-12-14 19:35:37 +0000 (Fri, 14 Dec 2012)
New Revision: 4968
URL: http://trac.nginx.org/nginx/changeset/4968/nginx

Log:
Geo: improved ngx_http_geo_block() code readability.


Modified:
   trunk/src/http/modules/ngx_http_geo_module.c

Modified: trunk/src/http/modules/ngx_http_geo_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_geo_module.c	2012-12-14 15:24:24 UTC (rev 4967)
+++ trunk/src/http/modules/ngx_http_geo_module.c	2012-12-14 19:35:37 UTC (rev 4968)
@@ -303,7 +303,6 @@
 ngx_http_geo_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
 {
     char                     *rv;
-    void                    **p;
     size_t                    len;
     ngx_str_t                *value, name;
     ngx_uint_t                i;
@@ -402,8 +401,8 @@
                     return NGX_CONF_ERROR;
                 }
 
-                p = (void **) ngx_cpymem(ctx.high.low[i], a->elts, len);
-                *p = NULL;
+                ngx_memcpy(ctx.high.low[i], a->elts, len);
+                ctx.high.low[i][a->nelts].value = NULL;
                 ctx.data_size += len + sizeof(void *);
             }
 


From ru at nginx.com  Fri Dec 14 19:56:03 2012
From: ru at nginx.com (ru at nginx.com)
Date: Fri, 14 Dec 2012 19:56:03 +0000
Subject: [nginx] svn commit:  r4969 - trunk/src/http/modules
Message-ID: <20121214195603.863083F9C18@mail.nginx.com>

Author: ru
Date: 2012-12-14 19:56:03 +0000 (Fri, 14 Dec 2012)
New Revision: 4969
URL: http://trac.nginx.org/nginx/changeset/4969/nginx

Log:
Geo: fixed the "ranges" without ranges case.

The following configuration returned an empty value for $geo:

    geo $geo {
        ranges;
        default default;
    }


Modified:
   trunk/src/http/modules/ngx_http_geo_module.c

Modified: trunk/src/http/modules/ngx_http_geo_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_geo_module.c	2012-12-14 19:35:37 UTC (rev 4968)
+++ trunk/src/http/modules/ngx_http_geo_module.c	2012-12-14 19:56:03 UTC (rev 4969)
@@ -189,19 +189,22 @@
 
     *v = *ctx->u.high.default_value;
 
-    addr = ngx_http_geo_addr(r, ctx);
-
-    range = ctx->u.high.low[addr >> 16];
-
-    if (range) {
-        n = addr & 0xffff;
-        do {
-            if (n >= (ngx_uint_t) range->start && n <= (ngx_uint_t) range->end)
-            {
-                *v = *range->value;
-                break;
-            }
-        } while ((++range)->value);
+    if (ctx->u.high.low) {
+        addr = ngx_http_geo_addr(r, ctx);
+    
+        range = ctx->u.high.low[addr >> 16];
+    
+        if (range) {
+            n = addr & 0xffff;
+            do {
+                if (n >= (ngx_uint_t) range->start
+                    && n <= (ngx_uint_t) range->end)
+                {
+                    *v = *range->value;
+                    break;
+                }
+            } while ((++range)->value);
+        }
     }
 
     ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
@@ -384,9 +387,9 @@
     geo->proxies = ctx.proxies;
     geo->proxy_recursive = ctx.proxy_recursive;
 
-    if (ctx.high.low) {
+    if (ctx.ranges) {
 
-        if (!ctx.binary_include) {
+        if (ctx.high.low && !ctx.binary_include) {
             for (i = 0; i < 0x10000; i++) {
                 a = (ngx_array_t *) ctx.high.low[i];
 


From ibobrik at gmail.com  Sun Dec 16 15:55:39 2012
From: ibobrik at gmail.com (ivan babrou)
Date: Sun, 16 Dec 2012 19:55:39 +0400
Subject: image_filter enhancement
In-Reply-To: <20121205145244.GO40452@mdounin.ru>
References: <CANWdNRDP_6+WHtCawX8KnoH4GFHHQeHC2tXY6-8gDctYq0Dnrg@mail.gmail.com>
 <20121106181601.GS40452@mdounin.ru>
 <CANWdNRDYuMDZOpcEwtsryfUpTBCsdB6VsSJfF+qUJV34y5j08g@mail.gmail.com>
 <20121114234732.GC40452@mdounin.ru>
 <CANWdNRB=bRCEjEBM+ksak8c6pEZw+VC1XfKBrPLEJ5zXv+B_=g@mail.gmail.com>
 <20121127160328.GZ40452@mdounin.ru>
 <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>
 <CANWdNRBXuGyFdo1NGUeHba2QnSYokMJpD61tW-VNcOo=TV-9gg@mail.gmail.com>
 <20121204130841.GI40452@mdounin.ru>
 <CANWdNRATaVeEC7W0UVos9AdDM4Xq5BNRQUDea+UZkCuZzGL8gA@mail.gmail.com>
 <20121205145244.GO40452@mdounin.ru>
Message-ID: <CANWdNRBW-uWR66FLqiEGu03n5wzLFfvQA+iXLxxVop-0J5bSbw@mail.gmail.com>

On 5 December 2012 18:52, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Tue, Dec 04, 2012 at 06:58:17PM +0400, ivan babrou wrote:
>
> > >
> > > Additionally, looking at the code I tend to think it's incorrect.
> > > That is, it's pattern you've followed is incorrect, not your code.
> > > E.g. the following config will result in $arg_q incorrectly used
> > > for quality in /image/, instead of "80" explicitly set:
> > >     image_filter_jpeg_quality $arg_q;
> > >     location /image/ {
> > >         image_filter crop $arg_w $arg_h;
> > >         image_filter_jpeg_quality 80;
> > >     }
> > > This needs fixing.
> >
> >
> > As I see other config variables are buggy too. Should I fix them too?
>
> It would be fine to produce a patch which fixes existing config
> variables, and the patch to add crop offsets on top of it.


Can you give me some good examples to see "good practices" in nginx
configuration handling?


>  > Anyway, this is latest version of my patch where I fixed other issues:
>
> I don't see at least some of previously expressed points
> addressed, so I assume you are still working on it.
>
> Please also see other point rightfully noted by Ruslan Khusnullin -
> you can't use ngx_strcmp() in ngx_http_image_filter_get_value() as
> it's used not only on null-terminated strings.
>
> --
> Maxim Dounin
> http://nginx.com/support.html
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>



-- 
Regards, Ian Babrou
http://bobrik.name http://twitter.com/ibobrik skype:i.babrou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121216/ca1695fa/attachment.html>

From ru at nginx.com  Mon Dec 17 09:31:53 2012
From: ru at nginx.com (ru at nginx.com)
Date: Mon, 17 Dec 2012 09:31:53 +0000
Subject: [nginx] svn commit:  r4970 - trunk/src/core
Message-ID: <20121217093153.DFD313F9C0F@mail.nginx.com>

Author: ru
Date: 2012-12-17 09:31:53 +0000 (Mon, 17 Dec 2012)
New Revision: 4970
URL: http://trac.nginx.org/nginx/changeset/4970/nginx

Log:
Simplified URL parsing code.

Except for the "listen" directive, "*" specified as a hostname is
no longer treated specially.


Modified:
   trunk/src/core/ngx_inet.c

Modified: trunk/src/core/ngx_inet.c
===================================================================
--- trunk/src/core/ngx_inet.c	2012-12-14 19:56:03 UTC (rev 4969)
+++ trunk/src/core/ngx_inet.c	2012-12-17 09:31:53 UTC (rev 4970)
@@ -705,6 +705,11 @@
         }
 
         u->no_port = 1;
+
+        if (!u->no_resolve) {
+            u->port = u->default_port;
+            sin->sin_port = htons(u->default_port);
+        }
     }
 
     len = last - host;
@@ -714,54 +719,45 @@
         return NGX_ERROR;
     }
 
-    if (len == 1 && *host == '*') {
-        len = 0;
-    }
-
     u->host.len = len;
     u->host.data = host;
 
+    if (u->listen && len == 1 && *host == '*') {
+        sin->sin_addr.s_addr = INADDR_ANY;
+        u->wildcard = 1;
+        return NGX_OK;
+    }
+
     if (u->no_resolve) {
         return NGX_OK;
     }
 
-    if (len) {
-        sin->sin_addr.s_addr = ngx_inet_addr(host, len);
+    sin->sin_addr.s_addr = ngx_inet_addr(host, len);
 
-        if (sin->sin_addr.s_addr == INADDR_NONE) {
-            p = ngx_alloc(++len, pool->log);
-            if (p == NULL) {
-                return NGX_ERROR;
-            }
+    if (sin->sin_addr.s_addr == INADDR_NONE) {
+        p = ngx_alloc(++len, pool->log);
+        if (p == NULL) {
+            return NGX_ERROR;
+        }
 
-            (void) ngx_cpystrn(p, host, len);
+        (void) ngx_cpystrn(p, host, len);
 
-            h = gethostbyname((const char *) p);
+        h = gethostbyname((const char *) p);
 
-            ngx_free(p);
+        ngx_free(p);
 
-            if (h == NULL || h->h_addr_list[0] == NULL) {
-                u->err = "host not found";
-                return NGX_ERROR;
-            }
-
-            sin->sin_addr.s_addr = *(in_addr_t *) (h->h_addr_list[0]);
+        if (h == NULL || h->h_addr_list[0] == NULL) {
+            u->err = "host not found";
+            return NGX_ERROR;
         }
 
-        if (sin->sin_addr.s_addr == INADDR_ANY) {
-            u->wildcard = 1;
-        }
+        sin->sin_addr.s_addr = *(in_addr_t *) (h->h_addr_list[0]);
+    }
 
-    } else {
-        sin->sin_addr.s_addr = INADDR_ANY;
+    if (sin->sin_addr.s_addr == INADDR_ANY) {
         u->wildcard = 1;
     }
 
-    if (u->no_port) {
-        u->port = u->default_port;
-        sin->sin_port = htons(u->default_port);
-    }
-
     if (u->listen) {
         return NGX_OK;
     }


From ru at nginx.com  Mon Dec 17 09:44:46 2012
From: ru at nginx.com (ru at nginx.com)
Date: Mon, 17 Dec 2012 09:44:46 +0000
Subject: [nginx] svn commit:  r4971 - trunk/src/core
Message-ID: <20121217094446.5D3833F9C4E@mail.nginx.com>

Author: ru
Date: 2012-12-17 09:44:46 +0000 (Mon, 17 Dec 2012)
New Revision: 4971
URL: http://trac.nginx.org/nginx/changeset/4971/nginx

Log:
Fixed URL parsing code.

The URL parsing code is not expected to initialize port from default port
when in "no_resolve" mode.  This got broken in r4671 for the case of IPv6
literals.


Modified:
   trunk/src/core/ngx_inet.c

Modified: trunk/src/core/ngx_inet.c
===================================================================
--- trunk/src/core/ngx_inet.c	2012-12-17 09:31:53 UTC (rev 4970)
+++ trunk/src/core/ngx_inet.c	2012-12-17 09:44:46 UTC (rev 4971)
@@ -828,6 +828,11 @@
 
         } else {
             u->no_port = 1;
+
+            if (!u->no_resolve) {
+                u->port = u->default_port;
+                sin6->sin6_port = htons(u->default_port);
+            }
         }
     }
 
@@ -850,11 +855,6 @@
         u->wildcard = 1;
     }
 
-    if (u->no_port) {
-        u->port = u->default_port;
-        sin6->sin6_port = htons(u->default_port);
-    }
-
     u->family = AF_INET6;
     u->naddrs = 1;
 


From ru at nginx.com  Mon Dec 17 12:08:54 2012
From: ru at nginx.com (ru at nginx.com)
Date: Mon, 17 Dec 2012 12:08:54 +0000
Subject: [nginx] svn commit: r4972 - in trunk: auto src/core src/mail
 src/os/win32
Message-ID: <20121217120854.5EF443FA130@mail.nginx.com>

Author: ru
Date: 2012-12-17 12:08:53 +0000 (Mon, 17 Dec 2012)
New Revision: 4972
URL: http://trac.nginx.org/nginx/changeset/4972/nginx

Log:
Implemented IPv6 support for URLs specified using domain names.

This includes "debug_connection", upstreams, "proxy_pass", etc.
(ticket #92)

To preserve compatibility, "listen" specified with a domain name
selects the first IPv4 address, if available.  If not available,
the first IPv6 address will be used (ticket #186).


Modified:
   trunk/auto/unix
   trunk/src/core/ngx_inet.c
   trunk/src/core/ngx_inet.h
   trunk/src/mail/ngx_mail_auth_http_module.c
   trunk/src/os/win32/ngx_win32_config.h

Modified: trunk/auto/unix
===================================================================
--- trunk/auto/unix	2012-12-17 09:44:46 UTC (rev 4971)
+++ trunk/auto/unix	2012-12-17 12:08:53 UTC (rev 4972)
@@ -778,3 +778,17 @@
                   openat(AT_FDCWD, \".\", O_RDONLY|O_NOFOLLOW);
                   fstatat(AT_FDCWD, \".\", &sb, AT_SYMLINK_NOFOLLOW);"
 . auto/feature
+
+
+ngx_feature="getaddrinfo()"
+ngx_feature_name="NGX_HAVE_GETADDRINFO"
+ngx_feature_run=no
+ngx_feature_incs="#include <sys/types.h>
+                  #include <sys/socket.h>
+                  #include <netdb.h>"
+ngx_feature_path=
+ngx_feature_libs=
+ngx_feature_test='struct addrinfo *res;
+                  if (getaddrinfo("localhost", NULL, NULL, &res) != 0) return 1;
+                  freeaddrinfo(res)'
+. auto/feature

Modified: trunk/src/core/ngx_inet.c
===================================================================
--- trunk/src/core/ngx_inet.c	2012-12-17 09:44:46 UTC (rev 4971)
+++ trunk/src/core/ngx_inet.c	2012-12-17 12:08:53 UTC (rev 4972)
@@ -611,11 +611,13 @@
 static ngx_int_t
 ngx_parse_inet_url(ngx_pool_t *pool, ngx_url_t *u)
 {
-    u_char              *p, *host, *port, *last, *uri, *args;
-    size_t               len;
-    ngx_int_t            n;
-    struct hostent      *h;
-    struct sockaddr_in  *sin;
+    u_char               *p, *host, *port, *last, *uri, *args;
+    size_t                len;
+    ngx_int_t             n;
+    struct sockaddr_in   *sin;
+#if (NGX_HAVE_INET6)
+    struct sockaddr_in6  *sin6;
+#endif
 
     u->socklen = sizeof(struct sockaddr_in);
     sin = (struct sockaddr_in *) &u->sockaddr;
@@ -728,41 +730,79 @@
         return NGX_OK;
     }
 
-    if (u->no_resolve) {
-        return NGX_OK;
-    }
-
     sin->sin_addr.s_addr = ngx_inet_addr(host, len);
 
-    if (sin->sin_addr.s_addr == INADDR_NONE) {
-        p = ngx_alloc(++len, pool->log);
-        if (p == NULL) {
+    if (sin->sin_addr.s_addr != INADDR_NONE) {
+
+        if (sin->sin_addr.s_addr == INADDR_ANY) {
+            u->wildcard = 1;
+        }
+
+        u->naddrs = 1;
+
+        u->addrs = ngx_pcalloc(pool, sizeof(ngx_addr_t));
+        if (u->addrs == NULL) {
             return NGX_ERROR;
         }
 
-        (void) ngx_cpystrn(p, host, len);
+        sin = ngx_pcalloc(pool, sizeof(struct sockaddr_in));
+        if (sin == NULL) {
+            return NGX_ERROR;
+        }
 
-        h = gethostbyname((const char *) p);
+        ngx_memcpy(sin, u->sockaddr, sizeof(struct sockaddr_in));
 
-        ngx_free(p);
+        u->addrs[0].sockaddr = (struct sockaddr *) sin;
+        u->addrs[0].socklen = sizeof(struct sockaddr_in);
 
-        if (h == NULL || h->h_addr_list[0] == NULL) {
-            u->err = "host not found";
+        p = ngx_pnalloc(pool, u->host.len + sizeof(":65535") - 1);
+        if (p == NULL) {
             return NGX_ERROR;
         }
 
-        sin->sin_addr.s_addr = *(in_addr_t *) (h->h_addr_list[0]);
-    }
+        u->addrs[0].name.len = ngx_sprintf(p, "%V:%d",
+                                           &u->host, u->port) - p;
+        u->addrs[0].name.data = p;
 
-    if (sin->sin_addr.s_addr == INADDR_ANY) {
-        u->wildcard = 1;
+        return NGX_OK;
     }
 
-    if (u->listen) {
+    if (u->no_resolve) {
         return NGX_OK;
     }
 
-    return ngx_inet_resolve_host(pool, u);
+    if (ngx_inet_resolve_host(pool, u) != NGX_OK) {
+        return NGX_ERROR;
+    }
+
+    u->family = u->addrs[0].sockaddr->sa_family;
+    u->socklen = u->addrs[0].socklen;
+    ngx_memcpy(u->sockaddr, u->addrs[0].sockaddr, u->addrs[0].socklen);
+
+    switch (u->family) {
+
+#if (NGX_HAVE_INET6)
+    case AF_INET6:
+        sin6 = (struct sockaddr_in6 *) &u->sockaddr;
+
+        if (IN6_IS_ADDR_UNSPECIFIED(&sin6->sin6_addr)) {
+            u->wildcard = 1;
+        }
+
+        break;
+#endif
+
+    default: /* AF_INET */
+        sin = (struct sockaddr_in *) &u->sockaddr;
+
+        if (sin->sin_addr.s_addr == INADDR_ANY) {
+            u->wildcard = 1;
+        }
+
+        break;
+    }
+
+    return NGX_OK;
 }
 
 
@@ -894,9 +934,153 @@
 }
 
 
+#if (NGX_HAVE_GETADDRINFO && NGX_HAVE_INET6)
+
 ngx_int_t
 ngx_inet_resolve_host(ngx_pool_t *pool, ngx_url_t *u)
 {
+    u_char               *p, *host;
+    size_t                len;
+    in_port_t             port;
+    ngx_uint_t            i;
+    struct addrinfo       hints, *res, *rp;
+    struct sockaddr_in   *sin;
+    struct sockaddr_in6  *sin6;
+
+    port = htons(u->port);
+
+    host = ngx_alloc(u->host.len + 1, pool->log);
+    if (host == NULL) {
+        return NGX_ERROR;
+    }
+
+    (void) ngx_cpystrn(host, u->host.data, u->host.len + 1);
+
+    ngx_memzero(&hints, sizeof(struct addrinfo));
+    hints.ai_family = AF_UNSPEC;
+    hints.ai_socktype = SOCK_STREAM;
+
+    if (getaddrinfo((char *) host, NULL, &hints, &res) != 0) {
+        u->err = "host not found";
+        ngx_free(host);
+        return NGX_ERROR;
+    }
+
+    ngx_free(host);
+
+    for (i = 0, rp = res; rp != NULL; rp = rp->ai_next) {
+
+        switch (rp->ai_family) {
+
+        case AF_INET:
+        case AF_INET6:
+            break;
+
+        default:
+            continue;
+        }
+
+        i++;
+    }
+
+    if (i == 0) {
+        u->err = "host not found";
+        goto failed;
+    }
+
+    /* MP: ngx_shared_palloc() */
+
+    u->addrs = ngx_pcalloc(pool, i * sizeof(ngx_addr_t));
+    if (u->addrs == NULL) {
+        goto failed;
+    }
+
+    u->naddrs = i;
+
+    i = 0;
+
+    /* AF_INET addresses first */
+
+    for (rp = res; rp != NULL; rp = rp->ai_next) {
+
+        if (rp->ai_family != AF_INET) {
+            continue;
+        }
+
+        sin = ngx_pcalloc(pool, rp->ai_addrlen);
+        if (sin == NULL) {
+            goto failed;
+        }
+
+        ngx_memcpy(sin, rp->ai_addr, rp->ai_addrlen);
+
+        sin->sin_port = port;
+
+        u->addrs[i].sockaddr = (struct sockaddr *) sin;
+        u->addrs[i].socklen = rp->ai_addrlen;
+
+        len = NGX_INET_ADDRSTRLEN + sizeof(":65535") - 1;
+
+        p = ngx_pnalloc(pool, len);
+        if (p == NULL) {
+            goto failed;
+        }
+
+        len = ngx_sock_ntop((struct sockaddr *) sin, p, len, 1);
+
+        u->addrs[i].name.len = len;
+        u->addrs[i].name.data = p;
+
+        i++;
+    }
+
+    for (rp = res; rp != NULL; rp = rp->ai_next) {
+
+        if (rp->ai_family != AF_INET6) {
+            continue;
+        }
+
+        sin6 = ngx_pcalloc(pool, rp->ai_addrlen);
+        if (sin6 == NULL) {
+            goto failed;
+        }
+
+        ngx_memcpy(sin6, rp->ai_addr, rp->ai_addrlen);
+
+        sin6->sin6_port = port;
+
+        u->addrs[i].sockaddr = (struct sockaddr *) sin6;
+        u->addrs[i].socklen = rp->ai_addrlen;
+
+        len = NGX_INET6_ADDRSTRLEN + sizeof("[]:65535") - 1;
+
+        p = ngx_pnalloc(pool, len);
+        if (p == NULL) {
+            goto failed;
+        }
+
+        len = ngx_sock_ntop((struct sockaddr *) sin6, p, len, 1);
+
+        u->addrs[i].name.len = len;
+        u->addrs[i].name.data = p;
+
+        i++;
+    }
+
+    freeaddrinfo(res);
+    return NGX_OK;
+
+failed:
+
+    freeaddrinfo(res);
+    return NGX_ERROR;
+}
+
+#else /* !NGX_HAVE_GETADDRINFO || !NGX_HAVE_INET6 */
+
+ngx_int_t
+ngx_inet_resolve_host(ngx_pool_t *pool, ngx_url_t *u)
+{
     u_char              *p, *host;
     size_t               len;
     in_port_t            port;
@@ -928,13 +1112,8 @@
             return NGX_ERROR;
         }
 
-        if (u->one_addr == 0) {
-            for (i = 0; h->h_addr_list[i] != NULL; i++) { /* void */ }
+        for (i = 0; h->h_addr_list[i] != NULL; i++) { /* void */ }
 
-        } else {
-            i = 1;
-        }
-
         /* MP: ngx_shared_palloc() */
 
         u->addrs = ngx_pcalloc(pool, i * sizeof(ngx_addr_t));
@@ -1006,3 +1185,5 @@
 
     return NGX_OK;
 }
+
+#endif /* NGX_HAVE_GETADDRINFO && NGX_HAVE_INET6 */

Modified: trunk/src/core/ngx_inet.h
===================================================================
--- trunk/src/core/ngx_inet.h	2012-12-17 09:44:46 UTC (rev 4971)
+++ trunk/src/core/ngx_inet.h	2012-12-17 12:08:53 UTC (rev 4972)
@@ -87,7 +87,7 @@
     unsigned                  listen:1;
     unsigned                  uri_part:1;
     unsigned                  no_resolve:1;
-    unsigned                  one_addr:1;
+    unsigned                  one_addr:1;  /* compatibility */
 
     unsigned                  no_port:1;
     unsigned                  wildcard:1;

Modified: trunk/src/mail/ngx_mail_auth_http_module.c
===================================================================
--- trunk/src/mail/ngx_mail_auth_http_module.c	2012-12-17 09:44:46 UTC (rev 4971)
+++ trunk/src/mail/ngx_mail_auth_http_module.c	2012-12-17 12:08:53 UTC (rev 4972)
@@ -1388,7 +1388,6 @@
     u.url = value[1];
     u.default_port = 80;
     u.uri_part = 1;
-    u.one_addr = 1;
 
     if (ngx_strncmp(u.url.data, "http://", 7) == 0) {
         u.url.len -= 7;

Modified: trunk/src/os/win32/ngx_win32_config.h
===================================================================
--- trunk/src/os/win32/ngx_win32_config.h	2012-12-17 09:44:46 UTC (rev 4971)
+++ trunk/src/os/win32/ngx_win32_config.h	2012-12-17 12:08:53 UTC (rev 4972)
@@ -182,6 +182,7 @@
 #define NGX_HAVE_SO_SNDLOWAT         0
 #endif
 
+#define NGX_HAVE_GETADDRINFO         1
 
 #define ngx_random               rand
 #define ngx_debug_init()


From ru at nginx.com  Mon Dec 17 19:03:33 2012
From: ru at nginx.com (ru at nginx.com)
Date: Mon, 17 Dec 2012 19:03:33 +0000
Subject: [nginx] svn commit: r4973 - in trunk/src/http: . modules modules/perl
Message-ID: <20121217190334.073163F9F53@mail.nginx.com>

Author: ru
Date: 2012-12-17 19:03:33 +0000 (Mon, 17 Dec 2012)
New Revision: 4973
URL: http://trac.nginx.org/nginx/changeset/4973/nginx

Log:
Added checks that disallow adding a variable with an empty name.
Added variable name syntax checks to "geo" and "map" directives.


Modified:
   trunk/src/http/modules/ngx_http_geo_module.c
   trunk/src/http/modules/ngx_http_limit_conn_module.c
   trunk/src/http/modules/ngx_http_map_module.c
   trunk/src/http/modules/ngx_http_rewrite_module.c
   trunk/src/http/modules/ngx_http_split_clients_module.c
   trunk/src/http/modules/perl/ngx_http_perl_module.c
   trunk/src/http/ngx_http_variables.c

Modified: trunk/src/http/modules/ngx_http_geo_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_geo_module.c	2012-12-17 12:08:53 UTC (rev 4972)
+++ trunk/src/http/modules/ngx_http_geo_module.c	2012-12-17 19:03:33 UTC (rev 4973)
@@ -325,7 +325,7 @@
 
     name = value[1];
 
-    if (name.len < 2 || name.data[0] != '$') {
+    if (name.data[0] != '$') {
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "invalid variable name \"%V\"", &name);
         return NGX_CONF_ERROR;
@@ -342,6 +342,13 @@
         }
 
         name = value[2];
+
+        if (name.data[0] != '$') {
+            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                               "invalid variable name \"%V\"", &name);
+            return NGX_CONF_ERROR;
+        }
+
         name.len--;
         name.data++;
 

Modified: trunk/src/http/modules/ngx_http_limit_conn_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_limit_conn_module.c	2012-12-17 12:08:53 UTC (rev 4972)
+++ trunk/src/http/modules/ngx_http_limit_conn_module.c	2012-12-17 19:03:33 UTC (rev 4973)
@@ -540,7 +540,7 @@
             continue;
         }
 
-        if (value[i].len > 1 && value[i].data[0] == '$') {
+        if (value[i].data[0] == '$') {
 
             value[i].len--;
             value[i].data++;
@@ -613,7 +613,7 @@
 
     value = cf->args->elts;
 
-    if (value[2].len < 2 || value[2].data[0] != '$') {
+    if (value[2].data[0] != '$') {
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "invalid variable name \"%V\"", &value[2]);
         return NGX_CONF_ERROR;

Modified: trunk/src/http/modules/ngx_http_map_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_map_module.c	2012-12-17 12:08:53 UTC (rev 4972)
+++ trunk/src/http/modules/ngx_http_map_module.c	2012-12-17 19:03:33 UTC (rev 4973)
@@ -209,6 +209,13 @@
     }
 
     name = value[2];
+
+    if (name.data[0] != '$') {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "invalid variable name \"%V\"", &name);
+        return NGX_CONF_ERROR;
+    }
+
     name.len--;
     name.data++;
 

Modified: trunk/src/http/modules/ngx_http_rewrite_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_rewrite_module.c	2012-12-17 12:08:53 UTC (rev 4972)
+++ trunk/src/http/modules/ngx_http_rewrite_module.c	2012-12-17 19:03:33 UTC (rev 4973)
@@ -908,7 +908,7 @@
 
     value = cf->args->elts;
 
-    if (value[1].len < 2 || value[1].data[0] != '$') {
+    if (value[1].data[0] != '$') {
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "invalid variable name \"%V\"", &value[1]);
         return NGX_CONF_ERROR;

Modified: trunk/src/http/modules/ngx_http_split_clients_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_split_clients_module.c	2012-12-17 12:08:53 UTC (rev 4972)
+++ trunk/src/http/modules/ngx_http_split_clients_module.c	2012-12-17 19:03:33 UTC (rev 4973)
@@ -139,7 +139,7 @@
 
     name = value[2];
 
-    if (name.len < 2 || name.data[0] != '$') {
+    if (name.data[0] != '$') {
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "invalid variable name \"%V\"", &name);
         return NGX_CONF_ERROR;

Modified: trunk/src/http/modules/perl/ngx_http_perl_module.c
===================================================================
--- trunk/src/http/modules/perl/ngx_http_perl_module.c	2012-12-17 12:08:53 UTC (rev 4972)
+++ trunk/src/http/modules/perl/ngx_http_perl_module.c	2012-12-17 19:03:33 UTC (rev 4973)
@@ -968,7 +968,7 @@
 
     value = cf->args->elts;
 
-    if (value[1].len < 2 || value[1].data[0] != '$') {
+    if (value[1].data[0] != '$') {
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "invalid variable name \"%V\"", &value[1]);
         return NGX_CONF_ERROR;

Modified: trunk/src/http/ngx_http_variables.c
===================================================================
--- trunk/src/http/ngx_http_variables.c	2012-12-17 12:08:53 UTC (rev 4972)
+++ trunk/src/http/ngx_http_variables.c	2012-12-17 19:03:33 UTC (rev 4973)
@@ -330,6 +330,12 @@
     ngx_http_variable_t        *v;
     ngx_http_core_main_conf_t  *cmcf;
 
+    if (name->len == 0) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "invalid variable name \"$\"");
+        return NULL;
+    }
+
     cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module);
 
     key = cmcf->variables_keys->keys.elts;
@@ -393,6 +399,12 @@
     ngx_http_variable_t        *v;
     ngx_http_core_main_conf_t  *cmcf;
 
+    if (name->len == 0) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "invalid variable name \"$\"");
+        return NGX_ERROR;
+    }
+
     cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module);
 
     v = cmcf->variables.elts;


From mdounin at mdounin.ru  Tue Dec 18 10:53:41 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 18 Dec 2012 14:53:41 +0400
Subject: image_filter enhancement
In-Reply-To: <CANWdNRBW-uWR66FLqiEGu03n5wzLFfvQA+iXLxxVop-0J5bSbw@mail.gmail.com>
References: <CANWdNRDYuMDZOpcEwtsryfUpTBCsdB6VsSJfF+qUJV34y5j08g@mail.gmail.com>
 <20121114234732.GC40452@mdounin.ru>
 <CANWdNRB=bRCEjEBM+ksak8c6pEZw+VC1XfKBrPLEJ5zXv+B_=g@mail.gmail.com>
 <20121127160328.GZ40452@mdounin.ru>
 <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>
 <CANWdNRBXuGyFdo1NGUeHba2QnSYokMJpD61tW-VNcOo=TV-9gg@mail.gmail.com>
 <20121204130841.GI40452@mdounin.ru>
 <CANWdNRATaVeEC7W0UVos9AdDM4Xq5BNRQUDea+UZkCuZzGL8gA@mail.gmail.com>
 <20121205145244.GO40452@mdounin.ru>
 <CANWdNRBW-uWR66FLqiEGu03n5wzLFfvQA+iXLxxVop-0J5bSbw@mail.gmail.com>
Message-ID: <20121218105341.GK40452@mdounin.ru>

Hello!

On Sun, Dec 16, 2012 at 07:55:39PM +0400, ivan babrou wrote:

> On 5 December 2012 18:52, Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> > Hello!
> >
> > On Tue, Dec 04, 2012 at 06:58:17PM +0400, ivan babrou wrote:
> >
> > > >
> > > > Additionally, looking at the code I tend to think it's incorrect.
> > > > That is, it's pattern you've followed is incorrect, not your code.
> > > > E.g. the following config will result in $arg_q incorrectly used
> > > > for quality in /image/, instead of "80" explicitly set:
> > > >     image_filter_jpeg_quality $arg_q;
> > > >     location /image/ {
> > > >         image_filter crop $arg_w $arg_h;
> > > >         image_filter_jpeg_quality 80;
> > > >     }
> > > > This needs fixing.
> > >
> > >
> > > As I see other config variables are buggy too. Should I fix them too?
> >
> > It would be fine to produce a patch which fixes existing config
> > variables, and the patch to add crop offsets on top of it.
> 
> 
> Can you give me some good examples to see "good practices" in nginx
> configuration handling?

I would recommend looking into standard ngx_conf_merge_*_value() 
macros from src/core/ngx_conf_file.h, and rolling your code based 
on them with needed modifications and/or code reuse.  Something 
like this should work:

    if (conf->jpeg_quality == NGX_CONF_UNSET_UINT) {
        ngx_conf_merge_uint_value(conf->jpeg_quality, prev->jpeg_quality, 75);

        if (conf->jqcv == NULL) {
            conf->jqcv = prev->jqcv;
        }
    }

-- 
Maxim Dounin
http://nginx.com/support.html


From gadh98 at gmail.com  Tue Dec 18 15:27:34 2012
From: gadh98 at gmail.com (Gad Hayisraeli)
Date: Tue, 18 Dec 2012 17:27:34 +0200
Subject: No subject
Message-ID: <CALAYyuQrk9H5j-CXTWJ8B1wiecpLb8W=A6tH8LtnqocQB-frjg@mail.gmail.com>

I'm developing a module that uses a handler module and header+body filter
module
at my handler phase (registered in the nginx rewrite phase) i stop the
request (return NGX_OK), then issue a subrequest to another server, get the
result back (hook function), then continue to the header+body filter,
change the headers + body and calls the next filters accordingly.
also, the subrequest runs thru a proxy , here's the relevant conf lines:
----------------------------------
(general:)
keepalive_timeout  65;
gzip  on;
proxy_http_version  1.1;
worker_connections  1024;

 location / { // this location is where the ab tries to get (see below the
ab test)
        include <my module conf>
        proxy_pass http://server
    }

        location /def1 { // this is the subrequest uri
        proxy_buffers           8 128k;
        proxy_buffer_size       128k;
        proxy_busy_buffers_size 128k;

        proxy_pass              http://server2/page.php // another nginx
runs fastcgi
        }
------------------------------------------

all works fine, untill i run an "ab" test against my nginx (10/12 workers)
that runs on a multi-core, heavy duty linux server, and here's my ab line:
ab -c 50 -n 20000 http://...myserver...
the actual rate is ~1000 requests per second, very high...

but almost exactly after 18500-18600 reuests (that ran smoothly without any
errors, debug level INFO with printouts of my own) - the nginx hangs - not
stuck - since i see in debug level "debug" its epoll loop runing ok, but
not recv nor proceesing any other requests although they continue to come
from my ab test

the stuck occurs right after the last (~18600) subrequest has been sent to
the server (def1)

any suggestions please ?

thanks

Gad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121218/e211ab1f/attachment.html>

From mdounin at mdounin.ru  Tue Dec 18 15:47:36 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 18 Dec 2012 19:47:36 +0400
Subject: your mail
In-Reply-To: <CALAYyuQrk9H5j-CXTWJ8B1wiecpLb8W=A6tH8LtnqocQB-frjg@mail.gmail.com>
References: <CALAYyuQrk9H5j-CXTWJ8B1wiecpLb8W=A6tH8LtnqocQB-frjg@mail.gmail.com>
Message-ID: <20121218154735.GD40452@mdounin.ru>

Hello!

On Tue, Dec 18, 2012 at 05:27:34PM +0200, Gad Hayisraeli wrote:

> I'm developing a module that uses a handler module and header+body filter
> module
> at my handler phase (registered in the nginx rewrite phase) i stop the
> request (return NGX_OK), then issue a subrequest to another server, get the

[...]

While this list is indeed better fits the question than nginx@ one 
you've posted your question earlier, it's might not be good idea 
to crosspost.  Repeating here the answer already sent on nginx@:

Most likely your module causes some resouce leak, which later 
results in a hang.  Hard to say more without seeing the code which 
causes the problem.

-- 
Maxim Dounin
http://nginx.com/support.html


From mdounin at mdounin.ru  Tue Dec 18 18:39:39 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Tue, 18 Dec 2012 18:39:39 +0000
Subject: [nginx] svn commit:  r4974 - trunk/src/http
Message-ID: <20121218183940.0F6CA3F9F2F@mail.nginx.com>

Author: mdounin
Date: 2012-12-18 18:39:39 +0000 (Tue, 18 Dec 2012)
New Revision: 4974
URL: http://trac.nginx.org/nginx/changeset/4974/nginx

Log:
Avoid sending "100 Continue" on 413 Request Entity Too Large.

Patch by Igor Sysoev.


Modified:
   trunk/src/http/ngx_http_core_module.c

Modified: trunk/src/http/ngx_http_core_module.c
===================================================================
--- trunk/src/http/ngx_http_core_module.c	2012-12-17 19:03:33 UTC (rev 4973)
+++ trunk/src/http/ngx_http_core_module.c	2012-12-18 18:39:39 UTC (rev 4974)
@@ -1001,6 +1001,7 @@
                       "client intended to send too large body: %O bytes",
                       r->headers_in.content_length_n);
 
+        r->expect_tested = 1;
         (void) ngx_http_discard_request_body(r);
         ngx_http_finalize_request(r, NGX_HTTP_REQUEST_ENTITY_TOO_LARGE);
         return NGX_OK;


From ru at nginx.com  Wed Dec 19 10:30:45 2012
From: ru at nginx.com (ru at nginx.com)
Date: Wed, 19 Dec 2012 10:30:45 +0000
Subject: [nginx] svn commit:  r4975 - trunk/src/http/modules
Message-ID: <20121219103045.BF1B33FA02C@mail.nginx.com>

Author: ru
Date: 2012-12-19 10:30:45 +0000 (Wed, 19 Dec 2012)
New Revision: 4975
URL: http://trac.nginx.org/nginx/changeset/4975/nginx

Log:
Slightly optimized code that handles special headers in "add_header".


Modified:
   trunk/src/http/modules/ngx_http_headers_filter_module.c

Modified: trunk/src/http/modules/ngx_http_headers_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_headers_filter_module.c	2012-12-18 18:39:39 UTC (rev 4974)
+++ trunk/src/http/modules/ngx_http_headers_filter_module.c	2012-12-19 10:30:45 UTC (rev 4975)
@@ -74,7 +74,9 @@
 
     { ngx_string("Cache-Control"), 0, ngx_http_add_cache_control },
 
-    { ngx_string("Last-Modified"), 0, ngx_http_set_last_modified },
+    { ngx_string("Last-Modified"),
+                 offsetof(ngx_http_headers_out_t, last_modified),
+                 ngx_http_set_last_modified },
 
     { ngx_string("ETag"),
                  offsetof(ngx_http_headers_out_t, etag),
@@ -372,28 +374,13 @@
 ngx_http_set_last_modified(ngx_http_request_t *r, ngx_http_header_val_t *hv,
     ngx_str_t *value)
 {
-    ngx_table_elt_t  *h;
-
-    ngx_http_clear_last_modified(r);
-
-    if (value->len == 0) {
-        return NGX_OK;
-    }
-
-    r->headers_out.last_modified_time = ngx_http_parse_time(value->data,
-                                                            value->len);
-
-    h = ngx_list_push(&r->headers_out.headers);
-    if (h == NULL) {
+    if (ngx_http_set_response_header(r, hv, value) != NGX_OK) {
         return NGX_ERROR;
     }
 
-    r->headers_out.last_modified = h;
+    r->headers_out.last_modified_time =
+        (value->len) ? ngx_http_parse_time(value->data, value->len) : -1;
 
-    h->hash = 1;
-    h->key = hv->key;
-    h->value = *value;
-
     return NGX_OK;
 }
 
@@ -406,22 +393,27 @@
 
     old = (ngx_table_elt_t **) ((char *) &r->headers_out + hv->offset);
 
-    if (*old) {
-        (*old)->hash = 0;
-        *old = NULL;
-    }
+    if (value->len == 0) {
+        if (*old) {
+            (*old)->hash = 0;
+            *old = NULL;
+        }
 
-    if (value->len == 0) {
         return NGX_OK;
     }
 
-    h = ngx_list_push(&r->headers_out.headers);
-    if (h == NULL) {
-        return NGX_ERROR;
+    if (*old) {
+        h = *old;
+
+    } else {
+        h = ngx_list_push(&r->headers_out.headers);
+        if (h == NULL) {
+            return NGX_ERROR;
+        }
+
+        *old = h;
     }
 
-    *old = h;
-
     h->hash = 1;
     h->key = hv->key;
     h->value = *value;


From ru at nginx.com  Wed Dec 19 10:33:56 2012
From: ru at nginx.com (ru at nginx.com)
Date: Wed, 19 Dec 2012 10:33:56 +0000
Subject: [nginx] svn commit:  r4976 - trunk/src/http/modules
Message-ID: <20121219103356.A17E03FAAB9@mail.nginx.com>

Author: ru
Date: 2012-12-19 10:33:56 +0000 (Wed, 19 Dec 2012)
New Revision: 4976
URL: http://trac.nginx.org/nginx/changeset/4976/nginx

Log:
Let "add_header" affect 201 responses (ticket #125).


Modified:
   trunk/src/http/modules/ngx_http_headers_filter_module.c

Modified: trunk/src/http/modules/ngx_http_headers_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_headers_filter_module.c	2012-12-19 10:30:45 UTC (rev 4975)
+++ trunk/src/http/modules/ngx_http_headers_filter_module.c	2012-12-19 10:33:56 UTC (rev 4976)
@@ -155,6 +155,7 @@
     if ((conf->expires == NGX_HTTP_EXPIRES_OFF && conf->headers == NULL)
         || r != r->main
         || (r->headers_out.status != NGX_HTTP_OK
+            && r->headers_out.status != NGX_HTTP_CREATED
             && r->headers_out.status != NGX_HTTP_NO_CONTENT
             && r->headers_out.status != NGX_HTTP_PARTIAL_CONTENT
             && r->headers_out.status != NGX_HTTP_MOVED_PERMANENTLY


From ru at nginx.com  Thu Dec 20 11:16:04 2012
From: ru at nginx.com (ru at nginx.com)
Date: Thu, 20 Dec 2012 11:16:04 +0000
Subject: [nginx] svn commit:  r4977 - trunk/src/core
Message-ID: <20121220111605.6296C3FA02C@mail.nginx.com>

Author: ru
Date: 2012-12-20 11:16:03 +0000 (Thu, 20 Dec 2012)
New Revision: 4977
URL: http://trac.nginx.org/nginx/changeset/4977/nginx

Log:
Fixed return type of internal function that allocates radix tree nodes.


Modified:
   trunk/src/core/ngx_radix_tree.c

Modified: trunk/src/core/ngx_radix_tree.c
===================================================================
--- trunk/src/core/ngx_radix_tree.c	2012-12-19 10:33:56 UTC (rev 4976)
+++ trunk/src/core/ngx_radix_tree.c	2012-12-20 11:16:03 UTC (rev 4977)
@@ -9,7 +9,7 @@
 #include <ngx_core.h>
 
 
-static void *ngx_radix_alloc(ngx_radix_tree_t *tree);
+static ngx_radix_node_t *ngx_radix_alloc(ngx_radix_tree_t *tree);
 
 
 ngx_radix_tree_t *
@@ -263,13 +263,13 @@
 }
 
 
-static void *
+static ngx_radix_node_t *
 ngx_radix_alloc(ngx_radix_tree_t *tree)
 {
-    char  *p;
+    ngx_radix_node_t  *p;
 
     if (tree->free) {
-        p = (char *) tree->free;
+        p = tree->free;
         tree->free = tree->free->right;
         return p;
     }
@@ -283,7 +283,7 @@
         tree->size = ngx_pagesize;
     }
 
-    p = tree->start;
+    p = (ngx_radix_node_t *) tree->start;
     tree->start += sizeof(ngx_radix_node_t);
     tree->size -= sizeof(ngx_radix_node_t);
 


From ru at nginx.com  Thu Dec 20 15:34:38 2012
From: ru at nginx.com (ru at nginx.com)
Date: Thu, 20 Dec 2012 15:34:38 +0000
Subject: [nginx] svn commit:  r4978 - trunk/src/http/modules/perl
Message-ID: <20121220153438.2A9FF3F9C13@mail.nginx.com>

Author: ru
Date: 2012-12-20 15:34:37 +0000 (Thu, 20 Dec 2012)
New Revision: 4978
URL: http://trac.nginx.org/nginx/changeset/4978/nginx

Log:
Brought the link to ngx_http_perl_module documentation up to date.


Modified:
   trunk/src/http/modules/perl/nginx.pm

Modified: trunk/src/http/modules/perl/nginx.pm
===================================================================
--- trunk/src/http/modules/perl/nginx.pm	2012-12-20 11:16:03 UTC (rev 4977)
+++ trunk/src/http/modules/perl/nginx.pm	2012-12-20 15:34:37 UTC (rev 4978)
@@ -123,7 +123,7 @@
 
 =head1 SEE ALSO
 
-http://sysoev.ru/nginx/docs/http/ngx_http_perl_module.html
+http://nginx.org/en/docs/http/ngx_http_perl_module.html
 
 =head1 AUTHOR
 


From ibobrik at gmail.com  Thu Dec 20 16:52:30 2012
From: ibobrik at gmail.com (ivan babrou)
Date: Thu, 20 Dec 2012 20:52:30 +0400
Subject: image_filter configuration inheritance
Message-ID: <CANWdNRA7bBbcJ_Tj53+4Q15JtLtyk4vc9mg+jMbCsVdwsZs0YA@mail.gmail.com>

For now nginx ignore explicit configuration of some image_filter directives.

For example, in this case nginx will use jpeg quality 10 instead of 80:

server {
    listen 8080;

    root /home/bobrik/dev;

    set $arg_q 10;

    image_filter_jpeg_quality $arg_q;

    location / {
        image_filter_jpeg_quality 80;
        image_filter crop 128 128;
    }
}

Maxim pointed me how to fix it, so here's the patch to fix it:

diff --git a/ngx_http_image_filter_module.c b/ngx_http_image_filter_module.c
index c853c33..3aee1a4 100644
--- a/ngx_http_image_filter_module.c
+++ b/ngx_http_image_filter_module.c
@@ -1201,21 +1201,28 @@ ngx_http_image_filter_merge_conf(ngx_conf_t *cf,
void *parent, void *child)
     }

     /* 75 is libjpeg default quality */
-    ngx_conf_merge_uint_value(conf->jpeg_quality, prev->jpeg_quality, 75);
+    if (conf->jpeg_quality == NGX_CONF_UNSET_UINT) {
+        ngx_conf_merge_uint_value(conf->jpeg_quality, prev->jpeg_quality,
75);

-    if (conf->jqcv == NULL) {
-        conf->jqcv = prev->jqcv;
+        if (conf->jqcv == NULL) {
+            conf->jqcv = prev->jqcv;
+        }
     }

-    ngx_conf_merge_uint_value(conf->sharpen, prev->sharpen, 0);
+    if (conf->sharpen == NGX_CONF_UNSET_UINT) {
+        ngx_conf_merge_uint_value(conf->sharpen, prev->sharpen, 0);

-    if (conf->shcv == NULL) {
-        conf->shcv = prev->shcv;
+        if (conf->shcv == NULL) {
+            conf->shcv = prev->shcv;
+        }
     }

-    ngx_conf_merge_uint_value(conf->angle, prev->angle, 0);
-    if (conf->acv == NULL) {
-        conf->acv = prev->acv;
+    if (conf->angle == NGX_CONF_UNSET_UINT) {
+        ngx_conf_merge_uint_value(conf->angle, prev->angle, 0);
+
+        if (conf->acv == NULL) {
+            conf->acv = prev->acv;
+        }
     }

     ngx_conf_merge_value(conf->transparency, prev->transparency, 1);

-- 
Regards, Ian Babrou
http://bobrik.name http://twitter.com/ibobrik skype:i.babrou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121220/7edbf74b/attachment.html>

From ibobrik at gmail.com  Thu Dec 20 17:01:38 2012
From: ibobrik at gmail.com (ivan babrou)
Date: Thu, 20 Dec 2012 21:01:38 +0400
Subject: image_filter enhancement
In-Reply-To: <20121218105341.GK40452@mdounin.ru>
References: <CANWdNRDYuMDZOpcEwtsryfUpTBCsdB6VsSJfF+qUJV34y5j08g@mail.gmail.com>
 <20121114234732.GC40452@mdounin.ru>
 <CANWdNRB=bRCEjEBM+ksak8c6pEZw+VC1XfKBrPLEJ5zXv+B_=g@mail.gmail.com>
 <20121127160328.GZ40452@mdounin.ru>
 <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>
 <CANWdNRBXuGyFdo1NGUeHba2QnSYokMJpD61tW-VNcOo=TV-9gg@mail.gmail.com>
 <20121204130841.GI40452@mdounin.ru>
 <CANWdNRATaVeEC7W0UVos9AdDM4Xq5BNRQUDea+UZkCuZzGL8gA@mail.gmail.com>
 <20121205145244.GO40452@mdounin.ru>
 <CANWdNRBW-uWR66FLqiEGu03n5wzLFfvQA+iXLxxVop-0J5bSbw@mail.gmail.com>
 <20121218105341.GK40452@mdounin.ru>
Message-ID: <CANWdNRDBkhQN_wAC9uCjL=SQ5GfyjnFJbJ43v9jpFXOUXp2NHA@mail.gmail.com>

I've send patch for configuration semantics in separate thread.

Here's lastest version of my patch that may be applied after. Is there
something else that I should fix?

diff --git a/ngx_http_image_filter_module.c b/ngx_http_image_filter_module.c
index 3aee1a4..9bb6e12 100644
--- a/ngx_http_image_filter_module.c
+++ b/ngx_http_image_filter_module.c
@@ -32,6 +32,11 @@
 #define NGX_HTTP_IMAGE_GIF       2
 #define NGX_HTTP_IMAGE_PNG       3

+#define NGX_HTTP_IMAGE_OFFSET_CENTER    0
+#define NGX_HTTP_IMAGE_OFFSET_LEFT      1
+#define NGX_HTTP_IMAGE_OFFSET_RIGHT     2
+#define NGX_HTTP_IMAGE_OFFSET_TOP       3
+#define NGX_HTTP_IMAGE_OFFSET_BOTTOM    4

 #define NGX_HTTP_IMAGE_BUFFERED  0x08

@@ -43,11 +48,15 @@ typedef struct {
     ngx_uint_t                   angle;
     ngx_uint_t                   jpeg_quality;
     ngx_uint_t                   sharpen;
+    ngx_uint_t                   crop_offset_x;
+    ngx_uint_t                   crop_offset_y;

     ngx_flag_t                   transparency;

     ngx_http_complex_value_t    *wcv;
     ngx_http_complex_value_t    *hcv;
+    ngx_http_complex_value_t    *oxcv;
+    ngx_http_complex_value_t    *oycv;
     ngx_http_complex_value_t    *acv;
     ngx_http_complex_value_t    *jqcv;
     ngx_http_complex_value_t    *shcv;
@@ -66,6 +75,8 @@ typedef struct {
     ngx_uint_t                   height;
     ngx_uint_t                   max_width;
     ngx_uint_t                   max_height;
+    ngx_uint_t                   crop_offset_x;
+    ngx_uint_t                   crop_offset_y;
     ngx_uint_t                   angle;

     ngx_uint_t                   phase;
@@ -110,6 +121,8 @@ static char
*ngx_http_image_filter_jpeg_quality(ngx_conf_t *cf,
     ngx_command_t *cmd, void *conf);
 static char *ngx_http_image_filter_sharpen(ngx_conf_t *cf, ngx_command_t
*cmd,
     void *conf);
+static char *ngx_http_image_filter_offset(ngx_conf_t *cf, ngx_command_t
*cmd,
+    void *conf);
 static ngx_int_t ngx_http_image_filter_init(ngx_conf_t *cf);


@@ -150,6 +163,13 @@ static ngx_command_t  ngx_http_image_filter_commands[]
= {
       offsetof(ngx_http_image_filter_conf_t, buffer_size),
       NULL },

+    { ngx_string("image_filter_crop_offset"),
+
 NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
+      ngx_http_image_filter_offset,
+      NGX_HTTP_LOC_CONF_OFFSET,
+      0,
+      NULL },
+
       ngx_null_command
 };

@@ -737,7 +757,8 @@ ngx_http_image_resize(ngx_http_request_t *r,
ngx_http_image_filter_ctx_t *ctx)
 {
     int                            sx, sy, dx, dy, ox, oy, ax, ay, size,
                                    colors, palette, transparent, sharpen,
-                                   red, green, blue, t;
+                                   red, green, blue, t,
+                                   offset_x, offset_y;
     u_char                        *out;
     ngx_buf_t                     *b;
     ngx_uint_t                     resize;
@@ -932,8 +953,24 @@ transparent:
                 return NULL;
             }

-            ox /= 2;
-            oy /= 2;
+            offset_x = ngx_http_image_filter_get_value(r, conf->oxcv,
+
conf->crop_offset_x);
+            offset_y = ngx_http_image_filter_get_value(r, conf->oycv,
+
conf->crop_offset_y);
+
+            if (offset_x == NGX_HTTP_IMAGE_OFFSET_LEFT) {
+                ox = 0;
+
+            } else if (offset_x == NGX_HTTP_IMAGE_OFFSET_CENTER) {
+                ox /= 2;
+            }
+
+            if (offset_y == NGX_HTTP_IMAGE_OFFSET_TOP) {
+                oy = 0;
+
+            } else if (offset_y == NGX_HTTP_IMAGE_OFFSET_CENTER) {
+                oy /= 2;
+            }

             ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                            "image crop: %d x %d @ %d x %d",
@@ -1151,7 +1188,24 @@ ngx_http_image_filter_value(ngx_str_t *value)

     n = ngx_atoi(value->data, value->len);

-    if (n > 0) {
+    if (n == NGX_ERROR) {
+        if (ngx_strncmp(value->data, "left", value->len) == 0) {
+            return NGX_HTTP_IMAGE_OFFSET_LEFT;
+
+        } else if (ngx_strncmp(value->data, "right", value->len) == 0) {
+            return NGX_HTTP_IMAGE_OFFSET_RIGHT;
+
+        } else if (ngx_strncmp(value->data, "top", value->len) == 0) {
+            return NGX_HTTP_IMAGE_OFFSET_TOP;
+
+        } else if (ngx_strncmp(value->data, "bottom", value->len) == 0) {
+            return NGX_HTTP_IMAGE_OFFSET_BOTTOM;
+
+        } else {
+            return NGX_HTTP_IMAGE_OFFSET_CENTER;
+        }
+
+    } else if (n > 0) {
         return (ngx_uint_t) n;
     }

@@ -1175,6 +1229,8 @@ ngx_http_image_filter_create_conf(ngx_conf_t *cf)
     conf->angle = NGX_CONF_UNSET_UINT;
     conf->transparency = NGX_CONF_UNSET;
     conf->buffer_size = NGX_CONF_UNSET_SIZE;
+    conf->crop_offset_x = NGX_CONF_UNSET_UINT;
+    conf->crop_offset_y = NGX_CONF_UNSET_UINT;

     return conf;
 }
@@ -1230,6 +1286,24 @@ ngx_http_image_filter_merge_conf(ngx_conf_t *cf,
void *parent, void *child)
     ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size,
                               1 * 1024 * 1024);

+    if (conf->crop_offset_x == NGX_CONF_UNSET_UINT) {
+        ngx_conf_merge_uint_value(conf->crop_offset_x, prev->crop_offset_x,
+                                  NGX_HTTP_IMAGE_OFFSET_CENTER);
+
+        if (conf->oxcv == NULL) {
+            conf->oxcv = prev->oxcv;
+        }
+    }
+
+    if (conf->crop_offset_y == NGX_CONF_UNSET_UINT) {
+        ngx_conf_merge_uint_value(conf->crop_offset_y, prev->crop_offset_y,
+                                  NGX_HTTP_IMAGE_OFFSET_CENTER);
+
+        if (conf->oycv == NULL) {
+            conf->oycv = prev->oycv;
+        }
+    }
+
     return NGX_CONF_OK;
 }

@@ -1481,6 +1555,68 @@ ngx_http_image_filter_sharpen(ngx_conf_t *cf,
ngx_command_t *cmd,
 }


+static char *
+ngx_http_image_filter_offset(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf)
+{
+    ngx_http_image_filter_conf_t *imcf = conf;
+
+    ngx_str_t                         *value;
+    ngx_http_complex_value_t           cv;
+    ngx_http_compile_complex_value_t   ccv;
+
+    value = cf->args->elts;
+
+    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
+
+    ccv.cf = cf;
+    ccv.value = &value[1];
+    ccv.complex_value = &cv;
+    ccv.zero = 1;
+
+    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
+        return NGX_CONF_ERROR;
+    }
+
+    if (cv.lengths == NULL) {
+        imcf->crop_offset_x = ngx_http_image_filter_value(&value[1]);
+
+    } else {
+        imcf->oxcv = ngx_palloc(cf->pool,
sizeof(ngx_http_complex_value_t));
+        if (imcf->oxcv == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        *imcf->oxcv = cv;
+    }
+
+    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
+
+    ccv.cf = cf;
+    ccv.value = &value[2];
+    ccv.complex_value = &cv;
+    ccv.zero = 1;
+
+    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
+        return NGX_CONF_ERROR;
+    }
+
+    if (cv.lengths == NULL) {
+        imcf->crop_offset_y = ngx_http_image_filter_value(&value[2]);
+
+    } else {
+        imcf->oycv = ngx_palloc(cf->pool,
sizeof(ngx_http_complex_value_t));
+        if (imcf->oycv == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        *imcf->oycv = cv;
+    }
+
+    return NGX_CONF_OK;
+}
+
+
 static ngx_int_t
 ngx_http_image_filter_init(ngx_conf_t *cf)
 {



On 18 December 2012 14:53, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Sun, Dec 16, 2012 at 07:55:39PM +0400, ivan babrou wrote:
>
> > On 5 December 2012 18:52, Maxim Dounin <mdounin at mdounin.ru> wrote:
> >
> > > Hello!
> > >
> > > On Tue, Dec 04, 2012 at 06:58:17PM +0400, ivan babrou wrote:
> > >
> > > > >
> > > > > Additionally, looking at the code I tend to think it's incorrect.
> > > > > That is, it's pattern you've followed is incorrect, not your code.
> > > > > E.g. the following config will result in $arg_q incorrectly used
> > > > > for quality in /image/, instead of "80" explicitly set:
> > > > >     image_filter_jpeg_quality $arg_q;
> > > > >     location /image/ {
> > > > >         image_filter crop $arg_w $arg_h;
> > > > >         image_filter_jpeg_quality 80;
> > > > >     }
> > > > > This needs fixing.
> > > >
> > > >
> > > > As I see other config variables are buggy too. Should I fix them too?
> > >
> > > It would be fine to produce a patch which fixes existing config
> > > variables, and the patch to add crop offsets on top of it.
> >
> >
> > Can you give me some good examples to see "good practices" in nginx
> > configuration handling?
>
> I would recommend looking into standard ngx_conf_merge_*_value()
> macros from src/core/ngx_conf_file.h, and rolling your code based
> on them with needed modifications and/or code reuse.  Something
> like this should work:
>
>     if (conf->jpeg_quality == NGX_CONF_UNSET_UINT) {
>         ngx_conf_merge_uint_value(conf->jpeg_quality, prev->jpeg_quality,
> 75);
>
>         if (conf->jqcv == NULL) {
>             conf->jqcv = prev->jqcv;
>         }
>     }
>
> --
> Maxim Dounin
> http://nginx.com/support.html
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>



-- 
Regards, Ian Babrou
http://bobrik.name http://twitter.com/ibobrik skype:i.babrou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121220/5e2832a0/attachment-0001.html>

From mdounin at mdounin.ru  Thu Dec 20 19:04:29 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 20 Dec 2012 19:04:29 +0000
Subject: [nginx] svn commit:  r4979 - trunk/src/http/modules
Message-ID: <20121220190429.A8BB93F9C0F@mail.nginx.com>

Author: mdounin
Date: 2012-12-20 19:04:28 +0000 (Thu, 20 Dec 2012)
New Revision: 4979
URL: http://trac.nginx.org/nginx/changeset/4979/nginx

Log:
Image filter: configuration inheritance fixes.

The image_filter_jpeg_quality, image_filter_sharpen and "image_filter rotate"
were inherited incorrectly if a directive with variables was defined, and
then redefined to a literal value, i.e. in configurations like

    image_filter_jpeg_quality $arg_q;

    location / {
        image_filter_jpeg_quality 50;
    }

Patch by Ian Babrou, with minor changes.


Modified:
   trunk/src/http/modules/ngx_http_image_filter_module.c

Modified: trunk/src/http/modules/ngx_http_image_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_image_filter_module.c	2012-12-20 15:34:37 UTC (rev 4978)
+++ trunk/src/http/modules/ngx_http_image_filter_module.c	2012-12-20 19:04:28 UTC (rev 4979)
@@ -1200,22 +1200,30 @@
         }
     }
 
-    /* 75 is libjpeg default quality */
-    ngx_conf_merge_uint_value(conf->jpeg_quality, prev->jpeg_quality, 75);
+    if (conf->jpeg_quality == NGX_CONF_UNSET_UINT) {
 
-    if (conf->jqcv == NULL) {
-        conf->jqcv = prev->jqcv;
+        /* 75 is libjpeg default quality */
+        ngx_conf_merge_uint_value(conf->jpeg_quality, prev->jpeg_quality, 75);
+
+        if (conf->jqcv == NULL) {
+            conf->jqcv = prev->jqcv;
+        }
     }
 
-    ngx_conf_merge_uint_value(conf->sharpen, prev->sharpen, 0);
+    if (conf->sharpen == NGX_CONF_UNSET_UINT) {
+        ngx_conf_merge_uint_value(conf->sharpen, prev->sharpen, 0);
 
-    if (conf->shcv == NULL) {
-        conf->shcv = prev->shcv;
+        if (conf->shcv == NULL) {
+            conf->shcv = prev->shcv;
+        }
     }
 
-    ngx_conf_merge_uint_value(conf->angle, prev->angle, 0);
-    if (conf->acv == NULL) {
-        conf->acv = prev->acv;
+    if (conf->angle == NGX_CONF_UNSET_UINT) {
+        ngx_conf_merge_uint_value(conf->angle, prev->angle, 0);
+
+        if (conf->acv == NULL) {
+            conf->acv = prev->acv;
+        }
     }
 
     ngx_conf_merge_value(conf->transparency, prev->transparency, 1);


From crazynorman at gmail.com  Thu Dec 20 19:51:44 2012
From: crazynorman at gmail.com (Daniel Church)
Date: Thu, 20 Dec 2012 14:51:44 -0500
Subject: memcache_assume_gzipped for 1.2.6
Message-ID: <CAEZsydtdjgqAifPTO9aqmH4XDirATXHJPdoocOfiGU2XwDjxYA@mail.gmail.com>

I'm running nginx on a cheap VPS, which has pretty horrible disk i/o.  As
such I keep most of my content in memcached so that nginx never has to hit
the disk, a good mix of cached dynamic and cached static content.  I wanted
to use GZIP compression without wasting CPU cycles gzipping memcached
content over and over again.

I created a flag "memcache_assume_gzipped" to the memcache module.  If you
turn it on for a location, it will append "Content-type: gzip" to the
headers of the response inside the memcache module.  This way, you can gzip
content (level 9), store it straight into memcache, and then nginx will
return it with the appropriate headers.  Example here:

http://alpha.criticalmeltdown.com/demo/2

Is this something anyone is interested in incorporating into the current
stable branch, or is it too much of a niche need?  If anyone wants it, I
can throw together a patch file.

Thanks,
Dan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121220/a5380a0a/attachment.html>

From mdounin at mdounin.ru  Thu Dec 20 20:10:43 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 21 Dec 2012 00:10:43 +0400
Subject: image_filter configuration inheritance
In-Reply-To: <CANWdNRA7bBbcJ_Tj53+4Q15JtLtyk4vc9mg+jMbCsVdwsZs0YA@mail.gmail.com>
References: <CANWdNRA7bBbcJ_Tj53+4Q15JtLtyk4vc9mg+jMbCsVdwsZs0YA@mail.gmail.com>
Message-ID: <20121220201043.GG40452@mdounin.ru>

Hello!

On Thu, Dec 20, 2012 at 08:52:30PM +0400, ivan babrou wrote:

> For now nginx ignore explicit configuration of some image_filter directives.

[...]

> -    ngx_conf_merge_uint_value(conf->angle, prev->angle, 0);
> -    if (conf->acv == NULL) {
> -        conf->acv = prev->acv;
> +    if (conf->angle == NGX_CONF_UNSET_UINT) {
> +        ngx_conf_merge_uint_value(conf->angle, prev->angle, 0);
> +
> +        if (conf->acv == NULL) {
> +            conf->acv = prev->acv;
> +        }
>      }
> 
>      ngx_conf_merge_value(conf->transparency, prev->transparency, 1);

Thanks, committed with minor changes.  Could you please provide 
proper Russian transcription of your name for CHANGES.ru?

While writing the commit message for the patch, I've realized that 
conf->angle/conf->acv inheritance looks wrong even with your 
patch.

Suggested additional patch below.  Is it looks ok?

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1356029439 -14400
# Node ID 7cb7b792d932c4e95b85db964993b9e883973608
# Parent  4c61fe6fa25dd38df68b7952281716e26a7b4618
Image filter: fixed image_filter rotate inheritance.

Configurations like

    location /i/ {
        image_filter resize 200 200;
        image_filter rotate 180;

        location /i/foo/ {
            image_filter resize 200 200;
        }
   }

resulted in rotation incorrectly applied in the location /i/foo, without
any way to clear it.  Fix is to handle conf->angle/conf->acv consistently
with other filter variables and do not try to inherit them if there are
transformations defined for current location.

diff --git a/src/http/modules/ngx_http_image_filter_module.c b/src/http/modules/ngx_http_image_filter_module.c
--- a/src/http/modules/ngx_http_image_filter_module.c
+++ b/src/http/modules/ngx_http_image_filter_module.c
@@ -1169,10 +1169,22 @@ ngx_http_image_filter_create_conf(ngx_co
         return NULL;
     }
 
+    /*
+     * set by ngx_pcalloc():
+     *
+     *     conf->width = 0;
+     *     conf->height = 0;
+     *     conf->angle = 0;
+     *     conf->wcv = NULL;
+     *     conf->hcv = NULL;
+     *     conf->acv = NULL;
+     *     conf->jqcv = NULL;
+     *     conf->shcv = NULL;
+     */
+
     conf->filter = NGX_CONF_UNSET_UINT;
     conf->jpeg_quality = NGX_CONF_UNSET_UINT;
     conf->sharpen = NGX_CONF_UNSET_UINT;
-    conf->angle = NGX_CONF_UNSET_UINT;
     conf->transparency = NGX_CONF_UNSET;
     conf->buffer_size = NGX_CONF_UNSET_SIZE;
 
@@ -1195,8 +1207,10 @@ ngx_http_image_filter_merge_conf(ngx_con
             conf->filter = prev->filter;
             conf->width = prev->width;
             conf->height = prev->height;
+            conf->angle = prev->angle;
             conf->wcv = prev->wcv;
             conf->hcv = prev->hcv;
+            conf->acv = prev->hcv;
         }
     }
 
@@ -1218,14 +1232,6 @@ ngx_http_image_filter_merge_conf(ngx_con
         }
     }
 
-    if (conf->angle == NGX_CONF_UNSET_UINT) {
-        ngx_conf_merge_uint_value(conf->angle, prev->angle, 0);
-
-        if (conf->acv == NULL) {
-            conf->acv = prev->acv;
-        }
-    }
-
     ngx_conf_merge_value(conf->transparency, prev->transparency, 1);
 
     ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size,

-- 
Maxim Dounin
http://nginx.com/support.html

p.s. It looks like your mail client corrupts patches, I had to fix 
line wrapping manually to apply your patch.  You may also want to 
avoid posting html emails here, thnx.


From ibobrik at gmail.com  Thu Dec 20 20:23:22 2012
From: ibobrik at gmail.com (ivan babrou)
Date: Fri, 21 Dec 2012 00:23:22 +0400
Subject: image_filter configuration inheritance
In-Reply-To: <20121220201043.GG40452@mdounin.ru>
References: <CANWdNRA7bBbcJ_Tj53+4Q15JtLtyk4vc9mg+jMbCsVdwsZs0YA@mail.gmail.com>
 <20121220201043.GG40452@mdounin.ru>
Message-ID: <CANWdNRC2uEvVqk2Z16P6J4sTMdWOnMi3jSzx20rhobnc4BKF+w@mail.gmail.com>

Looks like you're right about angle inheritance.

Russian description:
???????????: ????????? ?????? image_filter ??????????? ?????????????, ?????
????? ???????? ???????? ????? ?? ??????????? ??-?? ??????????? ???????? ??
??????????-???????????.


On 21 December 2012 00:10, Maxim Dounin <mdounin at mdounin.ru> wrote:

> proper Russian transcription of your name for CHANGES.ru?
>



-- 
Regards, Ian Babrou
http://bobrik.name http://twitter.com/ibobrik skype:i.babrou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121221/72399041/attachment.html>

From mdounin at mdounin.ru  Thu Dec 20 22:03:33 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 21 Dec 2012 02:03:33 +0400
Subject: memcache_assume_gzipped for 1.2.6
In-Reply-To: <CAEZsydtdjgqAifPTO9aqmH4XDirATXHJPdoocOfiGU2XwDjxYA@mail.gmail.com>
References: <CAEZsydtdjgqAifPTO9aqmH4XDirATXHJPdoocOfiGU2XwDjxYA@mail.gmail.com>
Message-ID: <20121220220333.GJ40452@mdounin.ru>

Hello!

On Thu, Dec 20, 2012 at 02:51:44PM -0500, Daniel Church wrote:

> I'm running nginx on a cheap VPS, which has pretty horrible disk i/o.  As
> such I keep most of my content in memcached so that nginx never has to hit
> the disk, a good mix of cached dynamic and cached static content.  I wanted
> to use GZIP compression without wasting CPU cycles gzipping memcached
> content over and over again.
> 
> I created a flag "memcache_assume_gzipped" to the memcache module.  If you
> turn it on for a location, it will append "Content-type: gzip" to the
> headers of the response inside the memcache module.  This way, you can gzip
> content (level 9), store it straight into memcache, and then nginx will
> return it with the appropriate headers.  Example here:
> 
> http://alpha.criticalmeltdown.com/demo/2
> 
> Is this something anyone is interested in incorporating into the current
> stable branch, or is it too much of a niche need?  If anyone wants it, I
> can throw together a patch file.

There is already memcached_gzip_flag, which allows to add 
"Content-Type: gzip" based on flags a memcached item.

http://nginx.org/r/memcached_gzip_flag

And even gunzip module which allows to gunzip such responses for 
clients which doesn't support gzip.

http://nginx.org/r/gunzip

Not sure why you want to introduce another directive for this.

-- 
Maxim Dounin
http://nginx.com/support.html


From crazynorman at gmail.com  Thu Dec 20 22:08:58 2012
From: crazynorman at gmail.com (Daniel Church)
Date: Thu, 20 Dec 2012 17:08:58 -0500
Subject: memcache_assume_gzipped for 1.2.6
In-Reply-To: <20121220220333.GJ40452@mdounin.ru>
References: <CAEZsydtdjgqAifPTO9aqmH4XDirATXHJPdoocOfiGU2XwDjxYA@mail.gmail.com>
 <20121220220333.GJ40452@mdounin.ru>
Message-ID: <CAEZsydtzLUiOvgj9edC7HUgT5_bg18r9T2Qz_zGJAGBq_rEZFg@mail.gmail.com>

Looks like that's only in the "development" version though and not the
"stable" version.  May have been smarter to back-port the feature rather
than adding a different one though.

On Thu, Dec 20, 2012 at 5:03 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Thu, Dec 20, 2012 at 02:51:44PM -0500, Daniel Church wrote:
>
> > I'm running nginx on a cheap VPS, which has pretty horrible disk i/o.  As
> > such I keep most of my content in memcached so that nginx never has to
> hit
> > the disk, a good mix of cached dynamic and cached static content.  I
> wanted
> > to use GZIP compression without wasting CPU cycles gzipping memcached
> > content over and over again.
> >
> > I created a flag "memcache_assume_gzipped" to the memcache module.  If
> you
> > turn it on for a location, it will append "Content-type: gzip" to the
> > headers of the response inside the memcache module.  This way, you can
> gzip
> > content (level 9), store it straight into memcache, and then nginx will
> > return it with the appropriate headers.  Example here:
> >
> > http://alpha.criticalmeltdown.com/demo/2
> >
> > Is this something anyone is interested in incorporating into the current
> > stable branch, or is it too much of a niche need?  If anyone wants it, I
> > can throw together a patch file.
>
> There is already memcached_gzip_flag, which allows to add
> "Content-Type: gzip" based on flags a memcached item.
>
> http://nginx.org/r/memcached_gzip_flag
>
> And even gunzip module which allows to gunzip such responses for
> clients which doesn't support gzip.
>
> http://nginx.org/r/gunzip
>
> Not sure why you want to introduce another directive for this.
>
> --
> Maxim Dounin
> http://nginx.com/support.html
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121220/fea66923/attachment.html>

From mdounin at mdounin.ru  Thu Dec 20 22:40:56 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 21 Dec 2012 02:40:56 +0400
Subject: image_filter configuration inheritance
In-Reply-To: <CANWdNRC2uEvVqk2Z16P6J4sTMdWOnMi3jSzx20rhobnc4BKF+w@mail.gmail.com>
References: <CANWdNRA7bBbcJ_Tj53+4Q15JtLtyk4vc9mg+jMbCsVdwsZs0YA@mail.gmail.com>
 <20121220201043.GG40452@mdounin.ru>
 <CANWdNRC2uEvVqk2Z16P6J4sTMdWOnMi3jSzx20rhobnc4BKF+w@mail.gmail.com>
Message-ID: <20121220224056.GM40452@mdounin.ru>

Hello!

On Fri, Dec 21, 2012 at 12:23:22AM +0400, ivan babrou wrote:

> Russian description:
> ???????????: ????????? ?????? image_filter ??????????? ?????????????, ?????

Thnx, but I asked about proper spelling of your name for "??????? ..." part.  :)

[...]

-- 
Maxim Dounin
http://nginx.com/support.html


From piotr at cloudflare.com  Fri Dec 21 04:13:04 2012
From: piotr at cloudflare.com (Piotr Sikora)
Date: Thu, 20 Dec 2012 20:13:04 -0800
Subject: [PATCH] Ignore unexpected 1xx status messages from the upstream.
Message-ID: <CADMhe6dUcH+wiFkYwFnsmSUMUrWPL1Nv+SkAQO+cXoCgdtX3cA@mail.gmail.com>

Ignore unexpected 1xx status messages from the upstream.

RFC2616 10.1 Informational 1xx says:
A client MUST be prepared to accept one or more 1xx status responses
prior to a regular response, even if the client does not expect a 100
(Continue) status message. Unexpected 1xx status responses MAY be
ignored by a user agent.

Signed-off-by: Piotr Sikora <piotr at cloudflare.com>
---
 src/http/ngx_http_parse.c | 71 +++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 68 insertions(+), 3 deletions(-)

diff --git a/src/http/ngx_http_parse.c b/src/http/ngx_http_parse.c
index a6ee74e..835edda 100644
--- a/src/http/ngx_http_parse.c
+++ b/src/http/ngx_http_parse.c
@@ -1432,7 +1432,11 @@
         sw_status,
         sw_space_after_status,
         sw_status_text,
-        sw_almost_done
+        sw_almost_done,
+        sw_ignore_space_after_status,
+        sw_ignore_start,
+        sw_ignore_text,
+        sw_ignore_almost_done
     } state;

     state = r->state;
@@ -1554,8 +1558,17 @@
             status->code = status->code * 10 + ch - '0';

             if (++status->count == 3) {
-                state = sw_space_after_status;
-                status->start = p - 2;
+                if (status->code == 100
+                    && r->http_major == 1 && r->http_minor == 1)
+                {
+                    state = sw_ignore_space_after_status;
+                    status->code = 0;
+                    status->count = 0;
+
+                } else {
+                    state = sw_space_after_status;
+                    status->start = p - 2;
+                }
             }

             break;
@@ -1600,6 +1613,58 @@
             default:
                 return NGX_ERROR;
             }
+            break;
+
+        /* space or end of line in ignored part */
+        case sw_ignore_space_after_status:
+            switch (ch) {
+            case ' ':
+                state = sw_ignore_text;
+                break;
+            case CR:
+                state = sw_ignore_text;
+                break;
+            case LF:
+                state = sw_ignore_start;
+                break;
+            default:
+                return NGX_ERROR;
+            }
+            break;
+
+        /* new line in ignored part */
+        case sw_ignore_start:
+            switch (ch) {
+            case CR:
+                state = sw_ignore_almost_done;
+                break;
+            case LF:
+                state = sw_start;
+                break;
+            default:
+                state = sw_ignore_text;
+                break;
+            }
+            break;
+
+        /* any text in ignored part until end of line */
+        case sw_ignore_text:
+            switch (ch) {
+            case LF:
+                state = sw_ignore_start;
+                break;
+            }
+            break;
+
+        /* end of ignored part */
+        case sw_ignore_almost_done:
+            switch (ch) {
+            case LF:
+                state = sw_start;
+                break;
+            default:
+                return NGX_ERROR;
+            }
         }
     }

-- 
1.8.0.2


From ibobrik at gmail.com  Fri Dec 21 05:05:42 2012
From: ibobrik at gmail.com (ivan babrou)
Date: Fri, 21 Dec 2012 09:05:42 +0400
Subject: image_filter configuration inheritance
In-Reply-To: <20121220224056.GM40452@mdounin.ru>
References: <CANWdNRA7bBbcJ_Tj53+4Q15JtLtyk4vc9mg+jMbCsVdwsZs0YA@mail.gmail.com>
 <20121220201043.GG40452@mdounin.ru>
 <CANWdNRC2uEvVqk2Z16P6J4sTMdWOnMi3jSzx20rhobnc4BKF+w@mail.gmail.com>
 <20121220224056.GM40452@mdounin.ru>
Message-ID: <CANWdNRD6TfVb-JqAav8xRxmm_0bW4nGE-0r2gEB9gpDGr57ULQ@mail.gmail.com>

Hah, by name is ???? ?????? in Russian, btw.


On 21 December 2012 02:40, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Fri, Dec 21, 2012 at 12:23:22AM +0400, ivan babrou wrote:
>
> > Russian description:
> > ???????????: ????????? ?????? image_filter ??????????? ?????????????,
> ?????
>
> Thnx, but I asked about proper spelling of your name for "??????? ..."
> part.  :)
>
> [...]
>
> --
> Maxim Dounin
> http://nginx.com/support.html
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>



-- 
Regards, Ian Babrou
http://bobrik.name http://twitter.com/ibobrik skype:i.babrou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121221/52b3d01b/attachment.html>

From ru at nginx.com  Fri Dec 21 08:44:40 2012
From: ru at nginx.com (ru at nginx.com)
Date: Fri, 21 Dec 2012 08:44:40 +0000
Subject: [nginx] svn commit:  r4980 - trunk/src/http/modules
Message-ID: <20121221084440.5F1AA3F9C14@mail.nginx.com>

Author: ru
Date: 2012-12-21 08:44:39 +0000 (Fri, 21 Dec 2012)
New Revision: 4980
URL: http://trac.nginx.org/nginx/changeset/4980/nginx

Log:
There's no need to normalize address returned by ngx_ptocidr().


Modified:
   trunk/src/http/modules/ngx_http_geo_module.c

Modified: trunk/src/http/modules/ngx_http_geo_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_geo_module.c	2012-12-20 19:04:28 UTC (rev 4979)
+++ trunk/src/http/modules/ngx_http_geo_module.c	2012-12-21 08:44:39 UTC (rev 4980)
@@ -998,7 +998,7 @@
         /* rc == NGX_BUSY */
 
         old = (ngx_http_variable_value_t *)
-              ngx_radix32tree_find(ctx->tree, cidr.u.in.addr & cidr.u.in.mask);
+              ngx_radix32tree_find(ctx->tree, cidr.u.in.addr);
 
         ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
                 "duplicate network \"%V\", value: \"%v\", old value: \"%v\"",


From ru at nginx.com  Fri Dec 21 08:46:53 2012
From: ru at nginx.com (ru at nginx.com)
Date: Fri, 21 Dec 2012 08:46:53 +0000
Subject: [nginx] svn commit:  r4981 - trunk/src/http/modules
Message-ID: <20121221084653.1E8D93FA013@mail.nginx.com>

Author: ru
Date: 2012-12-21 08:46:52 +0000 (Fri, 21 Dec 2012)
New Revision: 4981
URL: http://trac.nginx.org/nginx/changeset/4981/nginx

Log:
Geo: ensure that default entry is always present.

If 0.0.0.0/32 entry was present and there was no explicit "default",
we failed to add an empty string as a default value.


Modified:
   trunk/src/http/modules/ngx_http_geo_module.c

Modified: trunk/src/http/modules/ngx_http_geo_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_geo_module.c	2012-12-21 08:44:39 UTC (rev 4980)
+++ trunk/src/http/modules/ngx_http_geo_module.c	2012-12-21 08:46:52 UTC (rev 4981)
@@ -453,16 +453,14 @@
         ngx_destroy_pool(ctx.temp_pool);
         ngx_destroy_pool(pool);
 
-        if (ngx_radix32tree_find(ctx.tree, 0) != NGX_RADIX_NO_VALUE) {
-            return rv;
-        }
-
         if (ngx_radix32tree_insert(ctx.tree, 0, 0,
                                    (uintptr_t) &ngx_http_variable_null_value)
             == NGX_ERROR)
         {
             return NGX_CONF_ERROR;
         }
+
+        /* NGX_BUSY is okay (default was set explicitly) */
     }
 
     return rv;


From yuriymagnum at gmail.com  Fri Dec 21 10:08:33 2012
From: yuriymagnum at gmail.com (Yuri Bayteryakov)
Date: Fri, 21 Dec 2012 16:08:33 +0600
Subject: spdy+eval+memcached=signal 11
Message-ID: <CACRxKGVxzj6arUJv81HzYNL-6xCQPz+QRWxV349D9zw+8vfMTg@mail.gmail.com>

Hello.

Testing spdy + eval + memcached module for our deployment. Without spdy
this works fine. Worker processes die with following configuration:

# cat /etc/nginx/nginx.conf
user              nginx;
worker_processes  1;

worker_rlimit_nofile    2048;

error_log  /var/log/nginx/error.log debug;

pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

worker_rlimit_core  500M;
working_directory   /tmp/dumps/;

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    access_log  /var/log/nginx/access.log;

    sendfile        on;
    keepalive_timeout  70;

    ssl_session_cache    shared:SSL:10m;
    ssl_session_timeout  10m;

    server {
        listen  443 ssl spdy;
        ssl_certificate ssl/ca.crt;
        ssl_certificate_key     ssl/ca.key;

        location / {
    eval_escalate                       on;
    eval $auth_cookie_value {
        set                             $memcached_key $host-$cookie_abc;
        memcached_next_upstream         error timeout invalid_response
not_found;
        memcached_pass                  192.168.0.55:11211;
    }
        proxy_pass                      http://192.168.0.158:8081;
}
    }
}

Linux hostname 2.6.32-279.el6.x86_64 #1 SMP Thu Jun 21 15:00:18 EDT 2012
x86_64 x86_64 x86_64 GNU/Linux

error.log:
2012/12/21 15:53:12 [notice] 16252#0: start worker process 16273
2012/12/21 15:53:22 [info] 16273#0: *20 SSL NPN ADVERT while SSL
handshaking, client: 192.168.3.70, server:
2012/12/21 15:53:22 [notice] 16252#0: signal 17 (SIGCHLD) received
2012/12/21 15:53:22 [alert] 16252#0: worker process 16273 exited on signal
11 (core dumped)
2012/12/21 15:53:22 [notice] 16252#0: start worker process 16275

# /usr/sbin/nginx -V
nginx version: nginx/1.3.9
built by gcc 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC)
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx/ --sbin-path=/usr/sbin/nginx
--conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log
--http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid
--lock-path=/var/run/nginx.lock
--http-client-body-temp-path=/var/cache/nginx/client_temp
--http-proxy-temp-path=/var/cache/nginx/proxy_temp
--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp
--http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp
--http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx
--with-http_ssl_module --with-http_realip_module
--with-http_addition_module --with-http_sub_module
--with-http_gzip_static_module --with-http_random_index_module
--with-http_stub_status_module --with-openssl=/tmp/openssl101-1.0.1c
--with-cc-opt='-O0 -g -I/opt/openssl101/include'
--with-ld-opt=-L/usr/openssl101/lib
--add-module=/home/bld/rpmbuild/BUILD/nginx-1.3.9/nginx_eval_module-1.0.1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121221/e0ab08d8/attachment-0001.html>
-------------- next part --------------
# gdb /usr/sbin/nginx core.16273
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-56.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/sbin/nginx...Reading symbols from /usr/lib/debug/usr/sbin/nginx.debug...done.
done.
[New Thread 16273]
Missing separate debuginfo for 
Try: yum --disablerepo='*' --enablerepo='*-debug*' install /usr/lib/debug/.build-id/80/1b9608daa2cd5f7035ad415e9c7dd06ebdb0a2
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /lib64/libpcre.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libpcre.so.0
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libz.so.1
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libfreebl3.so...(no debugging symbols found)...done.
Loaded symbols for /lib64/libfreebl3.so
Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libnss_files.so.2
Core was generated by `nginx: worker process                   '.
Program terminated with signal 11, Segmentation fault.
#0  0x000000000043352c in ngx_epoll_add_event (ev=0xfdd1e0, event=4, flags=18446744071562067968) at src/event/modules/ngx_epoll_module.c:404
404             e = c->read;
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.80.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64 pcre-7.8-4.el6.x86_64 zlib-1.2.3-27.el6.x86_64
(gdb) bt
#0  0x000000000043352c in ngx_epoll_add_event (ev=0xfdd1e0, event=4, flags=18446744071562067968) at src/event/modules/ngx_epoll_module.c:404
#1  0x000000000046271d in ngx_http_upstream_init (r=0xfde6c8) at src/http/ngx_http_upstream.c:444
#2  0x00000000004b0a9b in ngx_http_memcached_handler (r=0xfde6c8) at src/http/modules/ngx_http_memcached_module.c:211
#3  0x0000000000441724 in ngx_http_core_content_phase (r=0xfde6c8, ph=0xf8e790) at src/http/ngx_http_core_module.c:1404
#4  0x0000000000440665 in ngx_http_core_run_phases (r=0xfde6c8) at src/http/ngx_http_core_module.c:885
#5  0x00000000004405dc in ngx_http_handler (r=0xfde6c8) at src/http/ngx_http_core_module.c:868
#6  0x000000000044e548 in ngx_http_run_posted_requests (c=0xf77220) at src/http/ngx_http_request.c:1954
#7  0x000000000048815e in ngx_http_spdy_run_request (r=0xfdd278) at src/http/ngx_http_spdy.c:2026
#8  0x0000000000487d22 in ngx_http_spdy_process_headers (sc=0xfdbef0, pos=0x7fff27f2f298, size=0) at src/http/ngx_http_spdy.c:1887
#9  0x0000000000485e66 in ngx_http_spdy_read_handler (rev=0xf98820) at src/http/ngx_http_spdy.c:734
#10 0x000000000048a2ec in ngx_http_spdy_keepalive_handler (rev=0xf98820) at src/http/ngx_http_spdy.c:3254
#11 0x0000000000433c1e in ngx_epoll_process_events (cycle=0xf6ef50, timer=179999, flags=1) at src/event/modules/ngx_epoll_module.c:683
#12 0x0000000000426d16 in ngx_process_events_and_timers (cycle=0xf6ef50) at src/event/ngx_event.c:247
#13 0x0000000000432032 in ngx_worker_process_cycle (cycle=0xf6ef50, data=0x0) at src/os/unix/ngx_process_cycle.c:807
#14 0x000000000042efa2 in ngx_spawn_process (cycle=0xf6ef50, proc=0x431ea5 <ngx_worker_process_cycle>, data=0x0, name=0x5e4f98 "worker process", 
    respawn=0) at src/os/unix/ngx_process.c:198
#15 0x0000000000431ab8 in ngx_reap_children (cycle=0xf6ef50) at src/os/unix/ngx_process_cycle.c:619
#16 0x0000000000430b65 in ngx_master_process_cycle (cycle=0xf6ef50) at src/os/unix/ngx_process_cycle.c:180
#17 0x0000000000404336 in main (argc=3, argv=0x7fff27f2f948) at src/core/nginx.c:412
(gdb) backtrace full
#0  0x000000000043352c in ngx_epoll_add_event (ev=0xfdd1e0, event=4, flags=18446744071562067968) at src/event/modules/ngx_epoll_module.c:404
        op = 32767
        events = 4
        prev = 0
        e = 0x358
        c = 0x0
        ee = {events = 16675544, data = {ptr = 0xfe71c000000000, fd = 0, u32 = 0, u64 = 71619713532166144}}
#1  0x000000000046271d in ngx_http_upstream_init (r=0xfde6c8) at src/http/ngx_http_upstream.c:444
        c = 0xf77220
#2  0x00000000004b0a9b in ngx_http_memcached_handler (r=0xfde6c8) at src/http/modules/ngx_http_memcached_module.c:211
        rc = 0
        u = 0xfe72d8
        ctx = 0xfe7630
        mlcf = 0xf8a250
#3  0x0000000000441724 in ngx_http_core_content_phase (r=0xfde6c8, ph=0xf8e790) at src/http/ngx_http_core_module.c:1404
        root = 16312136
        rc = 16639688
        path = {len = 140733863620832, data = 0x4406c1 "H\211E\370H\203}", <incomplete sequence \370>}
#4  0x0000000000440665 in ngx_http_core_run_phases (r=0xfde6c8) at src/http/ngx_http_core_module.c:885
        rc = -2
        ph = 0xf8e670
        cmcf = 0xf707a8
#5  0x00000000004405dc in ngx_http_handler (r=0xfde6c8) at src/http/ngx_http_core_module.c:868
        cmcf = 0xf707a8
#6  0x000000000044e548 in ngx_http_run_posted_requests (c=0xf77220) at src/http/ngx_http_request.c:1954
        r = 0xfde6c8
        ctx = 0xfdd780
        pr = 0xfdf128
#7  0x000000000048815e in ngx_http_spdy_run_request (r=0xfdd278) at src/http/ngx_http_spdy.c:2026
        i = 13
        part = 0xfdd2e8
        h = 0xfde280
        fc = 0xf77220
        hh = 0x0
        cmcf = 0xf707a8
#8  0x0000000000487d22 in ngx_http_spdy_process_headers (sc=0xfdbef0, pos=0x7fff27f2f298, size=0) at src/http/ngx_http_spdy.c:1887
        z = 0
---Type <return> to continue, or q <return> to quit---
        buf = 0xfdd798
        rc = -4
        last = 1
        h = 0xfde4c0
        c = 0x7ff2164b9190
        r = 0xfdd278
#9  0x0000000000485e66 in ngx_http_spdy_read_handler (rev=0xf98820) at src/http/ngx_http_spdy.c:734
        p = 0x7ff2163b825e ""
        end = 0x7ff2163b82d8 ""
        available = 1048568
        n = 694
        rc = 0
        rest = 0
        c = 0x7ff2164b9190
        smcf = 0xf710c0
        sc = 0xfdbef0
#10 0x000000000048a2ec in ngx_http_spdy_keepalive_handler (rev=0xf98820) at src/http/ngx_http_spdy.c:3254
        c = 0x7ff2164b9190
#11 0x0000000000433c1e in ngx_epoll_process_events (cycle=0xf6ef50, timer=179999, flags=1) at src/event/modules/ngx_epoll_module.c:683
        events = 1
        revents = 1
        instance = 0
        i = 0
        level = 9043008
        err = 0
        rev = 0xf98820
        wev = 0x428b7a
        queue = 0x2bf1f
        c = 0x7ff2164b9190
#12 0x0000000000426d16 in ngx_process_events_and_timers (cycle=0xf6ef50) at src/event/ngx_event.c:247
        flags = 1
        timer = 179999
        delta = 1356083602067
#13 0x0000000000432032 in ngx_worker_process_cycle (cycle=0xf6ef50, data=0x0) at src/os/unix/ngx_process_cycle.c:807
        worker = 0
        i = 140733863622976
        c = 0x0
---Type <return> to continue, or q <return> to quit---
#14 0x000000000042efa2 in ngx_spawn_process (cycle=0xf6ef50, proc=0x431ea5 <ngx_worker_process_cycle>, data=0x0, name=0x5e4f98 "worker process", 
    respawn=0) at src/os/unix/ngx_process.c:198
        on = 1
        pid = 0
        s = 0
#15 0x0000000000431ab8 in ngx_reap_children (cycle=0xf6ef50) at src/os/unix/ngx_process_cycle.c:619
        i = 0
        n = 1
        live = 0
        ch = {command = 2, pid = 16271, slot = 0, fd = -1}
        ccf = 0x50d43188
#16 0x0000000000430b65 in ngx_master_process_cycle (cycle=0xf6ef50) at src/os/unix/ngx_process_cycle.c:180
        title = 0xf8e895 "master process /usr/sbin/nginx -c /etc/nginx/nginx.conf"
        p = 0xf8e8cc ""
        size = 56
        i = 3
        n = 0
        sigio = 0
        set = {__val = {0 <repeats 16 times>}}
        itv = {it_interval = {tv_sec = 0, tv_usec = 1}, it_value = {tv_sec = 7, tv_usec = 18}}
        live = 1
        delay = 0
        ls = 0x6
        ccf = 0xf6ff50
#17 0x0000000000404336 in main (argc=3, argv=0x7fff27f2f948) at src/core/nginx.c:412
        i = 57
        log = 0x89da40
        cycle = 0xf6ef50
        init_cycle = {conf_ctx = 0x0, pool = 0xf6ea20, log = 0x89da40, new_log = {log_level = 0, file = 0x0, connection = 0, handler = 0, 
            data = 0x0, action = 0x0}, files = 0x0, free_connections = 0x0, free_connection_n = 0, reusable_connections_queue = {prev = 0x0, 
            next = 0x0}, listening = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, paths = {elts = 0x0, nelts = 0, size = 0, 
            nalloc = 0, pool = 0x0}, open_files = {last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, 
          shared_memory = {last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, connection_n = 0, files_n = 0, 
          connections = 0x0, read_events = 0x0, write_events = 0x0, old_cycle = 0x0, conf_file = {len = 21, data = 0x7fff27f2ff73 "ss"}, 
          conf_param = {len = 0, data = 0x0}, conf_prefix = {len = 11, data = 0x7fff27f2ff73 "ss"}, prefix = {len = 12, 
            data = 0x5e1f0c "/etc/nginx//"}, lock_file = {len = 0, data = 0x0}, hostname = {len = 0, data = 0x0}}
        ccf = 0xf6ff50
(gdb) 


Linux hostname 2.6.32-279.el6.x86_64 #1 SMP Thu Jun 21 15:00:18 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux


# cat /etc/nginx/nginx.conf 
user              nginx;
worker_processes  1;

worker_rlimit_nofile    2048;

error_log  /var/log/nginx/error.log debug;

pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

worker_rlimit_core  500M;
working_directory   /tmp/dumps/;

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    access_log  /var/log/nginx/access.log;

    sendfile        on;
    keepalive_timeout  70;

    ssl_session_cache    shared:SSL:10m;
    ssl_session_timeout  10m;

    server {
        listen  443 ssl spdy;
        ssl_certificate ssl/ca.crt;
        ssl_certificate_key     ssl/ca.key;

        location / {
    #error_page                          404 = /nginx/getRedirect;
    eval_escalate                       on;
    eval $auth_cookie_value {
        set                             $memcached_key $host-$cookie_abc;
        memcached_next_upstream         error timeout invalid_response not_found;
        memcached_pass                  192.168.0.55:11211;
    }
        proxy_pass                      http://192.168.0.158:8081;
}
    }
}



error.log:
2012/12/21 15:53:12 [notice] 16252#0: start worker process 16273
2012/12/21 15:53:22 [info] 16273#0: *20 SSL NPN ADVERT while SSL handshaking, client: 192.168.3.70, server: 
2012/12/21 15:53:22 [notice] 16252#0: signal 17 (SIGCHLD) received
2012/12/21 15:53:22 [alert] 16252#0: worker process 16273 exited on signal 11 (core dumped)
2012/12/21 15:53:22 [notice] 16252#0: start worker process 16275



# /usr/sbin/nginx -V
nginx version: nginx/1.3.9
built by gcc 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) 
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx/ --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_gzip_static_module --with-http_random_index_module --with-http_stub_status_module --with-openssl=/tmp/openssl101-1.0.1c --with-cc-opt='-O0 -g -I/opt/openssl101/include' --with-ld-opt=-L/usr/openssl101/lib --add-module=/home/bld/rpmbuild/BUILD/nginx-1.3.9/nginx_eval_module-1.0.1

From mdounin at mdounin.ru  Fri Dec 21 14:46:44 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 21 Dec 2012 18:46:44 +0400
Subject: [PATCH] Ignore unexpected 1xx status messages from the upstream.
In-Reply-To: <CADMhe6dUcH+wiFkYwFnsmSUMUrWPL1Nv+SkAQO+cXoCgdtX3cA@mail.gmail.com>
References: <CADMhe6dUcH+wiFkYwFnsmSUMUrWPL1Nv+SkAQO+cXoCgdtX3cA@mail.gmail.com>
Message-ID: <20121221144644.GZ40452@mdounin.ru>

Hello!

On Thu, Dec 20, 2012 at 08:13:04PM -0800, Piotr Sikora wrote:

> Ignore unexpected 1xx status messages from the upstream.
> 
> RFC2616 10.1 Informational 1xx says:
> A client MUST be prepared to accept one or more 1xx status responses
> prior to a regular response, even if the client does not expect a 100
> (Continue) status message. Unexpected 1xx status responses MAY be
> ignored by a user agent.
> 
> Signed-off-by: Piotr Sikora <piotr at cloudflare.com>
> ---
>  src/http/ngx_http_parse.c | 71 +++++++++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 68 insertions(+), 3 deletions(-)
> 
> diff --git a/src/http/ngx_http_parse.c b/src/http/ngx_http_parse.c
> index a6ee74e..835edda 100644
> --- a/src/http/ngx_http_parse.c
> +++ b/src/http/ngx_http_parse.c
> @@ -1432,7 +1432,11 @@
>          sw_status,
>          sw_space_after_status,
>          sw_status_text,
> -        sw_almost_done
> +        sw_almost_done,
> +        sw_ignore_space_after_status,
> +        sw_ignore_start,
> +        sw_ignore_text,
> +        sw_ignore_almost_done
>      } state;
> 
>      state = r->state;
> @@ -1554,8 +1558,17 @@
>              status->code = status->code * 10 + ch - '0';
> 
>              if (++status->count == 3) {
> -                state = sw_space_after_status;
> -                status->start = p - 2;
> +                if (status->code == 100
> +                    && r->http_major == 1 && r->http_minor == 1)
> +                {
> +                    state = sw_ignore_space_after_status;
> +                    status->code = 0;
> +                    status->count = 0;
> +
> +                } else {
> +                    state = sw_space_after_status;
> +                    status->start = p - 2;
> +                }

I don't like this to be done this way.  It's protocol module (== 
proxy) which should decide what to do based on a response code, 
not a parser.  And, BTW, another valid 1xx code which upstream 
servers may send unexpectedly seems to be 102 Processing.

Additionally, proper handling of 101 will be required for upcoming 
websockets support.

-- 
Maxim Dounin
http://nginx.com/support.html


From mdounin at mdounin.ru  Fri Dec 21 15:07:46 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Fri, 21 Dec 2012 15:07:46 +0000
Subject: [nginx] svn commit:  r4982 - trunk/src/http/modules
Message-ID: <20121221150746.18ED63FA9A6@mail.nginx.com>

Author: mdounin
Date: 2012-12-21 15:07:45 +0000 (Fri, 21 Dec 2012)
New Revision: 4982
URL: http://trac.nginx.org/nginx/changeset/4982/nginx

Log:
Image filter: fixed image_filter rotate inheritance.

Configurations like

    location /i/ {
        image_filter resize 200 200;
        image_filter rotate 180;

        location /i/foo/ {
            image_filter resize 200 200;
        }
   }

resulted in rotation incorrectly applied in the location /i/foo, without
any way to clear it.  Fix is to handle conf->angle/conf->acv consistently
with other filter variables and do not try to inherit them if there are
transformations defined for current location.


Modified:
   trunk/src/http/modules/ngx_http_image_filter_module.c

Modified: trunk/src/http/modules/ngx_http_image_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_image_filter_module.c	2012-12-21 08:46:52 UTC (rev 4981)
+++ trunk/src/http/modules/ngx_http_image_filter_module.c	2012-12-21 15:07:45 UTC (rev 4982)
@@ -1169,10 +1169,22 @@
         return NULL;
     }
 
+    /*
+     * set by ngx_pcalloc():
+     *
+     *     conf->width = 0;
+     *     conf->height = 0;
+     *     conf->angle = 0;
+     *     conf->wcv = NULL;
+     *     conf->hcv = NULL;
+     *     conf->acv = NULL;
+     *     conf->jqcv = NULL;
+     *     conf->shcv = NULL;
+     */
+
     conf->filter = NGX_CONF_UNSET_UINT;
     conf->jpeg_quality = NGX_CONF_UNSET_UINT;
     conf->sharpen = NGX_CONF_UNSET_UINT;
-    conf->angle = NGX_CONF_UNSET_UINT;
     conf->transparency = NGX_CONF_UNSET;
     conf->buffer_size = NGX_CONF_UNSET_SIZE;
 
@@ -1195,8 +1207,10 @@
             conf->filter = prev->filter;
             conf->width = prev->width;
             conf->height = prev->height;
+            conf->angle = prev->angle;
             conf->wcv = prev->wcv;
             conf->hcv = prev->hcv;
+            conf->acv = prev->acv;
         }
     }
 
@@ -1218,14 +1232,6 @@
         }
     }
 
-    if (conf->angle == NGX_CONF_UNSET_UINT) {
-        ngx_conf_merge_uint_value(conf->angle, prev->angle, 0);
-
-        if (conf->acv == NULL) {
-            conf->acv = prev->acv;
-        }
-    }
-
     ngx_conf_merge_value(conf->transparency, prev->transparency, 1);
 
     ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size,


From mdounin at mdounin.ru  Fri Dec 21 16:13:04 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Fri, 21 Dec 2012 16:13:04 +0000
Subject: [nginx] svn commit:  r4983 - trunk/src/os/unix
Message-ID: <20121221161304.52E4D3FAA59@mail.nginx.com>

Author: mdounin
Date: 2012-12-21 16:13:03 +0000 (Fri, 21 Dec 2012)
New Revision: 4983
URL: http://trac.nginx.org/nginx/changeset/4983/nginx

Log:
Core: crypt_r() error handling fixed.

The crypt_r() function returns NULL on errors, check it explicitly instead
of assuming errno will remain 0 if there are no errors (per POSIX, the
setting of errno after a successful call to a function is unspecified
unless the description of that function specifies that errno shall not
be modified).

Additionally, dropped unneeded ngx_set_errno(0) and fixed error handling
of memory allocation after normal crypt(), which was inapropriate and
resulted in null pointer dereference on allocation failures.


Modified:
   trunk/src/os/unix/ngx_user.c

Modified: trunk/src/os/unix/ngx_user.c
===================================================================
--- trunk/src/os/unix/ngx_user.c	2012-12-21 15:07:45 UTC (rev 4982)
+++ trunk/src/os/unix/ngx_user.c	2012-12-21 16:13:03 UTC (rev 4983)
@@ -28,30 +28,27 @@
 {
     char               *value;
     size_t              len;
-    ngx_err_t           err;
     struct crypt_data   cd;
 
-    ngx_set_errno(0);
-
     cd.initialized = 0;
     /* work around the glibc bug */
     cd.current_salt[0] = ~salt[0];
 
     value = crypt_r((char *) key, (char *) salt, &cd);
 
-    err = ngx_errno;
-
-    if (err == 0) {
+    if (value) {
         len = ngx_strlen(value) + 1;
 
         *encrypted = ngx_pnalloc(pool, len);
-        if (*encrypted) {
-            ngx_memcpy(*encrypted, value, len);
-            return NGX_OK;
+        if (*encrypted == NULL) {
+            return NGX_ERROR;
         }
+
+        ngx_memcpy(*encrypted, value, len);
+        return NGX_OK;
     }
 
-    ngx_log_error(NGX_LOG_CRIT, pool->log, err, "crypt_r() failed");
+    ngx_log_error(NGX_LOG_CRIT, pool->log, ngx_errno, "crypt_r() failed");
 
     return NGX_ERROR;
 }
@@ -75,18 +72,20 @@
 
 #endif
 
-    ngx_set_errno(0);
-
     value = crypt((char *) key, (char *) salt);
 
     if (value) {
         len = ngx_strlen(value) + 1;
 
         *encrypted = ngx_pnalloc(pool, len);
-        if (*encrypted) {
-            ngx_memcpy(*encrypted, value, len);
+        if (*encrypted == NULL) {
+#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT)
+            ngx_mutex_unlock(ngx_crypt_mutex);
+#endif
+            return NGX_ERROR;
         }
 
+        ngx_memcpy(*encrypted, value, len);
 #if (NGX_THREADS && NGX_NONREENTRANT_CRYPT)
         ngx_mutex_unlock(ngx_crypt_mutex);
 #endif


From ru at nginx.com  Sat Dec 22 20:03:39 2012
From: ru at nginx.com (ru at nginx.com)
Date: Sat, 22 Dec 2012 20:03:39 +0000
Subject: [nginx] svn commit:  r4984 - trunk/src/core
Message-ID: <20121222200339.753BE3FA00A@mail.nginx.com>

Author: ru
Date: 2012-12-22 20:03:38 +0000 (Sat, 22 Dec 2012)
New Revision: 4984
URL: http://trac.nginx.org/nginx/changeset/4984/nginx

Log:
Properly initialize "struct in6_addr" with zeroes.


Modified:
   trunk/src/core/ngx_inet.c

Modified: trunk/src/core/ngx_inet.c
===================================================================
--- trunk/src/core/ngx_inet.c	2012-12-21 16:13:03 UTC (rev 4983)
+++ trunk/src/core/ngx_inet.c	2012-12-22 20:03:38 UTC (rev 4984)
@@ -465,7 +465,7 @@
      * prevent MSVC8 warning:
      *    potentially uninitialized local variable 'inaddr6' used
      */
-    ngx_memzero(inaddr6.s6_addr, sizeof(struct in6_addr));
+    ngx_memzero(&inaddr6, sizeof(struct in6_addr));
 #endif
 
     inaddr = ngx_inet_addr(text, len);


From vbart at nginx.com  Sun Dec 23 15:27:56 2012
From: vbart at nginx.com (vbart at nginx.com)
Date: Sun, 23 Dec 2012 15:27:56 +0000
Subject: [nginx] svn commit:  r4985 - trunk/src/http/modules
Message-ID: <20121223152756.A96EE3F9DC4@mail.nginx.com>

Author: vbart
Date: 2012-12-23 15:27:55 +0000 (Sun, 23 Dec 2012)
New Revision: 4985
URL: http://trac.nginx.org/nginx/changeset/4985/nginx

Log:
Access log: fixed redundant buffer reallocation.

Previously a new buffer was allocated for every "access_log" directive with the
same file path and "buffer=" parameters, while only one buffer per file is used.



Modified:
   trunk/src/http/modules/ngx_http_log_module.c

Modified: trunk/src/http/modules/ngx_http_log_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_log_module.c	2012-12-22 20:03:38 UTC (rev 4984)
+++ trunk/src/http/modules/ngx_http_log_module.c	2012-12-23 15:27:55 UTC (rev 4985)
@@ -970,11 +970,15 @@
             return NGX_CONF_ERROR;
         }
 
-        if (log->file->buffer && log->file->last - log->file->pos != buf) {
-            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
-                               "access_log \"%V\" already defined "
-                               "with different buffer size", &value[1]);
-            return NGX_CONF_ERROR;
+        if (log->file->buffer) {
+            if (log->file->last - log->file->pos != buf) {
+                ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                                   "access_log \"%V\" already defined "
+                                   "with different buffer size", &value[1]);
+                return NGX_CONF_ERROR;
+            }
+
+            return NGX_CONF_OK;
         }
 
         log->file->buffer = ngx_palloc(cf->pool, buf);


From vbart at nginx.com  Sun Dec 23 15:36:52 2012
From: vbart at nginx.com (vbart at nginx.com)
Date: Sun, 23 Dec 2012 15:36:52 +0000
Subject: [nginx] svn commit:  r4986 - in trunk/src: core http/modules
Message-ID: <20121223153653.033DB3F9F41@mail.nginx.com>

Author: vbart
Date: 2012-12-23 15:36:52 +0000 (Sun, 23 Dec 2012)
New Revision: 4986
URL: http://trac.nginx.org/nginx/changeset/4986/nginx

Log:
Reopening log files code moved to a separate function.

The code refactored in a way to call custom handler that can do appropriate
cleanup work (if any), like flushing buffers, finishing compress streams,
finalizing connections to log daemon, etc..



Modified:
   trunk/src/core/ngx_conf_file.c
   trunk/src/core/ngx_conf_file.h
   trunk/src/core/ngx_cycle.c
   trunk/src/http/modules/ngx_http_log_module.c

Modified: trunk/src/core/ngx_conf_file.c
===================================================================
--- trunk/src/core/ngx_conf_file.c	2012-12-23 15:27:55 UTC (rev 4985)
+++ trunk/src/core/ngx_conf_file.c	2012-12-23 15:36:52 UTC (rev 4986)
@@ -945,7 +945,7 @@
         file->name = *name;
     }
 
-    file->buffer = NULL;
+    file->flush = NULL;
 
     return file;
 }
@@ -954,7 +954,6 @@
 static void
 ngx_conf_flush_files(ngx_cycle_t *cycle)
 {
-    ssize_t           n, len;
     ngx_uint_t        i;
     ngx_list_part_t  *part;
     ngx_open_file_t  *file;
@@ -975,24 +974,9 @@
             i = 0;
         }
 
-        len = file[i].pos - file[i].buffer;
-
-        if (file[i].buffer == NULL || len == 0) {
-            continue;
+        if (file[i].flush) {
+            file[i].flush(&file[i], cycle->log);
         }
-
-        n = ngx_write_fd(file[i].fd, file[i].buffer, len);
-
-        if (n == -1) {
-            ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno,
-                          ngx_write_fd_n " to \"%s\" failed",
-                          file[i].name.data);
-
-        } else if (n != len) {
-            ngx_log_error(NGX_LOG_ALERT, cycle->log, 0,
-                          ngx_write_fd_n " to \"%s\" was incomplete: %z of %uz",
-                          file[i].name.data, n, len);
-        }
     }
 }
 

Modified: trunk/src/core/ngx_conf_file.h
===================================================================
--- trunk/src/core/ngx_conf_file.h	2012-12-23 15:27:55 UTC (rev 4985)
+++ trunk/src/core/ngx_conf_file.h	2012-12-23 15:36:52 UTC (rev 4986)
@@ -91,17 +91,8 @@
     ngx_fd_t              fd;
     ngx_str_t             name;
 
-    u_char               *buffer;
-    u_char               *pos;
-    u_char               *last;
-
-#if 0
-    /* e.g. append mode, error_log */
-    ngx_uint_t            flags;
-    /* e.g. reopen db file */
-    ngx_uint_t          (*handler)(void *data, ngx_open_file_t *file);
+    void                (*flush)(ngx_open_file_t *file, ngx_log_t *log);
     void                 *data;
-#endif
 };
 
 

Modified: trunk/src/core/ngx_cycle.c
===================================================================
--- trunk/src/core/ngx_cycle.c	2012-12-23 15:27:55 UTC (rev 4985)
+++ trunk/src/core/ngx_cycle.c	2012-12-23 15:36:52 UTC (rev 4986)
@@ -1115,7 +1115,6 @@
 void
 ngx_reopen_files(ngx_cycle_t *cycle, ngx_uid_t user)
 {
-    ssize_t           n, len;
     ngx_fd_t          fd;
     ngx_uint_t        i;
     ngx_list_part_t  *part;
@@ -1139,24 +1138,8 @@
             continue;
         }
 
-        len = file[i].pos - file[i].buffer;
-
-        if (file[i].buffer && len != 0) {
-
-            n = ngx_write_fd(file[i].fd, file[i].buffer, len);
-
-            if (n == -1) {
-                ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno,
-                              ngx_write_fd_n " to \"%s\" failed",
-                              file[i].name.data);
-
-            } else if (n != len) {
-                ngx_log_error(NGX_LOG_ALERT, cycle->log, 0,
-                          ngx_write_fd_n " to \"%s\" was incomplete: %z of %uz",
-                          file[i].name.data, n, len);
-            }
-
-            file[i].pos = file[i].buffer;
+        if (file[i].flush) {
+            file[i].flush(&file[i], cycle->log);
         }
 
         fd = ngx_open_file(file[i].name.data, NGX_FILE_APPEND,

Modified: trunk/src/http/modules/ngx_http_log_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_log_module.c	2012-12-23 15:27:55 UTC (rev 4985)
+++ trunk/src/http/modules/ngx_http_log_module.c	2012-12-23 15:36:52 UTC (rev 4986)
@@ -41,6 +41,13 @@
 
 
 typedef struct {
+    u_char                     *start;
+    u_char                     *pos;
+    u_char                     *last;
+} ngx_http_log_buf_t;
+
+
+typedef struct {
     ngx_array_t                *lengths;
     ngx_array_t                *values;
 } ngx_http_log_script_t;
@@ -78,6 +85,8 @@
 static ssize_t ngx_http_log_script_write(ngx_http_request_t *r,
     ngx_http_log_script_t *script, u_char **name, u_char *buf, size_t len);
 
+static void ngx_http_log_flush(ngx_open_file_t *file, ngx_log_t *log);
+
 static u_char *ngx_http_log_pipe(ngx_http_request_t *r, u_char *buf,
     ngx_http_log_op_t *op);
 static u_char *ngx_http_log_time(ngx_http_request_t *r, u_char *buf,
@@ -216,8 +225,8 @@
     size_t                    len;
     ngx_uint_t                i, l;
     ngx_http_log_t           *log;
-    ngx_open_file_t          *file;
     ngx_http_log_op_t        *op;
+    ngx_http_log_buf_t       *buffer;
     ngx_http_log_loc_conf_t  *lcf;
 
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
@@ -258,21 +267,21 @@
 
         len += NGX_LINEFEED_SIZE;
 
-        file = log[l].file;
+        buffer = log[l].file ? log[l].file->data : NULL;
 
-        if (file && file->buffer) {
+        if (buffer) {
 
-            if (len > (size_t) (file->last - file->pos)) {
+            if (len > (size_t) (buffer->last - buffer->pos)) {
 
-                ngx_http_log_write(r, &log[l], file->buffer,
-                                   file->pos - file->buffer);
+                ngx_http_log_write(r, &log[l], buffer->start,
+                                   buffer->pos - buffer->start);
 
-                file->pos = file->buffer;
+                buffer->pos = buffer->start;
             }
 
-            if (len <= (size_t) (file->last - file->pos)) {
+            if (len <= (size_t) (buffer->last - buffer->pos)) {
 
-                p = file->pos;
+                p = buffer->pos;
 
                 for (i = 0; i < log[l].format->ops->nelts; i++) {
                     p = op[i].run(r, p, &op[i]);
@@ -280,7 +289,7 @@
 
                 ngx_linefeed(p);
 
-                file->pos = p;
+                buffer->pos = p;
 
                 continue;
             }
@@ -465,6 +474,38 @@
 }
 
 
+static void
+ngx_http_log_flush(ngx_open_file_t *file, ngx_log_t *log)
+{
+    size_t               len;
+    ssize_t              n;
+    ngx_http_log_buf_t  *buffer;
+
+    buffer = file->data;
+
+    len = buffer->pos - buffer->start;
+
+    if (len == 0) {
+        return;
+    }
+
+    n = ngx_write_fd(file->fd, buffer->start, len);
+
+    if (n == -1) {
+        ngx_log_error(NGX_LOG_ALERT, log, ngx_errno,
+                      ngx_write_fd_n " to \"%s\" failed",
+                      file->name.data);
+
+    } else if ((size_t) n != len) {
+        ngx_log_error(NGX_LOG_ALERT, log, 0,
+                      ngx_write_fd_n " to \"%s\" was incomplete: %z of %uz",
+                      file->name.data, n, len);
+    }
+
+    buffer->pos = buffer->start;
+}
+
+
 static u_char *
 ngx_http_log_copy_short(ngx_http_request_t *r, u_char *buf,
     ngx_http_log_op_t *op)
@@ -848,10 +889,11 @@
 {
     ngx_http_log_loc_conf_t *llcf = conf;
 
-    ssize_t                     buf;
+    ssize_t                     size;
     ngx_uint_t                  i, n;
     ngx_str_t                  *value, name;
     ngx_http_log_t             *log;
+    ngx_http_log_buf_t         *buffer;
     ngx_http_log_fmt_t         *fmt;
     ngx_http_log_main_conf_t   *lmcf;
     ngx_http_script_compile_t   sc;
@@ -962,16 +1004,18 @@
         name.len = value[3].len - 7;
         name.data = value[3].data + 7;
 
-        buf = ngx_parse_size(&name);
+        size = ngx_parse_size(&name);
 
-        if (buf == NGX_ERROR) {
+        if (size == NGX_ERROR) {
             ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                                "invalid buffer value \"%V\"", &name);
             return NGX_CONF_ERROR;
         }
 
-        if (log->file->buffer) {
-            if (log->file->last - log->file->pos != buf) {
+        if (log->file->data) {
+            buffer = log->file->data;
+
+            if (buffer->last - buffer->start != size) {
                 ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                                    "access_log \"%V\" already defined "
                                    "with different buffer size", &value[1]);
@@ -981,13 +1025,21 @@
             return NGX_CONF_OK;
         }
 
-        log->file->buffer = ngx_palloc(cf->pool, buf);
-        if (log->file->buffer == NULL) {
+        buffer = ngx_pcalloc(cf->pool, sizeof(ngx_http_log_buf_t));
+        if (buffer == NULL) {
             return NGX_CONF_ERROR;
         }
 
-        log->file->pos = log->file->buffer;
-        log->file->last = log->file->buffer + buf;
+        buffer->start = ngx_pnalloc(cf->pool, size);
+        if (buffer->start == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        buffer->pos = buffer->start;
+        buffer->last = buffer->start + size;
+
+        log->file->flush = ngx_http_log_flush;
+        log->file->data = buffer;
     }
 
     return NGX_CONF_OK;


From vbart at nginx.com  Sun Dec 23 15:51:47 2012
From: vbart at nginx.com (vbart at nginx.com)
Date: Sun, 23 Dec 2012 15:51:47 +0000
Subject: [nginx] svn commit:  r4987 - trunk/src/http/modules
Message-ID: <20121223155147.BCD3A3F9F06@mail.nginx.com>

Author: vbart
Date: 2012-12-23 15:51:47 +0000 (Sun, 23 Dec 2012)
New Revision: 4987
URL: http://trac.nginx.org/nginx/changeset/4987/nginx

Log:
Access log: the "flush" parameter of the "access_log" directive.



Modified:
   trunk/src/http/modules/ngx_http_log_module.c

Modified: trunk/src/http/modules/ngx_http_log_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_log_module.c	2012-12-23 15:36:52 UTC (rev 4986)
+++ trunk/src/http/modules/ngx_http_log_module.c	2012-12-23 15:51:47 UTC (rev 4987)
@@ -44,6 +44,9 @@
     u_char                     *start;
     u_char                     *pos;
     u_char                     *last;
+
+    ngx_event_t                *event;
+    ngx_msec_t                  flush;
 } ngx_http_log_buf_t;
 
 
@@ -86,6 +89,7 @@
     ngx_http_log_script_t *script, u_char **name, u_char *buf, size_t len);
 
 static void ngx_http_log_flush(ngx_open_file_t *file, ngx_log_t *log);
+static void ngx_http_log_flush_handler(ngx_event_t *ev);
 
 static u_char *ngx_http_log_pipe(ngx_http_request_t *r, u_char *buf,
     ngx_http_log_op_t *op);
@@ -141,7 +145,7 @@
 
     { ngx_string("access_log"),
       NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF
-                        |NGX_HTTP_LMT_CONF|NGX_CONF_TAKE123,
+                        |NGX_HTTP_LMT_CONF|NGX_CONF_1MORE,
       ngx_http_log_set_log,
       NGX_HTTP_LOC_CONF_OFFSET,
       0,
@@ -283,6 +287,10 @@
 
                 p = buffer->pos;
 
+                if (buffer->event && p == buffer->start) {
+                    ngx_add_timer(buffer->event, buffer->flush);
+                }
+
                 for (i = 0; i < log[l].format->ops->nelts; i++) {
                     p = op[i].run(r, p, &op[i]);
                 }
@@ -293,6 +301,10 @@
 
                 continue;
             }
+
+            if (buffer->event && buffer->event->timer_set) {
+                ngx_del_timer(buffer->event);
+            }
         }
 
         line = ngx_pnalloc(r->pool, len);
@@ -503,9 +515,23 @@
     }
 
     buffer->pos = buffer->start;
+
+    if (buffer->event && buffer->event->timer_set) {
+        ngx_del_timer(buffer->event);
+    }
 }
 
 
+static void
+ngx_http_log_flush_handler(ngx_event_t *ev)
+{
+    ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ev->log, 0,
+                   "http log buffer flush handler");
+
+    ngx_http_log_flush(ev->data, ev->log);
+}
+
+
 static u_char *
 ngx_http_log_copy_short(ngx_http_request_t *r, u_char *buf,
     ngx_http_log_op_t *op)
@@ -891,7 +917,8 @@
 
     ssize_t                     size;
     ngx_uint_t                  i, n;
-    ngx_str_t                  *value, name;
+    ngx_msec_t                  flush;
+    ngx_str_t                  *value, name, s;
     ngx_http_log_t             *log;
     ngx_http_log_buf_t         *buffer;
     ngx_http_log_fmt_t         *fmt;
@@ -978,47 +1005,81 @@
             && ngx_strcasecmp(fmt[i].name.data, name.data) == 0)
         {
             log->format = &fmt[i];
-            goto buffer;
+            break;
         }
     }
 
-    ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
-                       "unknown log format \"%V\"", &name);
-    return NGX_CONF_ERROR;
+    if (log->format == NULL) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "unknown log format \"%V\"", &name);
+        return NGX_CONF_ERROR;
+    }
 
-buffer:
+    size = 0;
+    flush = 0;
 
-    if (cf->args->nelts == 4) {
-        if (ngx_strncmp(value[3].data, "buffer=", 7) != 0) {
-            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
-                               "invalid parameter \"%V\"", &value[3]);
-            return NGX_CONF_ERROR;
+    for (i = 3; i < cf->args->nelts; i++) {
+
+        if (ngx_strncmp(value[i].data, "buffer=", 7) == 0) {
+            s.len = value[i].len - 7;
+            s.data = value[i].data + 7;
+
+            size = ngx_parse_size(&s);
+
+            if (size == NGX_ERROR || size == 0) {
+                ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                                   "invalid buffer size \"%V\"", &s);
+                return NGX_CONF_ERROR;
+            }
+
+            continue;
         }
 
-        if (log->script) {
-            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
-                               "buffered logs cannot have variables in name");
-            return NGX_CONF_ERROR;
+        if (ngx_strncmp(value[i].data, "flush=", 6) == 0) {
+            s.len = value[i].len - 6;
+            s.data = value[i].data + 6;
+
+            flush = ngx_parse_time(&s, 0);
+
+            if (flush == (ngx_msec_t) NGX_ERROR || flush == 0) {
+                ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                                   "invalid flush time \"%V\"", &s);
+                return NGX_CONF_ERROR;
+            }
+
+            continue;
         }
 
-        name.len = value[3].len - 7;
-        name.data = value[3].data + 7;
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "invalid parameter \"%V\"", &value[i]);
+        return NGX_CONF_ERROR;
+    }
 
-        size = ngx_parse_size(&name);
+    if (flush && size == 0) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "no buffer is defined for access_log \"%V\"",
+                           &value[1]);
+        return NGX_CONF_ERROR;
+    }
 
-        if (size == NGX_ERROR) {
+    if (size) {
+
+        if (log->script) {
             ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
-                               "invalid buffer value \"%V\"", &name);
+                               "buffered logs cannot have variables in name");
             return NGX_CONF_ERROR;
         }
 
         if (log->file->data) {
             buffer = log->file->data;
 
-            if (buffer->last - buffer->start != size) {
+            if (buffer->last - buffer->start != size
+                || buffer->flush != flush)
+            {
                 ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                                    "access_log \"%V\" already defined "
-                                   "with different buffer size", &value[1]);
+                                   "with conflicting parameters",
+                                   &value[1]);
                 return NGX_CONF_ERROR;
             }
 
@@ -1038,6 +1099,19 @@
         buffer->pos = buffer->start;
         buffer->last = buffer->start + size;
 
+        if (flush) {
+            buffer->event = ngx_pcalloc(cf->pool, sizeof(ngx_event_t));
+            if (buffer->event == NULL) {
+                return NGX_CONF_ERROR;
+            }
+
+            buffer->event->data = log->file;
+            buffer->event->handler = ngx_http_log_flush_handler;
+            buffer->event->log = &cf->cycle->new_log;
+
+            buffer->flush = flush;
+        }
+
         log->file->flush = ngx_http_log_flush;
         log->file->data = buffer;
     }


From vbart at nginx.com  Sun Dec 23 16:04:15 2012
From: vbart at nginx.com (vbart at nginx.com)
Date: Sun, 23 Dec 2012 16:04:15 +0000
Subject: [nginx] svn commit:  r4988 - trunk/auto/lib/zlib
Message-ID: <20121223160415.30D763F9C3E@mail.nginx.com>

Author: vbart
Date: 2012-12-23 16:04:14 +0000 (Sun, 23 Dec 2012)
New Revision: 4988
URL: http://trac.nginx.org/nginx/changeset/4988/nginx

Log:
Configure: added the NGX_ZLIB define.

This was introduced for conditional compilation of the code that requires
the zlib library.



Modified:
   trunk/auto/lib/zlib/conf

Modified: trunk/auto/lib/zlib/conf
===================================================================
--- trunk/auto/lib/zlib/conf	2012-12-23 15:51:47 UTC (rev 4987)
+++ trunk/auto/lib/zlib/conf	2012-12-23 16:04:14 UTC (rev 4988)
@@ -9,11 +9,13 @@
     case "$NGX_CC_NAME" in
 
         msvc* | owc* | bcc)
+            have=NGX_ZLIB . auto/have
             LINK_DEPS="$LINK_DEPS $ZLIB/zlib.lib"
             CORE_LIBS="$CORE_LIBS $ZLIB/zlib.lib"
         ;;
 
         icc*)
+            have=NGX_ZLIB . auto/have
             LINK_DEPS="$LINK_DEPS $ZLIB/libz.a"
 
             # to allow -ipo optimization we link with the *.o but not library
@@ -30,6 +32,7 @@
         ;;
 
         *)
+            have=NGX_ZLIB . auto/have
             LINK_DEPS="$LINK_DEPS $ZLIB/libz.a"
             CORE_LIBS="$CORE_LIBS $ZLIB/libz.a"
             #CORE_LIBS="$CORE_LIBS -L $ZLIB -lz"
@@ -45,7 +48,7 @@
         # FreeBSD, Solaris, Linux
 
         ngx_feature="zlib library"
-        ngx_feature_name=
+        ngx_feature_name="NGX_ZLIB"
         ngx_feature_run=no
         ngx_feature_incs="#include <zlib.h>"
         ngx_feature_path=


From vbart at nginx.com  Sun Dec 23 19:09:33 2012
From: vbart at nginx.com (vbart at nginx.com)
Date: Sun, 23 Dec 2012 19:09:33 +0000
Subject: [nginx] svn commit:  r4989 - trunk/src/http/modules
Message-ID: <20121223190933.D01C53F9C0F@mail.nginx.com>

Author: vbart
Date: 2012-12-23 19:09:33 +0000 (Sun, 23 Dec 2012)
New Revision: 4989
URL: http://trac.nginx.org/nginx/changeset/4989/nginx

Log:
Access log: the "gzip" parameter of the "access_log" directive.

Note: this requires zlib version 1.2.0.4 or above to work.



Modified:
   trunk/src/http/modules/ngx_http_log_module.c

Modified: trunk/src/http/modules/ngx_http_log_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_log_module.c	2012-12-23 16:04:14 UTC (rev 4988)
+++ trunk/src/http/modules/ngx_http_log_module.c	2012-12-23 19:09:33 UTC (rev 4989)
@@ -9,7 +9,11 @@
 #include <ngx_core.h>
 #include <ngx_http.h>
 
+#if (NGX_ZLIB)
+#include <zlib.h>
+#endif
 
+
 typedef struct ngx_http_log_op_s  ngx_http_log_op_t;
 
 typedef u_char *(*ngx_http_log_op_run_pt) (ngx_http_request_t *r, u_char *buf,
@@ -47,6 +51,7 @@
 
     ngx_event_t                *event;
     ngx_msec_t                  flush;
+    ngx_int_t                   gzip;
 } ngx_http_log_buf_t;
 
 
@@ -88,6 +93,14 @@
 static ssize_t ngx_http_log_script_write(ngx_http_request_t *r,
     ngx_http_log_script_t *script, u_char **name, u_char *buf, size_t len);
 
+#if (NGX_ZLIB)
+static ssize_t ngx_http_log_gzip(ngx_fd_t fd, u_char *buf, size_t len,
+    ngx_int_t level, ngx_log_t *log);
+
+static void *ngx_http_log_gzip_alloc(void *opaque, u_int items, u_int size);
+static void ngx_http_log_gzip_free(void *opaque, void *address);
+#endif
+
 static void ngx_http_log_flush(ngx_open_file_t *file, ngx_log_t *log);
 static void ngx_http_log_flush_handler(ngx_event_t *ev);
 
@@ -331,14 +344,29 @@
 ngx_http_log_write(ngx_http_request_t *r, ngx_http_log_t *log, u_char *buf,
     size_t len)
 {
-    u_char     *name;
-    time_t      now;
-    ssize_t     n;
-    ngx_err_t   err;
+    u_char              *name;
+    time_t               now;
+    ssize_t              n;
+    ngx_err_t            err;
+#if (NGX_ZLIB)
+    ngx_http_log_buf_t  *buffer;
+#endif
 
     if (log->script == NULL) {
         name = log->file->name.data;
+
+#if (NGX_ZLIB)
+        buffer = log->file->data;
+
+        if (buffer && buffer->gzip) {
+            n = ngx_http_log_gzip(log->file->fd, buf, len, buffer->gzip,
+                                  r->connection->log);
+        } else {
+            n = ngx_write_fd(log->file->fd, buf, len);
+        }
+#else
         n = ngx_write_fd(log->file->fd, buf, len);
+#endif
 
     } else {
         name = NULL;
@@ -486,7 +514,141 @@
 }
 
 
+#if (NGX_ZLIB)
+
+static ssize_t
+ngx_http_log_gzip(ngx_fd_t fd, u_char *buf, size_t len, ngx_int_t level,
+    ngx_log_t *log)
+{
+    int          rc, wbits, memlevel;
+    u_char      *out;
+    size_t       size;
+    ssize_t      n;
+    z_stream     zstream;
+    ngx_err_t    err;
+    ngx_pool_t  *pool;
+
+    wbits = MAX_WBITS;
+    memlevel = MAX_MEM_LEVEL - 1;
+
+    while ((ssize_t) len < ((1 << (wbits - 1)) - 262)) {
+        wbits--;
+        memlevel--;
+    }
+
+    /*
+     * This is a formula from deflateBound() for conservative upper bound of
+     * compressed data plus 18 bytes of gzip wrapper.
+     */
+
+    size = len + ((len + 7) >> 3) + ((len + 63) >> 6) + 5 + 18;
+
+    ngx_memzero(&zstream, sizeof(z_stream));
+
+    pool = ngx_create_pool(256, log);
+    if (pool == NULL) {
+        /* simulate successful logging */
+        return len;
+    }
+
+    pool->log = log;
+
+    zstream.zalloc = ngx_http_log_gzip_alloc;
+    zstream.zfree = ngx_http_log_gzip_free;
+    zstream.opaque = pool;
+
+    out = ngx_pnalloc(pool, size);
+    if (out == NULL) {
+        goto done;
+    }
+
+    zstream.next_in = buf;
+    zstream.avail_in = len;
+    zstream.next_out = out;
+    zstream.avail_out = size;
+
+    rc = deflateInit2(&zstream, (int) level, Z_DEFLATED, wbits + 16, memlevel,
+                      Z_DEFAULT_STRATEGY);
+
+    if (rc != Z_OK) {
+        ngx_log_error(NGX_LOG_ALERT, log, 0, "deflateInit2() failed: %d", rc);
+        goto done;
+    }
+
+    ngx_log_debug4(NGX_LOG_DEBUG_HTTP, log, 0,
+                   "deflate in: ni:%p no:%p ai:%ud ao:%ud",
+                   zstream.next_in, zstream.next_out,
+                   zstream.avail_in, zstream.avail_out);
+
+    rc = deflate(&zstream, Z_FINISH);
+
+    if (rc != Z_STREAM_END) {
+        ngx_log_error(NGX_LOG_ALERT, log, 0,
+                      "deflate(Z_FINISH) failed: %d", rc);
+        goto done;
+    }
+
+    ngx_log_debug5(NGX_LOG_DEBUG_HTTP, log, 0,
+                   "deflate out: ni:%p no:%p ai:%ud ao:%ud rc:%d",
+                   zstream.next_in, zstream.next_out,
+                   zstream.avail_in, zstream.avail_out,
+                   rc);
+
+    size -= zstream.avail_out;
+
+    rc = deflateEnd(&zstream);
+
+    if (rc != Z_OK) {
+        ngx_log_error(NGX_LOG_ALERT, log, 0, "deflateEnd() failed: %d", rc);
+        goto done;
+    }
+
+    n = ngx_write_fd(fd, out, size);
+
+    if (n != (ssize_t) size) {
+        err = (n == -1) ? ngx_errno : 0;
+
+        ngx_destroy_pool(pool);
+
+        ngx_set_errno(err);
+        return -1;
+    }
+
+done:
+
+    ngx_destroy_pool(pool);
+
+    /* simulate successful logging */
+    return len;
+}
+
+
+static void *
+ngx_http_log_gzip_alloc(void *opaque, u_int items, u_int size)
+{
+    ngx_pool_t *pool = opaque;
+
+    ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pool->log, 0,
+                   "gzip alloc: n:%ud s:%ud", items, size);
+
+    return ngx_palloc(pool, items * size);
+}
+
+
 static void
+ngx_http_log_gzip_free(void *opaque, void *address)
+{
+#if 0
+    ngx_pool_t *pool = opaque;
+
+    ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pool->log, 0, "gzip free: %p", address);
+#endif
+}
+
+#endif
+
+
+static void
 ngx_http_log_flush(ngx_open_file_t *file, ngx_log_t *log)
 {
     size_t               len;
@@ -501,7 +663,15 @@
         return;
     }
 
+#if (NGX_ZLIB)
+    if (buffer->gzip) {
+        n = ngx_http_log_gzip(file->fd, buffer->start, len, buffer->gzip, log);
+    } else {
+        n = ngx_write_fd(file->fd, buffer->start, len);
+    }
+#else
     n = ngx_write_fd(file->fd, buffer->start, len);
+#endif
 
     if (n == -1) {
         ngx_log_error(NGX_LOG_ALERT, log, ngx_errno,
@@ -916,6 +1086,7 @@
     ngx_http_log_loc_conf_t *llcf = conf;
 
     ssize_t                     size;
+    ngx_int_t                   gzip;
     ngx_uint_t                  i, n;
     ngx_msec_t                  flush;
     ngx_str_t                  *value, name, s;
@@ -1017,6 +1188,7 @@
 
     size = 0;
     flush = 0;
+    gzip = 0;
 
     for (i = 3; i < cf->args->nelts; i++) {
 
@@ -1050,6 +1222,39 @@
             continue;
         }
 
+        if (ngx_strncmp(value[i].data, "gzip", 4) == 0
+            && (value[i].len == 4 || value[i].data[4] == '='))
+        {
+#if (NGX_ZLIB)
+            if (size == 0) {
+                size = 64 * 1024;
+            }
+
+            if (value[i].len == 4) {
+                gzip = Z_BEST_SPEED;
+                continue;
+            }
+
+            s.len = value[i].len - 5;
+            s.data = value[i].data + 5;
+
+            gzip = ngx_atoi(s.data, s.len);
+
+            if (gzip < 1 || gzip > 9) {
+                ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                                   "invalid compression level \"%V\"", &s);
+                return NGX_CONF_ERROR;
+            }
+
+            continue;
+
+#else
+            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                               "nginx was built without zlib support");
+            return NGX_CONF_ERROR;
+#endif
+        }
+
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "invalid parameter \"%V\"", &value[i]);
         return NGX_CONF_ERROR;
@@ -1074,7 +1279,8 @@
             buffer = log->file->data;
 
             if (buffer->last - buffer->start != size
-                || buffer->flush != flush)
+                || buffer->flush != flush
+                || buffer->gzip != gzip)
             {
                 ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                                    "access_log \"%V\" already defined "
@@ -1112,6 +1318,8 @@
             buffer->flush = flush;
         }
 
+        buffer->gzip = gzip;
+
         log->file->flush = ngx_http_log_flush;
         log->file->data = buffer;
     }


From yaoweibin at gmail.com  Mon Dec 24 08:52:44 2012
From: yaoweibin at gmail.com (=?GB2312?B?0qbOsLHz?=)
Date: Mon, 24 Dec 2012 16:52:44 +0800
Subject: Update the Chinese documentation with the nginx.org website
Message-ID: <CAGxuLK1D-H-d8n=n+p6rdm7ByNayXGQQX8Bnz0H4VUUao4wt4A@mail.gmail.com>

Hi folks,

We have translated the nginx documentation into Chinese with the latest
version (nginx.org repository, Dec 20th).

More Modules whose Chinese description are available:
ngx_http_core_module
ngx_http_empty_gif_module
ngx_http_geo_module
ngx_http_geoip_module
ngx_http_image_filter_module
ngx_http_limit_req_module
ngx_http_log_module
ngx_http_map_module
ngx_http_proxy_module
ngx_http_referer_module
ngx_http_rewrite_module
ngx_http_upstream_module

I also updated the css to make it more beautiful for the Chinese document.
I'm very appreciated if anyone can review it.

Now you can preview the translation at
http://tengine.taobao.org/nginx_docs/cn/. We'll really appreciate it
if you report any errors or incorrectness to us. BTW, you are welcome
to join the Nginx Chinese Documentation Translation Project
(https://github.com/taobao/nginx-docs-cn). The process is quite easy:
fork the repository, do some translation then pull a request :)

Other parts of the translation will be posted when they are ready.

Merry Christmas. For all the Nginxers.

-- 
Weibin Yao
Developer @ Server Platform Team of Taobao
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121224/4d146454/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nginx_org_cn_20121220.diff
Type: application/octet-stream
Size: 186642 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121224/4d146454/attachment-0001.obj>

From ru at nginx.com  Mon Dec 24 10:09:48 2012
From: ru at nginx.com (Ruslan Ermilov)
Date: Mon, 24 Dec 2012 14:09:48 +0400
Subject: Update the Chinese documentation with the nginx.org website
In-Reply-To: <CAGxuLK1D-H-d8n=n+p6rdm7ByNayXGQQX8Bnz0H4VUUao4wt4A@mail.gmail.com>
References: <CAGxuLK1D-H-d8n=n+p6rdm7ByNayXGQQX8Bnz0H4VUUao4wt4A@mail.gmail.com>
Message-ID: <20121224100948.GA63924@lo0.su>

Hi there,

On Mon, Dec 24, 2012 at 04:52:44PM +0800, ??? wrote:
>    Hi folks,
>    We have translated the nginx documentation into Chinese with the latest
>    version ([1]nginx.org repository, Dec 20th).
>    More Modules whose Chinese description are available:
>    ngx_http_core_module
>    ngx_http_empty_gif_module
>    ngx_http_geo_module
>    ngx_http_geoip_module
>    ngx_http_image_filter_module
>    ngx_http_limit_req_module
>    ngx_http_log_module
>    ngx_http_map_module
>    ngx_http_proxy_module
>    ngx_http_referer_module
>    ngx_http_rewrite_module
>    ngx_http_upstream_module
>    I also updated the css to make it more beautiful?for the Chinese
>    document. I'm very appreciated if anyone can review it.
>    Now you can preview the translation at
>    [2]http://tengine.taobao.org/nginx_docs/cn/. We'll really appreciate it
>    if you report any errors or incorrectness to us. BTW, you are welcome
>    to join the Nginx Chinese Documentation Translation Project
>    ([3]https://github.com/taobao/nginx-docs-cn). The process is quite easy:
>    fork the repository, do some translation then pull a request :)
>    Other parts of the translation will be posted when they are ready.
>    Merry Christmas. For all the Nginxers.

I have no problem committing the "docs" part of the patch.  Do you
want this to appear on nginx.org now, or should I wait until after
you receive some feedback about possible "errors and incorrectness"?

Regarding the CSS changes.  The style.xsls patch can be minimized by
providing only what is actually different (text-align and font-family):
http://www.w3.org/TR/CSS2/cascade.html#cascading-order

Also, can you elaborate on why did you change the fonts, and what was
wrong with justified text?


From yaoweibin at gmail.com  Mon Dec 24 10:59:33 2012
From: yaoweibin at gmail.com (=?GB2312?B?0qbOsLHz?=)
Date: Mon, 24 Dec 2012 18:59:33 +0800
Subject: Update the Chinese documentation with the nginx.org website
In-Reply-To: <20121224100948.GA63924@lo0.su>
References: <CAGxuLK1D-H-d8n=n+p6rdm7ByNayXGQQX8Bnz0H4VUUao4wt4A@mail.gmail.com>
 <20121224100948.GA63924@lo0.su>
Message-ID: <CAGxuLK11OuWpmRQax3XY=M0opfkn9-hprYmSRy8Tr7dQBMkN2Q@mail.gmail.com>

2012/12/24 Ruslan Ermilov <ru at nginx.com>

> Hi there,
>
> On Mon, Dec 24, 2012 at 04:52:44PM +0800, ??? wrote:
> >    Hi folks,
> >    We have translated the nginx documentation into Chinese with the
> latest
> >    version ([1]nginx.org repository, Dec 20th).
> >    More Modules whose Chinese description are available:
> >    ngx_http_core_module
> >    ngx_http_empty_gif_module
> >    ngx_http_geo_module
> >    ngx_http_geoip_module
> >    ngx_http_image_filter_module
> >    ngx_http_limit_req_module
> >    ngx_http_log_module
> >    ngx_http_map_module
> >    ngx_http_proxy_module
> >    ngx_http_referer_module
> >    ngx_http_rewrite_module
> >    ngx_http_upstream_module
> >    I also updated the css to make it more beautiful for the Chinese
> >    document. I'm very appreciated if anyone can review it.
> >    Now you can preview the translation at
> >    [2]http://tengine.taobao.org/nginx_docs/cn/. We'll really appreciate
> it
> >    if you report any errors or incorrectness to us. BTW, you are welcome
> >    to join the Nginx Chinese Documentation Translation Project
> >    ([3]https://github.com/taobao/nginx-docs-cn). The process is quite
> easy:
> >    fork the repository, do some translation then pull a request :)
> >    Other parts of the translation will be posted when they are ready.
> >    Merry Christmas. For all the Nginxers.
>
> I have no problem committing the "docs" part of the patch.  Do you
> want this to appear on nginx.org now, or should I wait until after
> you receive some feedback about possible "errors and incorrectness"?
>
> Regarding the CSS changes.  The style.xsls patch can be minimized by
> providing only what is actually different (text-align and font-family):
> http://www.w3.org/TR/CSS2/cascade.html#cascading-order
>
> Also, can you elaborate on why did you change the fonts, and what was
> wrong with justified text?
>
Yes, if you think it's OK, of course you can push it to your website.

The font family is ugly for Chinese character with the original CSS,
especially in firefox. Maybe it can't find the right font and use other
font for replacement.

You can see the justified problem from this documentation (I'm using
Chrome): http://nginx.org/cn/docs/http/ngx_http_addition_module.html.
The space between two sentence is very large. It doesn't not change the
space between characters, it just changes the space between sentence when
justifying. In English or Russian, the justified work is good because the
extra space is added between each word. And you will not see large space
with them. Maybe the problem is not the CSS, but the browser.

-- 
Weibin Yao
Developer @ Server Platform Team of Taobao
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121224/ddb94962/attachment.html>

From crk_world at yahoo.com.cn  Mon Dec 24 11:18:52 2012
From: crk_world at yahoo.com.cn (chen cw)
Date: Mon, 24 Dec 2012 19:18:52 +0800
Subject: Update the Chinese documentation with the nginx.org website
In-Reply-To: <20121224100948.GA63924@lo0.su>
References: <CAGxuLK1D-H-d8n=n+p6rdm7ByNayXGQQX8Bnz0H4VUUao4wt4A@mail.gmail.com>
 <20121224100948.GA63924@lo0.su>
Message-ID: <CAKPCrs0PKEV=0JSkGFPYs_-F5=HPwxtjsYFPR-xDJJZt=TK30g@mail.gmail.com>

When using justified text, the white space among the text is over 30
pixels, very ugly. It is because there are much fewer white space in
Chinese text than in text written by other language.


On Mon, Dec 24, 2012 at 6:09 PM, Ruslan Ermilov <ru at nginx.com> wrote:

> Hi there,
>
> On Mon, Dec 24, 2012 at 04:52:44PM +0800, ??? wrote:
> >    Hi folks,
> >    We have translated the nginx documentation into Chinese with the
> latest
> >    version ([1]nginx.org repository, Dec 20th).
> >    More Modules whose Chinese description are available:
> >    ngx_http_core_module
> >    ngx_http_empty_gif_module
> >    ngx_http_geo_module
> >    ngx_http_geoip_module
> >    ngx_http_image_filter_module
> >    ngx_http_limit_req_module
> >    ngx_http_log_module
> >    ngx_http_map_module
> >    ngx_http_proxy_module
> >    ngx_http_referer_module
> >    ngx_http_rewrite_module
> >    ngx_http_upstream_module
> >    I also updated the css to make it more beautiful for the Chinese
> >    document. I'm very appreciated if anyone can review it.
> >    Now you can preview the translation at
> >    [2]http://tengine.taobao.org/nginx_docs/cn/. We'll really appreciate
> it
> >    if you report any errors or incorrectness to us. BTW, you are welcome
> >    to join the Nginx Chinese Documentation Translation Project
> >    ([3]https://github.com/taobao/nginx-docs-cn). The process is quite
> easy:
> >    fork the repository, do some translation then pull a request :)
> >    Other parts of the translation will be posted when they are ready.
> >    Merry Christmas. For all the Nginxers.
>
> I have no problem committing the "docs" part of the patch.  Do you
> want this to appear on nginx.org now, or should I wait until after
> you receive some feedback about possible "errors and incorrectness"?
>
> Regarding the CSS changes.  The style.xsls patch can be minimized by
> providing only what is actually different (text-align and font-family):
> http://www.w3.org/TR/CSS2/cascade.html#cascading-order
>
> Also, can you elaborate on why did you change the fonts, and what was
> wrong with justified text?
>



-- 

--

Charles Chen

Software Engineer

Server Platforms Team at Taobao.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121224/cb876fbb/attachment.html>

From ru at nginx.com  Mon Dec 24 16:40:55 2012
From: ru at nginx.com (ru at nginx.com)
Date: Mon, 24 Dec 2012 16:40:55 +0000
Subject: [nginx] svn commit:  r4990 - trunk/src/http/modules
Message-ID: <20121224164055.BD62C3F9F5D@mail.nginx.com>

Author: ru
Date: 2012-12-24 16:40:55 +0000 (Mon, 24 Dec 2012)
New Revision: 4990
URL: http://trac.nginx.org/nginx/changeset/4990/nginx

Log:
Trailing whitespace fix.


Modified:
   trunk/src/http/modules/ngx_http_geo_module.c

Modified: trunk/src/http/modules/ngx_http_geo_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_geo_module.c	2012-12-23 19:09:33 UTC (rev 4989)
+++ trunk/src/http/modules/ngx_http_geo_module.c	2012-12-24 16:40:55 UTC (rev 4990)
@@ -191,9 +191,9 @@
 
     if (ctx->u.high.low) {
         addr = ngx_http_geo_addr(r, ctx);
-    
+
         range = ctx->u.high.low[addr >> 16];
-    
+
         if (range) {
             n = addr & 0xffff;
             do {


From vbart at nginx.com  Mon Dec 24 17:32:53 2012
From: vbart at nginx.com (vbart at nginx.com)
Date: Mon, 24 Dec 2012 17:32:53 +0000
Subject: [nginx] svn commit:  r4991 - in trunk/src/http: . modules
Message-ID: <20121224173253.D0DEC3F9F41@mail.nginx.com>

Author: vbart
Date: 2012-12-24 17:32:53 +0000 (Mon, 24 Dec 2012)
New Revision: 4991
URL: http://trac.nginx.org/nginx/changeset/4991/nginx

Log:
Fixed HEAD requests handling when proxying is used (closes #261).


Modified:
   trunk/src/http/modules/ngx_http_gzip_filter_module.c
   trunk/src/http/ngx_http_upstream.c

Modified: trunk/src/http/modules/ngx_http_gzip_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_gzip_filter_module.c	2012-12-24 16:40:55 UTC (rev 4990)
+++ trunk/src/http/modules/ngx_http_gzip_filter_module.c	2012-12-24 17:32:53 UTC (rev 4991)
@@ -321,7 +321,7 @@
 
     ctx = ngx_http_get_module_ctx(r, ngx_http_gzip_filter_module);
 
-    if (ctx == NULL || ctx->done) {
+    if (ctx == NULL || ctx->done || r->header_only) {
         return ngx_http_next_body_filter(r, in);
     }
 

Modified: trunk/src/http/ngx_http_upstream.c
===================================================================
--- trunk/src/http/ngx_http_upstream.c	2012-12-24 16:40:55 UTC (rev 4990)
+++ trunk/src/http/ngx_http_upstream.c	2012-12-24 17:32:53 UTC (rev 4991)
@@ -3099,6 +3099,7 @@
     r->connection->log->action = "sending to client";
 
     if (rc == 0
+        && !r->header_only
 #if (NGX_HTTP_CACHE)
         && !r->cached
 #endif


From vbart at nginx.com  Tue Dec 25 08:02:22 2012
From: vbart at nginx.com (vbart at nginx.com)
Date: Tue, 25 Dec 2012 08:02:22 +0000
Subject: [nginx] svn commit:  r4992 - in trunk/src/http: . modules
Message-ID: <20121225080222.5383D3F9DF8@mail.nginx.com>

Author: vbart
Date: 2012-12-25 08:02:21 +0000 (Tue, 25 Dec 2012)
New Revision: 4992
URL: http://trac.nginx.org/nginx/changeset/4992/nginx

Log:
Upstream: fixed state resetting when switching to backup servers.

Based on patch by Thomas Chen (ticket #257).



Modified:
   trunk/src/http/modules/ngx_http_upstream_least_conn_module.c
   trunk/src/http/ngx_http_upstream_round_robin.c

Modified: trunk/src/http/modules/ngx_http_upstream_least_conn_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_upstream_least_conn_module.c	2012-12-24 17:32:53 UTC (rev 4991)
+++ trunk/src/http/modules/ngx_http_upstream_least_conn_module.c	2012-12-25 08:02:21 UTC (rev 4992)
@@ -313,7 +313,9 @@
         lcp->rrp.peers = peers->next;
         pc->tries = lcp->rrp.peers->number;
 
-        n = lcp->rrp.peers->number / (8 * sizeof(uintptr_t)) + 1;
+        n = (lcp->rrp.peers->number + (8 * sizeof(uintptr_t) - 1))
+                / (8 * sizeof(uintptr_t));
+
         for (i = 0; i < n; i++) {
              lcp->rrp.tried[i] = 0;
         }

Modified: trunk/src/http/ngx_http_upstream_round_robin.c
===================================================================
--- trunk/src/http/ngx_http_upstream_round_robin.c	2012-12-24 17:32:53 UTC (rev 4991)
+++ trunk/src/http/ngx_http_upstream_round_robin.c	2012-12-25 08:02:21 UTC (rev 4992)
@@ -474,7 +474,9 @@
         rrp->peers = peers->next;
         pc->tries = rrp->peers->number;
 
-        n = rrp->peers->number / (8 * sizeof(uintptr_t)) + 1;
+        n = (rrp->peers->number + (8 * sizeof(uintptr_t) - 1))
+                / (8 * sizeof(uintptr_t));
+
         for (i = 0; i < n; i++) {
              rrp->tried[i] = 0;
         }


From ru at nginx.com  Tue Dec 25 08:21:56 2012
From: ru at nginx.com (ru at nginx.com)
Date: Tue, 25 Dec 2012 08:21:56 +0000
Subject: [nginx] svn commit:  r4993 - in trunk/src: core http/modules
Message-ID: <20121225082156.CD71D3F9F6E@mail.nginx.com>

Author: ru
Date: 2012-12-25 08:21:56 +0000 (Tue, 25 Dec 2012)
New Revision: 4993
URL: http://trac.nginx.org/nginx/changeset/4993/nginx

Log:
Geo: IPv6 support.

The "ranges" mode is still limited to IPv4 only.


Modified:
   trunk/src/core/ngx_radix_tree.c
   trunk/src/core/ngx_radix_tree.h
   trunk/src/http/modules/ngx_http_geo_module.c

Modified: trunk/src/core/ngx_radix_tree.c
===================================================================
--- trunk/src/core/ngx_radix_tree.c	2012-12-25 08:02:21 UTC (rev 4992)
+++ trunk/src/core/ngx_radix_tree.c	2012-12-25 08:21:56 UTC (rev 4993)
@@ -263,6 +263,203 @@
 }
 
 
+#if (NGX_HAVE_INET6)
+
+ngx_int_t
+ngx_radix128tree_insert(ngx_radix_tree_t *tree, u_char *key, u_char *mask,
+    uintptr_t value)
+{
+    u_char             bit;
+    ngx_uint_t         i;
+    ngx_radix_node_t  *node, *next;
+
+    i = 0;
+    bit = 0x80;
+
+    node = tree->root;
+    next = tree->root;
+
+    while (bit & mask[i]) {
+        if (key[i] & bit) {
+            next = node->right;
+
+        } else {
+            next = node->left;
+        }
+
+        if (next == NULL) {
+            break;
+        }
+
+        bit >>= 1;
+        node = next;
+
+        if (bit == 0) {
+            if (++i == 16) {
+                break;
+            }
+
+            bit = 0x80;
+        }
+    }
+
+    if (next) {
+        if (node->value != NGX_RADIX_NO_VALUE) {
+            return NGX_BUSY;
+        }
+
+        node->value = value;
+        return NGX_OK;
+    }
+
+    while (bit & mask[i]) {
+        next = ngx_radix_alloc(tree);
+        if (next == NULL) {
+            return NGX_ERROR;
+        }
+
+        next->right = NULL;
+        next->left = NULL;
+        next->parent = node;
+        next->value = NGX_RADIX_NO_VALUE;
+
+        if (key[i] & bit) {
+            node->right = next;
+
+        } else {
+            node->left = next;
+        }
+
+        bit >>= 1;
+        node = next;
+
+        if (bit == 0) {
+            if (++i == 16) {
+                break;
+            }
+
+            bit = 0x80;
+        }
+    }
+
+    node->value = value;
+
+    return NGX_OK;
+}
+
+
+ngx_int_t
+ngx_radix128tree_delete(ngx_radix_tree_t *tree, u_char *key, u_char *mask)
+{
+    u_char             bit;
+    ngx_uint_t         i;
+    ngx_radix_node_t  *node;
+
+    i = 0;
+    bit = 0x80;
+    node = tree->root;
+
+    while (node && (bit & mask[i])) {
+        if (key[i] & bit) {
+            node = node->right;
+
+        } else {
+            node = node->left;
+        }
+
+        bit >>= 1;
+
+        if (bit == 0) {
+            if (++i == 16) {
+                break;
+            }
+
+            bit = 0x80;
+        }
+    }
+
+    if (node == NULL) {
+        return NGX_ERROR;
+    }
+
+    if (node->right || node->left) {
+        if (node->value != NGX_RADIX_NO_VALUE) {
+            node->value = NGX_RADIX_NO_VALUE;
+            return NGX_OK;
+        }
+
+        return NGX_ERROR;
+    }
+
+    for ( ;; ) {
+        if (node->parent->right == node) {
+            node->parent->right = NULL;
+
+        } else {
+            node->parent->left = NULL;
+        }
+
+        node->right = tree->free;
+        tree->free = node;
+
+        node = node->parent;
+
+        if (node->right || node->left) {
+            break;
+        }
+
+        if (node->value != NGX_RADIX_NO_VALUE) {
+            break;
+        }
+
+        if (node->parent == NULL) {
+            break;
+        }
+    }
+
+    return NGX_OK;
+}
+
+
+uintptr_t
+ngx_radix128tree_find(ngx_radix_tree_t *tree, u_char *key)
+{
+    u_char             bit;
+    uintptr_t          value;
+    ngx_uint_t         i;
+    ngx_radix_node_t  *node;
+
+    i = 0;
+    bit = 0x80;
+    value = NGX_RADIX_NO_VALUE;
+    node = tree->root;
+
+    while (node) {
+        if (node->value != NGX_RADIX_NO_VALUE) {
+            value = node->value;
+        }
+
+        if (key[i] & bit) {
+            node = node->right;
+
+        } else {
+            node = node->left;
+        }
+
+        bit >>= 1;
+
+        if (bit == 0) {
+            i++;
+            bit = 0x80;
+        }
+    }
+
+    return value;
+}
+
+#endif
+
+
 static ngx_radix_node_t *
 ngx_radix_alloc(ngx_radix_tree_t *tree)
 {

Modified: trunk/src/core/ngx_radix_tree.h
===================================================================
--- trunk/src/core/ngx_radix_tree.h	2012-12-25 08:02:21 UTC (rev 4992)
+++ trunk/src/core/ngx_radix_tree.h	2012-12-25 08:21:56 UTC (rev 4993)
@@ -36,11 +36,20 @@
 
 ngx_radix_tree_t *ngx_radix_tree_create(ngx_pool_t *pool,
     ngx_int_t preallocate);
+
 ngx_int_t ngx_radix32tree_insert(ngx_radix_tree_t *tree,
     uint32_t key, uint32_t mask, uintptr_t value);
 ngx_int_t ngx_radix32tree_delete(ngx_radix_tree_t *tree,
     uint32_t key, uint32_t mask);
 uintptr_t ngx_radix32tree_find(ngx_radix_tree_t *tree, uint32_t key);
 
+#if (NGX_HAVE_INET6)
+ngx_int_t ngx_radix128tree_insert(ngx_radix_tree_t *tree,
+    u_char *key, u_char *mask, uintptr_t value);
+ngx_int_t ngx_radix128tree_delete(ngx_radix_tree_t *tree,
+    u_char *key, u_char *mask);
+uintptr_t ngx_radix128tree_find(ngx_radix_tree_t *tree, u_char *key);
+#endif
 
+
 #endif /* _NGX_RADIX_TREE_H_INCLUDED_ */

Modified: trunk/src/http/modules/ngx_http_geo_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_geo_module.c	2012-12-25 08:02:21 UTC (rev 4992)
+++ trunk/src/http/modules/ngx_http_geo_module.c	2012-12-25 08:21:56 UTC (rev 4993)
@@ -18,6 +18,14 @@
 
 
 typedef struct {
+    ngx_radix_tree_t                *tree;
+#if (NGX_HAVE_INET6)
+    ngx_radix_tree_t                *tree6;
+#endif
+} ngx_http_geo_trees_t;
+
+
+typedef struct {
     ngx_http_geo_range_t           **low;
     ngx_http_variable_value_t       *default_value;
 } ngx_http_geo_high_ranges_t;
@@ -35,6 +43,9 @@
     ngx_str_t                       *net;
     ngx_http_geo_high_ranges_t       high;
     ngx_radix_tree_t                *tree;
+#if (NGX_HAVE_INET6)
+    ngx_radix_tree_t                *tree6;
+#endif
     ngx_rbtree_t                     rbtree;
     ngx_rbtree_node_t                sentinel;
     ngx_array_t                     *proxies;
@@ -57,7 +68,7 @@
 
 typedef struct {
     union {
-        ngx_radix_tree_t            *tree;
+        ngx_http_geo_trees_t         trees;
         ngx_http_geo_high_ranges_t   high;
     } u;
 
@@ -68,8 +79,8 @@
 } ngx_http_geo_ctx_t;
 
 
-static in_addr_t ngx_http_geo_addr(ngx_http_request_t *r,
-    ngx_http_geo_ctx_t *ctx);
+static ngx_int_t ngx_http_geo_addr(ngx_http_request_t *r,
+    ngx_http_geo_ctx_t *ctx, ngx_addr_t *addr);
 static ngx_int_t ngx_http_geo_real_addr(ngx_http_request_t *r,
     ngx_http_geo_ctx_t *ctx, ngx_addr_t *addr);
 static char *ngx_http_geo_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf);
@@ -155,7 +166,7 @@
 };
 
 
-/* AF_INET only */
+/* geo range is AF_INET only */
 
 static ngx_int_t
 ngx_http_geo_cidr_variable(ngx_http_request_t *r, ngx_http_variable_value_t *v,
@@ -163,11 +174,57 @@
 {
     ngx_http_geo_ctx_t *ctx = (ngx_http_geo_ctx_t *) data;
 
+    in_addr_t                   inaddr;
+    ngx_addr_t                  addr;
+    struct sockaddr_in         *sin;
     ngx_http_variable_value_t  *vv;
+#if (NGX_HAVE_INET6)
+    u_char                     *p;
+    struct in6_addr            *inaddr6;
+#endif
 
-    vv = (ngx_http_variable_value_t *)
-              ngx_radix32tree_find(ctx->u.tree, ngx_http_geo_addr(r, ctx));
+    if (ngx_http_geo_addr(r, ctx, &addr) != NGX_OK) {
+        vv = (ngx_http_variable_value_t *)
+                  ngx_radix32tree_find(ctx->u.trees.tree, INADDR_NONE);
+        goto done;
+    }
 
+    switch (addr.sockaddr->sa_family) {
+
+#if (NGX_HAVE_INET6)
+    case AF_INET6:
+        inaddr6 = &((struct sockaddr_in6 *) addr.sockaddr)->sin6_addr;
+        p = inaddr6->s6_addr;
+
+        if (IN6_IS_ADDR_V4MAPPED(inaddr6)) {
+            inaddr = p[12] << 24;
+            inaddr += p[13] << 16;
+            inaddr += p[14] << 8;
+            inaddr += p[15];
+
+            vv = (ngx_http_variable_value_t *)
+                      ngx_radix32tree_find(ctx->u.trees.tree, inaddr);
+
+        } else {
+            vv = (ngx_http_variable_value_t *)
+                      ngx_radix128tree_find(ctx->u.trees.tree6, p);
+        }
+
+        break;
+#endif
+
+    default: /* AF_INET */
+        sin = (struct sockaddr_in *) addr.sockaddr;
+        inaddr = ntohl(sin->sin_addr.s_addr);
+
+        vv = (ngx_http_variable_value_t *)
+                  ngx_radix32tree_find(ctx->u.trees.tree, inaddr);
+
+        break;
+    }
+
+done:
+
     *v = *vv;
 
     ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
@@ -183,19 +240,56 @@
 {
     ngx_http_geo_ctx_t *ctx = (ngx_http_geo_ctx_t *) data;
 
-    in_addr_t              addr;
+    in_addr_t              inaddr;
+    ngx_addr_t             addr;
     ngx_uint_t             n;
+    struct sockaddr_in    *sin;
     ngx_http_geo_range_t  *range;
+#if (NGX_HAVE_INET6)
+    u_char                *p;
+    struct in6_addr       *inaddr6;
+#endif
 
     *v = *ctx->u.high.default_value;
 
+    if (ngx_http_geo_addr(r, ctx, &addr) == NGX_OK) {
+
+        switch (addr.sockaddr->sa_family) {
+
+#if (NGX_HAVE_INET6)
+        case AF_INET6:
+            inaddr6 = &((struct sockaddr_in6 *) addr.sockaddr)->sin6_addr;
+
+            if (IN6_IS_ADDR_V4MAPPED(inaddr6)) {
+                p = inaddr6->s6_addr;
+
+                inaddr = p[12] << 24;
+                inaddr += p[13] << 16;
+                inaddr += p[14] << 8;
+                inaddr += p[15];
+
+            } else {
+                inaddr = INADDR_NONE;
+            }
+
+            break;
+#endif
+
+        default: /* AF_INET */
+            sin = (struct sockaddr_in *) addr.sockaddr;
+            inaddr = ntohl(sin->sin_addr.s_addr);
+            break;
+        }
+
+    } else {
+        inaddr = INADDR_NONE;
+    }
+
     if (ctx->u.high.low) {
-        addr = ngx_http_geo_addr(r, ctx);
+        range = ctx->u.high.low[inaddr >> 16];
 
-        range = ctx->u.high.low[addr >> 16];
-
         if (range) {
-            n = addr & 0xffff;
+            n = inaddr & 0xffff;
             do {
                 if (n >= (ngx_uint_t) range->start
                     && n <= (ngx_uint_t) range->end)
@@ -214,54 +308,25 @@
 }
 
 
-static in_addr_t
-ngx_http_geo_addr(ngx_http_request_t *r, ngx_http_geo_ctx_t *ctx)
+static ngx_int_t
+ngx_http_geo_addr(ngx_http_request_t *r, ngx_http_geo_ctx_t *ctx,
+    ngx_addr_t *addr)
 {
-    ngx_addr_t           addr;
-    ngx_table_elt_t     *xfwd;
-    struct sockaddr_in  *sin;
+    ngx_table_elt_t  *xfwd;
 
-    if (ngx_http_geo_real_addr(r, ctx, &addr) != NGX_OK) {
-        return INADDR_NONE;
+    if (ngx_http_geo_real_addr(r, ctx, addr) != NGX_OK) {
+        return NGX_ERROR;
     }
 
     xfwd = r->headers_in.x_forwarded_for;
 
     if (xfwd != NULL && ctx->proxies != NULL) {
-        (void) ngx_http_get_forwarded_addr(r, &addr, xfwd->value.data,
+        (void) ngx_http_get_forwarded_addr(r, addr, xfwd->value.data,
                                            xfwd->value.len, ctx->proxies,
                                            ctx->proxy_recursive);
     }
 
-#if (NGX_HAVE_INET6)
-
-    if (addr.sockaddr->sa_family == AF_INET6) {
-        u_char           *p;
-        in_addr_t         inaddr;
-        struct in6_addr  *inaddr6;
-
-        inaddr6 = &((struct sockaddr_in6 *) addr.sockaddr)->sin6_addr;
-
-        if (IN6_IS_ADDR_V4MAPPED(inaddr6)) {
-            p = inaddr6->s6_addr;
-
-            inaddr = p[12] << 24;
-            inaddr += p[13] << 16;
-            inaddr += p[14] << 8;
-            inaddr += p[15];
-
-            return inaddr;
-        }
-    }
-
-#endif
-
-    if (addr.sockaddr->sa_family != AF_INET) {
-        return INADDR_NONE;
-    }
-
-    sin = (struct sockaddr_in *) addr.sockaddr;
-    return ntohl(sin->sin_addr.s_addr);
+    return NGX_OK;
 }
 
 
@@ -315,6 +380,9 @@
     ngx_http_variable_t      *var;
     ngx_http_geo_ctx_t       *geo;
     ngx_http_geo_conf_ctx_t   ctx;
+#if (NGX_HAVE_INET6)
+    static struct in6_addr    zero;
+#endif
 
     value = cf->args->elts;
 
@@ -445,8 +513,19 @@
             }
         }
 
-        geo->u.tree = ctx.tree;
+        geo->u.trees.tree = ctx.tree;
 
+#if (NGX_HAVE_INET6)
+        if (ctx.tree6 == NULL) {
+            ctx.tree6 = ngx_radix_tree_create(cf->pool, -1);
+            if (ctx.tree6 == NULL) {
+                return NGX_CONF_ERROR;
+            }
+        }
+
+        geo->u.trees.tree6 = ctx.tree6;
+#endif
+
         var->get_handler = ngx_http_geo_cidr_variable;
         var->data = (uintptr_t) geo;
 
@@ -461,6 +540,15 @@
         }
 
         /* NGX_BUSY is okay (default was set explicitly) */
+
+#if (NGX_HAVE_INET6)
+        if (ngx_radix128tree_insert(ctx.tree6, zero.s6_addr, zero.s6_addr,
+                                    (uintptr_t) &ngx_http_variable_null_value)
+            == NGX_ERROR)
+        {
+            return NGX_CONF_ERROR;
+        }
+#endif
     }
 
     return rv;
@@ -483,7 +571,12 @@
 
         if (ngx_strcmp(value[0].data, "ranges") == 0) {
 
-            if (ctx->tree) {
+            if (ctx->tree
+#if (NGX_HAVE_INET6)
+                || ctx->tree6
+#endif
+               )
+            {
                 ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                                    "the \"ranges\" directive must be "
                                    "the first directive inside \"geo\" block");
@@ -934,6 +1027,15 @@
         }
     }
 
+#if (NGX_HAVE_INET6)
+    if (ctx->tree6 == NULL) {
+        ctx->tree6 = ngx_radix_tree_create(ctx->pool, -1);
+        if (ctx->tree6 == NULL) {
+            return NGX_CONF_ERROR;
+        }
+    }
+#endif
+
     if (ngx_strcmp(value[0].data, "default") == 0) {
         /* cidr.family = AF_INET; */
         cidr.u.in.addr = 0;
@@ -954,20 +1056,29 @@
             return NGX_CONF_ERROR;
         }
 
-        if (cidr.family != AF_INET) {
-            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
-                               "\"geo\" supports IPv4 only");
-            return NGX_CONF_ERROR;
+        if (cidr.family == AF_INET) {
+            cidr.u.in.addr = ntohl(cidr.u.in.addr);
+            cidr.u.in.mask = ntohl(cidr.u.in.mask);
         }
 
-        cidr.u.in.addr = ntohl(cidr.u.in.addr);
-        cidr.u.in.mask = ntohl(cidr.u.in.mask);
+        if (del) {
+            switch (cidr.family) {
 
-        if (del) {
-            if (ngx_radix32tree_delete(ctx->tree, cidr.u.in.addr,
-                                       cidr.u.in.mask)
-                != NGX_OK)
-            {
+#if (NGX_HAVE_INET6)
+            case AF_INET6:
+                rc = ngx_radix128tree_delete(ctx->tree6,
+                                             cidr.u.in6.addr.s6_addr,
+                                             cidr.u.in6.mask.s6_addr);
+                break;
+#endif
+
+            default: /* AF_INET */
+                rc = ngx_radix32tree_delete(ctx->tree, cidr.u.in.addr,
+                                            cidr.u.in.mask);
+                break;
+            }
+
+            if (rc != NGX_OK) {
                 ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
                                    "no network \"%V\" to delete", net);
             }
@@ -982,32 +1093,78 @@
         return NGX_CONF_ERROR;
     }
 
-    for (i = 2; i; i--) {
-        rc = ngx_radix32tree_insert(ctx->tree, cidr.u.in.addr, cidr.u.in.mask,
-                                    (uintptr_t) val);
-        if (rc == NGX_OK) {
-            return NGX_CONF_OK;
-        }
+    switch (cidr.family) {
 
-        if (rc == NGX_ERROR) {
-            return NGX_CONF_ERROR;
+#if (NGX_HAVE_INET6)
+    case AF_INET6:
+        for (i = 2; i; i--) {
+            rc = ngx_radix128tree_insert(ctx->tree6, cidr.u.in6.addr.s6_addr,
+                                         cidr.u.in6.mask.s6_addr,
+                                         (uintptr_t) val);
+
+            if (rc == NGX_OK) {
+                return NGX_CONF_OK;
+            }
+
+            if (rc == NGX_ERROR) {
+                return NGX_CONF_ERROR;
+            }
+
+            /* rc == NGX_BUSY */
+
+            old = (ngx_http_variable_value_t *)
+                       ngx_radix128tree_find(ctx->tree6,
+                                             cidr.u.in6.addr.s6_addr);
+
+            ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
+                  "duplicate network \"%V\", value: \"%v\", old value: \"%v\"",
+                  net, val, old);
+
+            rc = ngx_radix128tree_delete(ctx->tree6,
+                                         cidr.u.in6.addr.s6_addr,
+                                         cidr.u.in6.mask.s6_addr);
+
+            if (rc == NGX_ERROR) {
+                ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid radix tree");
+                return NGX_CONF_ERROR;
+            }
         }
 
-        /* rc == NGX_BUSY */
+        break;
+#endif
 
-        old = (ngx_http_variable_value_t *)
-              ngx_radix32tree_find(ctx->tree, cidr.u.in.addr);
+    default: /* AF_INET */
+        for (i = 2; i; i--) {
+            rc = ngx_radix32tree_insert(ctx->tree, cidr.u.in.addr,
+                                        cidr.u.in.mask, (uintptr_t) val);
 
-        ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
-                "duplicate network \"%V\", value: \"%v\", old value: \"%v\"",
-                net, val, old);
+            if (rc == NGX_OK) {
+                return NGX_CONF_OK;
+            }
 
-        rc = ngx_radix32tree_delete(ctx->tree, cidr.u.in.addr, cidr.u.in.mask);
+            if (rc == NGX_ERROR) {
+                return NGX_CONF_ERROR;
+            }
 
-        if (rc == NGX_ERROR) {
-            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid radix tree");
-            return NGX_CONF_ERROR;
+            /* rc == NGX_BUSY */
+
+            old = (ngx_http_variable_value_t *)
+                       ngx_radix32tree_find(ctx->tree, cidr.u.in.addr);
+
+            ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
+                  "duplicate network \"%V\", value: \"%v\", old value: \"%v\"",
+                  net, val, old);
+
+            rc = ngx_radix32tree_delete(ctx->tree,
+                                        cidr.u.in.addr, cidr.u.in.mask);
+
+            if (rc == NGX_ERROR) {
+                ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid radix tree");
+                return NGX_CONF_ERROR;
+            }
         }
+
+        break;
     }
 
     return NGX_CONF_ERROR;


From ru at nginx.com  Tue Dec 25 10:00:40 2012
From: ru at nginx.com (ru at nginx.com)
Date: Tue, 25 Dec 2012 10:00:40 +0000
Subject: [nginx] svn commit:  r4994 - trunk/src/http/modules
Message-ID: <20121225100041.2D27C3F9F7A@mail.nginx.com>

Author: ru
Date: 2012-12-25 10:00:39 +0000 (Tue, 25 Dec 2012)
New Revision: 4994
URL: http://trac.nginx.org/nginx/changeset/4994/nginx

Log:
Geo: properly initialize ngx_cidr_t when dealing with "default".


Modified:
   trunk/src/http/modules/ngx_http_geo_module.c

Modified: trunk/src/http/modules/ngx_http_geo_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_geo_module.c	2012-12-25 08:21:56 UTC (rev 4993)
+++ trunk/src/http/modules/ngx_http_geo_module.c	2012-12-25 10:00:39 UTC (rev 4994)
@@ -1037,7 +1037,7 @@
 #endif
 
     if (ngx_strcmp(value[0].data, "default") == 0) {
-        /* cidr.family = AF_INET; */
+        cidr.family = AF_INET;
         cidr.u.in.addr = 0;
         cidr.u.in.mask = 0;
         net = &value[0];


From mdounin at mdounin.ru  Tue Dec 25 14:23:46 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Tue, 25 Dec 2012 14:23:46 +0000
Subject: [nginx] svn commit:  r4995 - trunk/docs/xml/nginx
Message-ID: <20121225142347.0F53C3FA02D@mail.nginx.com>

Author: mdounin
Date: 2012-12-25 14:23:45 +0000 (Tue, 25 Dec 2012)
New Revision: 4995
URL: http://trac.nginx.org/nginx/changeset/4995/nginx

Log:
nginx-1.3.10-RELEASE


Modified:
   trunk/docs/xml/nginx/changes.xml

Modified: trunk/docs/xml/nginx/changes.xml
===================================================================
--- trunk/docs/xml/nginx/changes.xml	2012-12-25 10:00:39 UTC (rev 4994)
+++ trunk/docs/xml/nginx/changes.xml	2012-12-25 14:23:45 UTC (rev 4995)
@@ -4,7 +4,184 @@
 
 <change_log title="nginx">
 
+<changes ver="1.3.10" date="25.12.2012">
 
+<change type="change">
+<para lang="ru">
+??? ????????? ? ???????????????? ????? ???????? ???? ??????
+???????????? ?? ?????? IPv4, ?? ? IPv6 ??????.
+</para>
+<para lang="en">
+domain names specified in configuration file
+are now resolved to IPv6 addresses as well as IPv4 ones.
+</para>
+</change>
+
+<change type="change">
+<para lang="ru">
+?????? ??? ????????????? ????????? include ? ?????? ?? Unix-????????
+?????????? ????? ??????????? ? ?????????? ???????.
+</para>
+<para lang="en">
+now if the "include" directive with mask is used on Unix systems,
+included files are sorted in alphabetical order.
+</para>
+</change>
+
+<change type="change">
+<para lang="ru">
+????????? add_header ????????? ?????? ? ?????? ? ????? 201.
+</para>
+<para lang="en">
+the "add_header" directive adds headers to 201 responses.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+????????? geo ?????? ???????????? IPv6 ?????? ? ??????? CIDR.
+</para>
+<para lang="en">
+the "geo" directive now supports IPv6 addresses in CIDR notation.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+????????? flush ? gzip ? ????????? access_log.
+</para>
+<para lang="en">
+the "flush" and "gzip" parameters of the "access_log" directive.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+????????? auth_basic ???????????? ??????????.
+</para>
+<para lang="en">
+variables support in the "auth_basic" directive.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+nginx ? ????????? ??????? ?? ????????? ? ??????? ngx_http_perl_module.
+</para>
+<para lang="en">
+nginx could not be built with the ngx_http_perl_module in some cases.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ??????? ???????? ??? ????????? segmentation fault,
+???? ????????????? ?????? ngx_http_xslt_module.
+</para>
+<para lang="en">
+a segmentation fault might occur in a worker process
+if the ngx_http_xslt_module was used.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+nginx ??? ?? ?????????? ?? MacOSX.<br/>
+??????? Piotr Sikora.
+</para>
+<para lang="en">
+nginx could not be built on MacOSX in some cases.<br/>
+Thanks to Piotr Sikora.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ????????????? ????????? limit_rate ? ???????? ?????????? ????????
+?? 32-?????? ???????? ????? ??? ???????????? ?? ???????.<br/>
+??????? ??????? ?????????.
+</para>
+<para lang="en">
+the "limit_rate" directive with high rates
+might result in truncated responses on 32-bit platforms.<br/>
+Thanks to Alexey Antropov.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ??????? ???????? ??? ????????? segmentation fault,
+???? ?????????????? ????????? if.<br/>
+??????? Piotr Sikora.
+</para>
+<para lang="en">
+a segmentation fault might occur in a worker process
+if the "if" directive was used.<br/>
+Thanks to Piotr Sikora.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+????? "100 Continue" ?????????
+?????? ? ??????? "413 Request Entity Too Large".
+</para>
+<para lang="en">
+a "100 Continue" response was issued
+with "413 Request Entity Too Large" responses.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+????????? image_filter, image_filter_jpeg_quality ? image_filter_sharpen
+????? ????????????? ???????????.<br/>
+??????? ????? ???????.
+</para>
+<para lang="en">
+the "image_filter", "image_filter_jpeg_quality"
+and "image_filter_sharpen" directives
+might be inherited incorrectly.<br/>
+Thanks to Ian Babrou.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ????????????? ????????? auth_basic ??? Linux
+????? ????????? ?????? "crypt_r() failed".
+</para>
+<para lang="en">
+"crypt_r() failed" errors might appear
+if the "auth_basic" directive was used on Linux.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ????????? backup-????????.<br/>
+??????? Thomas Chen.
+</para>
+<para lang="en">
+in backup servers handling.<br/>
+Thanks to Thomas Chen.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ????????????? HEAD-???????? ??? ???????????? ???????????? ?????,
+???? ?????????????? ????????? gzip.
+</para>
+<para lang="en">
+proxied HEAD requests might return incorrect response
+if the "gzip" directive was used.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.3.9" date="27.11.2012">
 
 <change type="feature">


From mdounin at mdounin.ru  Tue Dec 25 14:24:12 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Tue, 25 Dec 2012 14:24:12 +0000
Subject: [nginx] svn commit:  r4996 - tags
Message-ID: <20121225142412.87B573F9F35@mail.nginx.com>

Author: mdounin
Date: 2012-12-25 14:24:12 +0000 (Tue, 25 Dec 2012)
New Revision: 4996
URL: http://trac.nginx.org/nginx/changeset/4996/nginx

Log:
release-1.3.10 tag


Added:
   tags/release-1.3.10/


From mdounin at mdounin.ru  Tue Dec 25 16:05:42 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 25 Dec 2012 20:05:42 +0400
Subject: image_filter enhancement
In-Reply-To: <CANWdNRDBkhQN_wAC9uCjL=SQ5GfyjnFJbJ43v9jpFXOUXp2NHA@mail.gmail.com>
References: <CANWdNRB=bRCEjEBM+ksak8c6pEZw+VC1XfKBrPLEJ5zXv+B_=g@mail.gmail.com>
 <20121127160328.GZ40452@mdounin.ru>
 <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>
 <CANWdNRBXuGyFdo1NGUeHba2QnSYokMJpD61tW-VNcOo=TV-9gg@mail.gmail.com>
 <20121204130841.GI40452@mdounin.ru>
 <CANWdNRATaVeEC7W0UVos9AdDM4Xq5BNRQUDea+UZkCuZzGL8gA@mail.gmail.com>
 <20121205145244.GO40452@mdounin.ru>
 <CANWdNRBW-uWR66FLqiEGu03n5wzLFfvQA+iXLxxVop-0J5bSbw@mail.gmail.com>
 <20121218105341.GK40452@mdounin.ru>
 <CANWdNRDBkhQN_wAC9uCjL=SQ5GfyjnFJbJ43v9jpFXOUXp2NHA@mail.gmail.com>
Message-ID: <20121225160542.GD40452@mdounin.ru>

Hello!

On Thu, Dec 20, 2012 at 09:01:38PM +0400, ivan babrou wrote:

> I've send patch for configuration semantics in separate thread.
> 
> Here's lastest version of my patch that may be applied after. Is there
> something else that I should fix?

I would like to see clarification on how are you going to handle 
just bare numbers specified.  As of now the code seems to don't do 
any special processing and seems to assume all values are correct, 
which might not be true and will result in some mostly arbitray 
crop offset selection.  It might make sense to actually support 
percents as in Maxim Bublis's patch[1].

[1] http://mailman.nginx.org/pipermail/nginx-devel/2012-April/002156.html

[...]

> @@ -1151,7 +1188,24 @@ ngx_http_image_filter_value(ngx_str_t *value)
> 
>      n = ngx_atoi(value->data, value->len);
> 
> -    if (n > 0) {
> +    if (n == NGX_ERROR) {
> +        if (ngx_strncmp(value->data, "left", value->len) == 0) {
> +            return NGX_HTTP_IMAGE_OFFSET_LEFT;
> +
> +        } else if (ngx_strncmp(value->data, "right", value->len) == 0) {
> +            return NGX_HTTP_IMAGE_OFFSET_RIGHT;
> +
> +        } else if (ngx_strncmp(value->data, "top", value->len) == 0) {
> +            return NGX_HTTP_IMAGE_OFFSET_TOP;
> +
> +        } else if (ngx_strncmp(value->data, "bottom", value->len) == 0) {
> +            return NGX_HTTP_IMAGE_OFFSET_BOTTOM;
> +
> +        } else {
> +            return NGX_HTTP_IMAGE_OFFSET_CENTER;
> +        }
> +
> +    } else if (n > 0) {
>          return (ngx_uint_t) n;
>      }
> 

The ngx_strncmp() checks are incorrect, see here:
http://mailman.nginx.org/pipermail/nginx-devel/2012-December/003065.html

You may also move them below "if (n > 0)" the check for better 
readability.

> @@ -1175,6 +1229,8 @@ ngx_http_image_filter_create_conf(ngx_conf_t *cf)
>      conf->angle = NGX_CONF_UNSET_UINT;
>      conf->transparency = NGX_CONF_UNSET;
>      conf->buffer_size = NGX_CONF_UNSET_SIZE;
> +    conf->crop_offset_x = NGX_CONF_UNSET_UINT;
> +    conf->crop_offset_y = NGX_CONF_UNSET_UINT;

The "crop_" prefix probably should be dropped here for better 
readability (and to match other names in the code like "oxcv").

[...]

-- 
Maxim Dounin
http://nginx.com/support.html


From ibobrik at gmail.com  Tue Dec 25 16:30:09 2012
From: ibobrik at gmail.com (ivan babrou)
Date: Tue, 25 Dec 2012 20:30:09 +0400
Subject: image_filter enhancement
In-Reply-To: <20121225160542.GD40452@mdounin.ru>
References: <CANWdNRB=bRCEjEBM+ksak8c6pEZw+VC1XfKBrPLEJ5zXv+B_=g@mail.gmail.com>
 <20121127160328.GZ40452@mdounin.ru>
 <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>
 <CANWdNRBXuGyFdo1NGUeHba2QnSYokMJpD61tW-VNcOo=TV-9gg@mail.gmail.com>
 <20121204130841.GI40452@mdounin.ru>
 <CANWdNRATaVeEC7W0UVos9AdDM4Xq5BNRQUDea+UZkCuZzGL8gA@mail.gmail.com>
 <20121205145244.GO40452@mdounin.ru>
 <CANWdNRBW-uWR66FLqiEGu03n5wzLFfvQA+iXLxxVop-0J5bSbw@mail.gmail.com>
 <20121218105341.GK40452@mdounin.ru>
 <CANWdNRDBkhQN_wAC9uCjL=SQ5GfyjnFJbJ43v9jpFXOUXp2NHA@mail.gmail.com>
 <20121225160542.GD40452@mdounin.ru>
Message-ID: <CANWdNRAW-SDng77mPdcH+Wp9iqYVLMEic_m+jRjuT4dW-FwGyw@mail.gmail.com>

On 25 December 2012 20:05, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Thu, Dec 20, 2012 at 09:01:38PM +0400, ivan babrou wrote:
>
> > I've send patch for configuration semantics in separate thread.
> >
> > Here's lastest version of my patch that may be applied after. Is there
> > something else that I should fix?
>
> I would like to see clarification on how are you going to handle
> just bare numbers specified.  As of now the code seems to don't do
> any special processing and seems to assume all values are correct,
> which might not be true and will result in some mostly arbitray
> crop offset selection.  It might make sense to actually support
> percents as in Maxim Bublis's patch[1].
>
> [1] http://mailman.nginx.org/pipermail/nginx-devel/2012-April/002156.html


I assume that ngx_atoi will return NGX_ERROR if it's not number. If number
is ok, (ngx_uint_t) n will be returned. If it's not a number, string checks
will be performed. If even crop offset value is wrong then
NGX_HTTP_IMAGE_OFFSET_CENTER
will be returned which is default behaviour and zero at the same time. So
if you pass incorrect number and expect a number then you'll just get zero.
Just like it was before.

Cropping with offsets that is specified in percents seems more specific
task. I don't think this is required.

[...]
>
> > @@ -1151,7 +1188,24 @@ ngx_http_image_filter_value(ngx_str_t *value)
> >
> >      n = ngx_atoi(value->data, value->len);
> >
> > -    if (n > 0) {
> > +    if (n == NGX_ERROR) {
> > +        if (ngx_strncmp(value->data, "left", value->len) == 0) {
> > +            return NGX_HTTP_IMAGE_OFFSET_LEFT;
> > +
> > +        } else if (ngx_strncmp(value->data, "right", value->len) == 0) {
> > +            return NGX_HTTP_IMAGE_OFFSET_RIGHT;
> > +
> > +        } else if (ngx_strncmp(value->data, "top", value->len) == 0) {
> > +            return NGX_HTTP_IMAGE_OFFSET_TOP;
> > +
> > +        } else if (ngx_strncmp(value->data, "bottom", value->len) == 0)
> {
> > +            return NGX_HTTP_IMAGE_OFFSET_BOTTOM;
> > +
> > +        } else {
> > +            return NGX_HTTP_IMAGE_OFFSET_CENTER;
> > +        }
> > +
> > +    } else if (n > 0) {
> >          return (ngx_uint_t) n;
> >      }
> >
>
> The ngx_strncmp() checks are incorrect, see here:
> http://mailman.nginx.org/pipermail/nginx-devel/2012-December/003065.html
>
> You may also move them below "if (n > 0)" the check for better
> readability.
>
> > @@ -1175,6 +1229,8 @@ ngx_http_image_filter_create_conf(ngx_conf_t *cf)
> >      conf->angle = NGX_CONF_UNSET_UINT;
> >      conf->transparency = NGX_CONF_UNSET;
> >      conf->buffer_size = NGX_CONF_UNSET_SIZE;
> > +    conf->crop_offset_x = NGX_CONF_UNSET_UINT;
> > +    conf->crop_offset_y = NGX_CONF_UNSET_UINT;
>
> The "crop_" prefix probably should be dropped here for better
> readability (and to match other names in the code like "oxcv").
>
> [...]
>
> --
> Maxim Dounin
> http://nginx.com/support.html
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>



-- 
Regards, Ian Babrou
http://bobrik.name http://twitter.com/ibobrik skype:i.babrou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121225/baefae89/attachment.html>

From ibobrik at gmail.com  Tue Dec 25 17:02:55 2012
From: ibobrik at gmail.com (ivan babrou)
Date: Tue, 25 Dec 2012 21:02:55 +0400
Subject: image_filter enhancement
In-Reply-To: <20121225160542.GD40452@mdounin.ru>
References: <CANWdNRB=bRCEjEBM+ksak8c6pEZw+VC1XfKBrPLEJ5zXv+B_=g@mail.gmail.com>
 <20121127160328.GZ40452@mdounin.ru>
 <CANWdNRBvsTXfg+n4yXrW6WJQxbpBtAtM1wn4_U1Qv2ydO5AxHg@mail.gmail.com>
 <CANWdNRBXuGyFdo1NGUeHba2QnSYokMJpD61tW-VNcOo=TV-9gg@mail.gmail.com>
 <20121204130841.GI40452@mdounin.ru>
 <CANWdNRATaVeEC7W0UVos9AdDM4Xq5BNRQUDea+UZkCuZzGL8gA@mail.gmail.com>
 <20121205145244.GO40452@mdounin.ru>
 <CANWdNRBW-uWR66FLqiEGu03n5wzLFfvQA+iXLxxVop-0J5bSbw@mail.gmail.com>
 <20121218105341.GK40452@mdounin.ru>
 <CANWdNRDBkhQN_wAC9uCjL=SQ5GfyjnFJbJ43v9jpFXOUXp2NHA@mail.gmail.com>
 <20121225160542.GD40452@mdounin.ru>
Message-ID: <CANWdNRAvHKZ_JcngFMCKRMvAYfRp67yjTURJMrVH=mKHqSfeWA@mail.gmail.com>

On 25 December 2012 20:05, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Thu, Dec 20, 2012 at 09:01:38PM +0400, ivan babrou wrote:
>
> > I've send patch for configuration semantics in separate thread.
> >
> > Here's lastest version of my patch that may be applied after. Is there
> > something else that I should fix?
>
> I would like to see clarification on how are you going to handle
> just bare numbers specified.  As of now the code seems to don't do
> any special processing and seems to assume all values are correct,
> which might not be true and will result in some mostly arbitray
> crop offset selection.  It might make sense to actually support
> percents as in Maxim Bublis's patch[1].
>
> [1] http://mailman.nginx.org/pipermail/nginx-devel/2012-April/002156.html
>
> [...]
>
> > @@ -1151,7 +1188,24 @@ ngx_http_image_filter_value(ngx_str_t *value)
> >
> >      n = ngx_atoi(value->data, value->len);
> >
> > -    if (n > 0) {
> > +    if (n == NGX_ERROR) {
> > +        if (ngx_strncmp(value->data, "left", value->len) == 0) {
> > +            return NGX_HTTP_IMAGE_OFFSET_LEFT;
> > +
> > +        } else if (ngx_strncmp(value->data, "right", value->len) == 0) {
> > +            return NGX_HTTP_IMAGE_OFFSET_RIGHT;
> > +
> > +        } else if (ngx_strncmp(value->data, "top", value->len) == 0) {
> > +            return NGX_HTTP_IMAGE_OFFSET_TOP;
> > +
> > +        } else if (ngx_strncmp(value->data, "bottom", value->len) == 0)
> {
> > +            return NGX_HTTP_IMAGE_OFFSET_BOTTOM;
> > +
> > +        } else {
> > +            return NGX_HTTP_IMAGE_OFFSET_CENTER;
> > +        }
> > +
> > +    } else if (n > 0) {
> >          return (ngx_uint_t) n;
> >      }
> >
>
> The ngx_strncmp() checks are incorrect, see here:
> http://mailman.nginx.org/pipermail/nginx-devel/2012-December/003065.html
>

But my strings are null-terminated. Should I remove null termination and
check as you said?


> You may also move them below "if (n > 0)" the check for better
> readability.
>
> > @@ -1175,6 +1229,8 @@ ngx_http_image_filter_create_conf(ngx_conf_t *cf)
> >      conf->angle = NGX_CONF_UNSET_UINT;
> >      conf->transparency = NGX_CONF_UNSET;
> >      conf->buffer_size = NGX_CONF_UNSET_SIZE;
> > +    conf->crop_offset_x = NGX_CONF_UNSET_UINT;
> > +    conf->crop_offset_y = NGX_CONF_UNSET_UINT;
>
> The "crop_" prefix probably should be dropped here for better
> readability (and to match other names in the code like "oxcv").
>
> [...]
>
> --
> Maxim Dounin
> http://nginx.com/support.html
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>



-- 
Regards, Ian Babrou
http://bobrik.name http://twitter.com/ibobrik skype:i.babrou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121225/60299eaf/attachment-0001.html>

From ru at nginx.com  Wed Dec 26 05:03:52 2012
From: ru at nginx.com (ru at nginx.com)
Date: Wed, 26 Dec 2012 05:03:52 +0000
Subject: [nginx] svn commit:  r4997 - trunk/src/http/modules
Message-ID: <20121226050352.322773FA10F@mail.nginx.com>

Author: ru
Date: 2012-12-26 05:03:51 +0000 (Wed, 26 Dec 2012)
New Revision: 4997
URL: http://trac.nginx.org/nginx/changeset/4997/nginx

Log:
Geo: made "default" affect both IPv4 and IPv6 when using prefixes.

Previously, "default" was equivalent to specifying 0.0.0.0/0, now
it's equivalent to specifying both 0.0.0.0/0 and ::/0 (if support
for IPv6 is enabled) with the same value.


Modified:
   trunk/src/http/modules/ngx_http_geo_module.c

Modified: trunk/src/http/modules/ngx_http_geo_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_geo_module.c	2012-12-25 14:24:12 UTC (rev 4996)
+++ trunk/src/http/modules/ngx_http_geo_module.c	2012-12-26 05:03:51 UTC (rev 4997)
@@ -93,6 +93,8 @@
     ngx_http_geo_conf_ctx_t *ctx, in_addr_t start, in_addr_t end);
 static char *ngx_http_geo_cidr(ngx_conf_t *cf, ngx_http_geo_conf_ctx_t *ctx,
     ngx_str_t *value);
+static char *ngx_http_geo_cidr_add(ngx_conf_t *cf, ngx_http_geo_conf_ctx_t *ctx,
+    ngx_cidr_t *cidr, ngx_str_t *value, ngx_str_t *net);
 static ngx_http_variable_value_t *ngx_http_geo_value(ngx_conf_t *cf,
     ngx_http_geo_conf_ctx_t *ctx, ngx_str_t *value);
 static char *ngx_http_geo_add_proxy(ngx_conf_t *cf,
@@ -1014,11 +1016,9 @@
 ngx_http_geo_cidr(ngx_conf_t *cf, ngx_http_geo_conf_ctx_t *ctx,
     ngx_str_t *value)
 {
-    ngx_int_t                        rc, del;
-    ngx_str_t                       *net;
-    ngx_uint_t                       i;
-    ngx_cidr_t                       cidr;
-    ngx_http_variable_value_t       *val, *old;
+    ngx_int_t    rc, del;
+    ngx_str_t   *net;
+    ngx_cidr_t   cidr;
 
     if (ctx->tree == NULL) {
         ctx->tree = ngx_radix_tree_create(ctx->pool, -1);
@@ -1040,66 +1040,95 @@
         cidr.family = AF_INET;
         cidr.u.in.addr = 0;
         cidr.u.in.mask = 0;
-        net = &value[0];
 
-    } else {
-        if (ngx_strcmp(value[0].data, "delete") == 0) {
-            net = &value[1];
-            del = 1;
-
-        } else {
-            net = &value[0];
-            del = 0;
+        if (ngx_http_geo_cidr_add(cf, ctx, &cidr, &value[1], &value[0])
+            != NGX_CONF_OK)
+        {
+            return NGX_CONF_ERROR;
         }
 
-        if (ngx_http_geo_cidr_value(cf, net, &cidr) != NGX_OK) {
+#if (NGX_HAVE_INET6)
+        cidr.family = AF_INET6;
+        ngx_memzero(&cidr.u.in6, sizeof(ngx_in6_cidr_t));
+
+        if (ngx_http_geo_cidr_add(cf, ctx, &cidr, &value[1], &value[0])
+            != NGX_CONF_OK)
+        {
             return NGX_CONF_ERROR;
         }
+#endif
 
-        if (cidr.family == AF_INET) {
-            cidr.u.in.addr = ntohl(cidr.u.in.addr);
-            cidr.u.in.mask = ntohl(cidr.u.in.mask);
-        }
+        return NGX_CONF_OK;
+    }
 
-        if (del) {
-            switch (cidr.family) {
+    if (ngx_strcmp(value[0].data, "delete") == 0) {
+        net = &value[1];
+        del = 1;
 
+    } else {
+        net = &value[0];
+        del = 0;
+    }
+
+    if (ngx_http_geo_cidr_value(cf, net, &cidr) != NGX_OK) {
+        return NGX_CONF_ERROR;
+    }
+
+    if (cidr.family == AF_INET) {
+        cidr.u.in.addr = ntohl(cidr.u.in.addr);
+        cidr.u.in.mask = ntohl(cidr.u.in.mask);
+    }
+
+    if (del) {
+        switch (cidr.family) {
+
 #if (NGX_HAVE_INET6)
-            case AF_INET6:
-                rc = ngx_radix128tree_delete(ctx->tree6,
-                                             cidr.u.in6.addr.s6_addr,
-                                             cidr.u.in6.mask.s6_addr);
-                break;
+        case AF_INET6:
+            rc = ngx_radix128tree_delete(ctx->tree6,
+                                         cidr.u.in6.addr.s6_addr,
+                                         cidr.u.in6.mask.s6_addr);
+            break;
 #endif
 
-            default: /* AF_INET */
-                rc = ngx_radix32tree_delete(ctx->tree, cidr.u.in.addr,
-                                            cidr.u.in.mask);
-                break;
-            }
+        default: /* AF_INET */
+            rc = ngx_radix32tree_delete(ctx->tree, cidr.u.in.addr,
+                                        cidr.u.in.mask);
+            break;
+        }
 
-            if (rc != NGX_OK) {
-                ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
-                                   "no network \"%V\" to delete", net);
-            }
+        if (rc != NGX_OK) {
+            ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
+                               "no network \"%V\" to delete", net);
+        }
 
-            return NGX_CONF_OK;
-        }
+        return NGX_CONF_OK;
     }
 
-    val = ngx_http_geo_value(cf, ctx, &value[1]);
+    return ngx_http_geo_cidr_add(cf, ctx, &cidr, &value[1], net);
+}
 
+
+static char *
+ngx_http_geo_cidr_add(ngx_conf_t *cf, ngx_http_geo_conf_ctx_t *ctx,
+    ngx_cidr_t *cidr, ngx_str_t *value, ngx_str_t *net)
+{
+    ngx_int_t                   rc;
+    ngx_uint_t                  i;
+    ngx_http_variable_value_t  *val, *old;
+
+    val = ngx_http_geo_value(cf, ctx, value);
+
     if (val == NULL) {
         return NGX_CONF_ERROR;
     }
 
-    switch (cidr.family) {
+    switch (cidr->family) {
 
 #if (NGX_HAVE_INET6)
     case AF_INET6:
         for (i = 2; i; i--) {
-            rc = ngx_radix128tree_insert(ctx->tree6, cidr.u.in6.addr.s6_addr,
-                                         cidr.u.in6.mask.s6_addr,
+            rc = ngx_radix128tree_insert(ctx->tree6, cidr->u.in6.addr.s6_addr,
+                                         cidr->u.in6.mask.s6_addr,
                                          (uintptr_t) val);
 
             if (rc == NGX_OK) {
@@ -1114,15 +1143,15 @@
 
             old = (ngx_http_variable_value_t *)
                        ngx_radix128tree_find(ctx->tree6,
-                                             cidr.u.in6.addr.s6_addr);
+                                             cidr->u.in6.addr.s6_addr);
 
             ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
                   "duplicate network \"%V\", value: \"%v\", old value: \"%v\"",
                   net, val, old);
 
             rc = ngx_radix128tree_delete(ctx->tree6,
-                                         cidr.u.in6.addr.s6_addr,
-                                         cidr.u.in6.mask.s6_addr);
+                                         cidr->u.in6.addr.s6_addr,
+                                         cidr->u.in6.mask.s6_addr);
 
             if (rc == NGX_ERROR) {
                 ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid radix tree");
@@ -1135,8 +1164,8 @@
 
     default: /* AF_INET */
         for (i = 2; i; i--) {
-            rc = ngx_radix32tree_insert(ctx->tree, cidr.u.in.addr,
-                                        cidr.u.in.mask, (uintptr_t) val);
+            rc = ngx_radix32tree_insert(ctx->tree, cidr->u.in.addr,
+                                        cidr->u.in.mask, (uintptr_t) val);
 
             if (rc == NGX_OK) {
                 return NGX_CONF_OK;
@@ -1149,14 +1178,14 @@
             /* rc == NGX_BUSY */
 
             old = (ngx_http_variable_value_t *)
-                       ngx_radix32tree_find(ctx->tree, cidr.u.in.addr);
+                       ngx_radix32tree_find(ctx->tree, cidr->u.in.addr);
 
             ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
                   "duplicate network \"%V\", value: \"%v\", old value: \"%v\"",
                   net, val, old);
 
             rc = ngx_radix32tree_delete(ctx->tree,
-                                        cidr.u.in.addr, cidr.u.in.mask);
+                                        cidr->u.in.addr, cidr->u.in.mask);
 
             if (rc == NGX_ERROR) {
                 ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid radix tree");


From ru at nginx.com  Wed Dec 26 09:29:38 2012
From: ru at nginx.com (ru at nginx.com)
Date: Wed, 26 Dec 2012 09:29:38 +0000
Subject: [nginx] svn commit:  r4998 - in trunk/src: core http/modules/perl
Message-ID: <20121226092938.5A2FB3F9F46@mail.nginx.com>

Author: ru
Date: 2012-12-26 09:29:37 +0000 (Wed, 26 Dec 2012)
New Revision: 4998
URL: http://trac.nginx.org/nginx/changeset/4998/nginx

Log:
Version bump.


Modified:
   trunk/src/core/nginx.h
   trunk/src/http/modules/perl/nginx.pm

Modified: trunk/src/core/nginx.h
===================================================================
--- trunk/src/core/nginx.h	2012-12-26 05:03:51 UTC (rev 4997)
+++ trunk/src/core/nginx.h	2012-12-26 09:29:37 UTC (rev 4998)
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1003010
-#define NGINX_VERSION      "1.3.10"
+#define nginx_version      1003011
+#define NGINX_VERSION      "1.3.11"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #define NGINX_VAR          "NGINX"

Modified: trunk/src/http/modules/perl/nginx.pm
===================================================================
--- trunk/src/http/modules/perl/nginx.pm	2012-12-26 05:03:51 UTC (rev 4997)
+++ trunk/src/http/modules/perl/nginx.pm	2012-12-26 09:29:37 UTC (rev 4998)
@@ -50,7 +50,7 @@
     HTTP_INSUFFICIENT_STORAGE
 );
 
-our $VERSION = '1.3.10';
+our $VERSION = '1.3.11';
 
 require XSLoader;
 XSLoader::load('nginx', $VERSION);


From ru at nginx.com  Wed Dec 26 14:46:06 2012
From: ru at nginx.com (ru at nginx.com)
Date: Wed, 26 Dec 2012 14:46:06 +0000
Subject: [nginx] svn commit:  r4999 - trunk/src/http/modules
Message-ID: <20121226144606.7D0BF3F9C45@mail.nginx.com>

Author: ru
Date: 2012-12-26 14:46:06 +0000 (Wed, 26 Dec 2012)
New Revision: 4999
URL: http://trac.nginx.org/nginx/changeset/4999/nginx

Log:
Upstream keepalive: detect duplicate "keepalive" directive.

A failure to detect duplicate "keepalive" directive resulted in
stack exhaustion.


Modified:
   trunk/src/http/modules/ngx_http_upstream_keepalive_module.c

Modified: trunk/src/http/modules/ngx_http_upstream_keepalive_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_upstream_keepalive_module.c	2012-12-26 09:29:37 UTC (rev 4998)
+++ trunk/src/http/modules/ngx_http_upstream_keepalive_module.c	2012-12-26 14:46:06 UTC (rev 4999)
@@ -502,6 +502,10 @@
     kcf = ngx_http_conf_upstream_srv_conf(uscf,
                                           ngx_http_upstream_keepalive_module);
 
+    if (kcf->original_init_upstream) {
+        return "is duplicate";
+    }
+
     kcf->original_init_upstream = uscf->peer.init_upstream
                                   ? uscf->peer.init_upstream
                                   : ngx_http_upstream_init_round_robin;


From ru at nginx.com  Thu Dec 27 21:35:47 2012
From: ru at nginx.com (ru at nginx.com)
Date: Thu, 27 Dec 2012 21:35:47 +0000
Subject: [nginx] svn commit:  r5000 - trunk/src/http/modules
Message-ID: <20121227213547.8B7343F9C48@mail.nginx.com>

Author: ru
Date: 2012-12-27 21:35:47 +0000 (Thu, 27 Dec 2012)
New Revision: 5000
URL: http://trac.nginx.org/nginx/changeset/5000/nginx

Log:
Geo: improved code readability.


Modified:
   trunk/src/http/modules/ngx_http_geo_module.c

Modified: trunk/src/http/modules/ngx_http_geo_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_geo_module.c	2012-12-26 14:46:06 UTC (rev 4999)
+++ trunk/src/http/modules/ngx_http_geo_module.c	2012-12-27 21:35:47 UTC (rev 5000)
@@ -1016,6 +1016,7 @@
 ngx_http_geo_cidr(ngx_conf_t *cf, ngx_http_geo_conf_ctx_t *ctx,
     ngx_str_t *value)
 {
+    char        *rv;
     ngx_int_t    rc, del;
     ngx_str_t   *net;
     ngx_cidr_t   cidr;
@@ -1041,20 +1042,20 @@
         cidr.u.in.addr = 0;
         cidr.u.in.mask = 0;
 
-        if (ngx_http_geo_cidr_add(cf, ctx, &cidr, &value[1], &value[0])
-            != NGX_CONF_OK)
-        {
-            return NGX_CONF_ERROR;
+        rv = ngx_http_geo_cidr_add(cf, ctx, &cidr, &value[1], &value[0]);
+
+        if (rv != NGX_CONF_OK) {
+            return rv;
         }
 
 #if (NGX_HAVE_INET6)
         cidr.family = AF_INET6;
         ngx_memzero(&cidr.u.in6, sizeof(ngx_in6_cidr_t));
 
-        if (ngx_http_geo_cidr_add(cf, ctx, &cidr, &value[1], &value[0])
-            != NGX_CONF_OK)
-        {
-            return NGX_CONF_ERROR;
+        rv = ngx_http_geo_cidr_add(cf, ctx, &cidr, &value[1], &value[0]);
+
+        if (rv != NGX_CONF_OK) {
+            return rv;
         }
 #endif
 
@@ -1113,7 +1114,6 @@
     ngx_cidr_t *cidr, ngx_str_t *value, ngx_str_t *net)
 {
     ngx_int_t                   rc;
-    ngx_uint_t                  i;
     ngx_http_variable_value_t  *val, *old;
 
     val = ngx_http_geo_value(cf, ctx, value);
@@ -1126,76 +1126,83 @@
 
 #if (NGX_HAVE_INET6)
     case AF_INET6:
-        for (i = 2; i; i--) {
-            rc = ngx_radix128tree_insert(ctx->tree6, cidr->u.in6.addr.s6_addr,
-                                         cidr->u.in6.mask.s6_addr,
-                                         (uintptr_t) val);
+        rc = ngx_radix128tree_insert(ctx->tree6, cidr->u.in6.addr.s6_addr,
+                                     cidr->u.in6.mask.s6_addr,
+                                     (uintptr_t) val);
 
-            if (rc == NGX_OK) {
-                return NGX_CONF_OK;
-            }
+        if (rc == NGX_OK) {
+            return NGX_CONF_OK;
+        }
 
-            if (rc == NGX_ERROR) {
-                return NGX_CONF_ERROR;
-            }
+        if (rc == NGX_ERROR) {
+            return NGX_CONF_ERROR;
+        }
 
-            /* rc == NGX_BUSY */
+        /* rc == NGX_BUSY */
 
-            old = (ngx_http_variable_value_t *)
-                       ngx_radix128tree_find(ctx->tree6,
-                                             cidr->u.in6.addr.s6_addr);
+        old = (ngx_http_variable_value_t *)
+                   ngx_radix128tree_find(ctx->tree6,
+                                         cidr->u.in6.addr.s6_addr);
 
-            ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
-                  "duplicate network \"%V\", value: \"%v\", old value: \"%v\"",
-                  net, val, old);
+        ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
+              "duplicate network \"%V\", value: \"%v\", old value: \"%v\"",
+              net, val, old);
 
-            rc = ngx_radix128tree_delete(ctx->tree6,
-                                         cidr->u.in6.addr.s6_addr,
-                                         cidr->u.in6.mask.s6_addr);
+        rc = ngx_radix128tree_delete(ctx->tree6,
+                                     cidr->u.in6.addr.s6_addr,
+                                     cidr->u.in6.mask.s6_addr);
 
-            if (rc == NGX_ERROR) {
-                ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid radix tree");
-                return NGX_CONF_ERROR;
-            }
+        if (rc == NGX_ERROR) {
+            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid radix tree");
+            return NGX_CONF_ERROR;
         }
 
+        rc = ngx_radix128tree_insert(ctx->tree6, cidr->u.in6.addr.s6_addr,
+                                     cidr->u.in6.mask.s6_addr,
+                                     (uintptr_t) val);
+
         break;
 #endif
 
     default: /* AF_INET */
-        for (i = 2; i; i--) {
-            rc = ngx_radix32tree_insert(ctx->tree, cidr->u.in.addr,
-                                        cidr->u.in.mask, (uintptr_t) val);
+        rc = ngx_radix32tree_insert(ctx->tree, cidr->u.in.addr,
+                                    cidr->u.in.mask, (uintptr_t) val);
 
-            if (rc == NGX_OK) {
-                return NGX_CONF_OK;
-            }
+        if (rc == NGX_OK) {
+            return NGX_CONF_OK;
+        }
 
-            if (rc == NGX_ERROR) {
-                return NGX_CONF_ERROR;
-            }
+        if (rc == NGX_ERROR) {
+            return NGX_CONF_ERROR;
+        }
 
-            /* rc == NGX_BUSY */
+        /* rc == NGX_BUSY */
 
-            old = (ngx_http_variable_value_t *)
-                       ngx_radix32tree_find(ctx->tree, cidr->u.in.addr);
+        old = (ngx_http_variable_value_t *)
+                   ngx_radix32tree_find(ctx->tree, cidr->u.in.addr);
 
-            ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
-                  "duplicate network \"%V\", value: \"%v\", old value: \"%v\"",
-                  net, val, old);
+        ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
+              "duplicate network \"%V\", value: \"%v\", old value: \"%v\"",
+              net, val, old);
 
-            rc = ngx_radix32tree_delete(ctx->tree,
-                                        cidr->u.in.addr, cidr->u.in.mask);
+        rc = ngx_radix32tree_delete(ctx->tree,
+                                    cidr->u.in.addr, cidr->u.in.mask);
 
-            if (rc == NGX_ERROR) {
-                ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid radix tree");
-                return NGX_CONF_ERROR;
-            }
+        if (rc == NGX_ERROR) {
+            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid radix tree");
+            return NGX_CONF_ERROR;
         }
 
+        rc = ngx_radix32tree_insert(ctx->tree, cidr->u.in.addr,
+                                    cidr->u.in.mask, (uintptr_t) val);
+
         break;
     }
 
+    if (rc == NGX_OK) {
+        return NGX_CONF_OK;
+    }
+
     return NGX_CONF_ERROR;
 }
 


From agentzh at gmail.com  Fri Dec 28 05:08:10 2012
From: agentzh at gmail.com (agentzh)
Date: Thu, 27 Dec 2012 21:08:10 -0800
Subject: [BUG] ngx_http_sub_filter_module can not handle ambiguous patterns
Message-ID: <CAB4Tn6P2eKVsWD91txBQA6uiN20EVqJSRar+=vFy12MQPxjUQA@mail.gmail.com>

Hello!

I found an issue in the standard ngx_http_sub_filter_module shipped
with the Nginx core while playing with it tonight.

The minimal use case to reproduce this issue is as follows:

    location = /t {
        default_type text/html;
        return 200 ababac;
        sub_filter abac X;
    }

The output of location = /t is "ababac" instead of "abX".
ngx_http_sub_filter_module just fails to find a match when it really
should.

It seems that the parser in this module is just too simple and its
state machine does not take into account the potential ambiguity
involved in the user patterns.

If this limitation is by design, then IMHO it should be mentioned in
the official documentation:

    http://nginx.org/en/docs/http/ngx_http_sub_module.html

Best regards,
-agentzh


From mdounin at mdounin.ru  Fri Dec 28 13:34:43 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 28 Dec 2012 17:34:43 +0400
Subject: [BUG] ngx_http_sub_filter_module can not handle ambiguous patterns
In-Reply-To: <CAB4Tn6P2eKVsWD91txBQA6uiN20EVqJSRar+=vFy12MQPxjUQA@mail.gmail.com>
References: <CAB4Tn6P2eKVsWD91txBQA6uiN20EVqJSRar+=vFy12MQPxjUQA@mail.gmail.com>
Message-ID: <20121228133443.GP40452@mdounin.ru>

Hello!

On Thu, Dec 27, 2012 at 09:08:10PM -0800, agentzh wrote:

> Hello!
> 
> I found an issue in the standard ngx_http_sub_filter_module shipped
> with the Nginx core while playing with it tonight.
> 
> The minimal use case to reproduce this issue is as follows:
> 
>     location = /t {
>         default_type text/html;
>         return 200 ababac;
>         sub_filter abac X;
>     }
> 
> The output of location = /t is "ababac" instead of "abX".
> ngx_http_sub_filter_module just fails to find a match when it really
> should.
> 
> It seems that the parser in this module is just too simple and its
> state machine does not take into account the potential ambiguity
> involved in the user patterns.
> 
> If this limitation is by design, then IMHO it should be mentioned in
> the official documentation:
> 
>     http://nginx.org/en/docs/http/ngx_http_sub_module.html

It's a bug, and it's expected to be fixed soon.

-- 
Maxim Dounin
http://nginx.com/support.html


From rm at oft-media.ru  Fri Dec 28 16:23:43 2012
From: rm at oft-media.ru (Ruslan Mullakhmetov)
Date: Fri, 28 Dec 2012 20:23:43 +0400
Subject: nginx  custom module thread local ctx
Message-ID: <F0E55AD4-682A-4918-A1DC-FEA73003DCF6@oft-media.ru>

Hi!

I develop some module for nginx and need to use c++ source files in it.

I found the way to compile it (extern c + compiler recognize .cpp files), but know i need to specify some flags. 

Could you suggest a method to tell build system which compiler (CXX) and which flags to use in module/config file?

I saw in Makefile that it use $(CC) despite of there is unused definition of $(CPP) which is actually preprocessor and no CXXFLAFS /CPPFLAGS.

I will greatly appreciate your help.

I could ?orrect Makefile myself but it get's irrating with each reconfigure.

--
BR, Ruslan Mullakhmetov



From agentzh at gmail.com  Sat Dec 29 06:04:01 2012
From: agentzh at gmail.com (agentzh)
Date: Fri, 28 Dec 2012 22:04:01 -0800
Subject: [BUG] ngx_http_sub_filter_module can not handle ambiguous patterns
In-Reply-To: <20121228133443.GP40452@mdounin.ru>
References: <CAB4Tn6P2eKVsWD91txBQA6uiN20EVqJSRar+=vFy12MQPxjUQA@mail.gmail.com>
 <20121228133443.GP40452@mdounin.ru>
Message-ID: <CAB4Tn6O+Va6dmZHSUGAX3imQeJpWSHh7xDhpc8OxWSxZpBK+YA@mail.gmail.com>

Hello!

On Fri, Dec 28, 2012 at 5:34 AM, Maxim Dounin <mdounin at mdounin.ru> wrote:
>
> It's a bug, and it's expected to be fixed soon.
>

Thank you :)

BTW, I've also noticed that this sub filter module cannot handle
single-byte user patterns either:

    location = /t {
       default_type text/html;
        return 200 abc;
        sub_filter a X;
    }

Accessing /t yields "abc" instead of "Xbc" (tested with Nginx 1.2.6).

The state machine in the ngx_http_sub_parse function seems to assume
that the user pattern (ctx->match) is at least 2 bytes long, which
looks incorrect to me :)

Best regards,
-agentzh


From niq at apache.org  Sat Dec 29 19:47:21 2012
From: niq at apache.org (Nick Kew)
Date: Sat, 29 Dec 2012 19:47:21 +0000
Subject: nginx regex module screws up pcre
Message-ID: <615DA901-78B1-4727-AB10-ABE5FB2CB3B3@apache.org>

I've just spent a long debug session figuring this out!

nginx's regex module sets pcre_malloc and pcre_free globally:

void
ngx_regex_init(void)
{
    pcre_malloc = ngx_regex_malloc;
    pcre_free = ngx_regex_free;
}

So any other module or library using pcre inherits nginx's
functions.  But those functions rely on an nginx pool, which
gets set and unset in ngx_regex_module_init and in
ngx_regex_compile.  So anything else calling pcre_compile
gets a NULL pool, and allocation fails.

Another module may be able to work around that by using
ngx_regex_compile in place of pcre_compile.  For a third-
party library that's not an option: we're basically screwed!

If nginx is setting pcre_malloc and pcre_free, it should set
them to functions that will work at any time a third-party
might call them.

The outline of a scheme might be for ngx_regex_init to set
ngx_pcre_pool to the cycle pool, and thereafter each pool
creation/cleanup event gets a callback that may push/pop
the new/old pool.   I guess that still needs care to limit it to
connection and request pools, and not automatically trigger
on inappropriate special-purpose pools.

Does that make sense?

-- 
Nick Kew


From mat999 at gmail.com  Mon Dec 31 12:03:17 2012
From: mat999 at gmail.com (SplitIce)
Date: Mon, 31 Dec 2012 23:03:17 +1100
Subject: Please add HTML support for http_xslt_module (there's an nginx
 fork which has it already)
In-Reply-To: <CABLXxgHLVdWDNaFu_ZjL6C12vQQ-XNPhu9xR246=Ha8+Xj6HUA@mail.gmail.com>
References: <CABLXxgF=z8X2JEJ3CP8rZ-UN_Hh1TM6R5PFQNE4tV2Q0NFZ6ig@mail.gmail.com>
 <CAOycyLTNZRjpeCytNUY5aShDYwr3bWjBc_Mzs4t99OJVo1pZ-w@mail.gmail.com>
 <CABLXxgHLVdWDNaFu_ZjL6C12vQQ-XNPhu9xR246=Ha8+Xj6HUA@mail.gmail.com>
Message-ID: <CALFfGYZ1_MFMRhfTGf8C-_i9mL3XxEfP3-6TuvZyG4CF973K=Q@mail.gmail.com>

Just thought I would say that the files from bitbucket work on the latest
1.3.x branch without any major modification.

Just copy in the modified module file and make the modifications to the
other files to match the changes.


On Sun, Mar 11, 2012 at 2:45 PM, Peter Halasz <list at pengo.org> wrote:

> Hi Laurence Rowe,
>
> Thanks for the update! I'm using your newly patched version without
> any problems. I've just set it up in our buildout as a drop in
> replacement for the old version:
>
> [nginx]
> recipe = zc.recipe.cmmi
> # url =
> http://html-xslt.googlecode.com/files/nginx-0.7.67-html-xslt-4.tar.gz
> # old nginx
> url =
> https://bitbucket.org/lrowe/nginx-xslt-html-parser/get/7145bd8cc1e2.tar.gz
>
> and left everything else the same and it's working well on our staging
> server right now.
>
> We have a pretty straight forward setup (no SSI and our encoding is
> utf8), so I guess those cases may still need testing, but it's working
> well for me.
>
> Thanks again,
>
> Peter Halasz
>
> P.S. I've started a rough Wikipedia page on Diazo. Hopefully it
> doesn't have too many mistakes:
> http://en.wikipedia.org/wiki/Diazo_%28software%29
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20121231/8e09535c/attachment.html>

From ru at nginx.com  Mon Dec 31 22:08:19 2012
From: ru at nginx.com (ru at nginx.com)
Date: Mon, 31 Dec 2012 22:08:19 +0000
Subject: [nginx] svn commit:  r5001 - in trunk/docs: man text
Message-ID: <20121231220819.E1DC23F9F8A@mail.nginx.com>

Author: ru
Date: 2012-12-31 22:08:19 +0000 (Mon, 31 Dec 2012)
New Revision: 5001
URL: http://trac.nginx.org/nginx/changeset/5001/nginx

Log:
Year 2013.


Modified:
   trunk/docs/man/nginx.8
   trunk/docs/text/LICENSE

Modified: trunk/docs/man/nginx.8
===================================================================
--- trunk/docs/man/nginx.8	2012-12-27 21:35:47 UTC (rev 5000)
+++ trunk/docs/man/nginx.8	2012-12-31 22:08:19 UTC (rev 5001)
@@ -1,6 +1,6 @@
 .\"
-.\" Copyright (c) 2010 Sergey A. Osokin
-.\" Copyright (c) 2011,2012 Nginx, Inc.
+.\" Copyright (C) 2010 Sergey A. Osokin
+.\" Copyright (C) Nginx, Inc.
 .\" All rights reserved.
 .\"
 .\" Redistribution and use in source and binary forms, with or without

Modified: trunk/docs/text/LICENSE
===================================================================
--- trunk/docs/text/LICENSE	2012-12-27 21:35:47 UTC (rev 5000)
+++ trunk/docs/text/LICENSE	2012-12-31 22:08:19 UTC (rev 5001)
@@ -1,6 +1,6 @@
 /* 
- * Copyright (C) 2002-2012 Igor Sysoev
- * Copyright (C) 2011,2012 Nginx, Inc.
+ * Copyright (C) 2002-2013 Igor Sysoev
+ * Copyright (C) 2011-2013 Nginx, Inc.
  * All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without