From goelvivek2011 at gmail.com  Thu Mar  1 10:38:04 2012
From: goelvivek2011 at gmail.com (vivek goel)
Date: Thu, 1 Mar 2012 16:08:04 +0530
Subject: How to automatically restart worker process on crash ?
Message-ID: <CAB+eYNUhcb3wFPO_C32WJkkPw7TdbpiAgEYXFRBSesTwM+GNUg@mail.gmail.com>

Is it possible to automatically restart worker process on crash ?
I am writing an nginx module. Question is if somehow it got crashed there I
want to automatically start worker process.

regards
Vivek Goel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120301/d5eb5bcf/attachment.html>

From mdounin at mdounin.ru  Thu Mar  1 10:54:06 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 1 Mar 2012 14:54:06 +0400
Subject: How to automatically restart worker process on crash ?
In-Reply-To: <CAB+eYNUhcb3wFPO_C32WJkkPw7TdbpiAgEYXFRBSesTwM+GNUg@mail.gmail.com>
References: <CAB+eYNUhcb3wFPO_C32WJkkPw7TdbpiAgEYXFRBSesTwM+GNUg@mail.gmail.com>
Message-ID: <20120301105406.GY67687@mdounin.ru>

Hello!

On Thu, Mar 01, 2012 at 04:08:04PM +0530, vivek goel wrote:

> Is it possible to automatically restart worker process on crash ?
> I am writing an nginx module. Question is if somehow it got crashed there I
> want to automatically start worker process.

Worker processes are automatically restarted by a master process 
if they crash.

Maxim Dounin


From lucas.brasilino at gmail.com  Thu Mar  1 17:22:27 2012
From: lucas.brasilino at gmail.com (Lucas Brasilino)
Date: Thu, 1 Mar 2012 14:22:27 -0300
Subject: Filter module callback not been called
Message-ID: <CAF19W4UJZ8Dj-ePoEa0uxUtAoN3--yecxdqyNGG9rX3u6s7z1Q@mail.gmail.com>

Hi!

I gonna write a filter module so I'm starting to do some experiments.
I did a simple handler module that can be seen at [1] and is working fine.

I'm facing a little problem. It should be some detail I can't see right
now. My filter module callback is not been called.

The code is here: http://pastebin.com/wUB5H0A3

I registered the filter to the 'filters chain' at
'ngx_http_static_header_init()'.

ngx_http_static_header_init() is called, but ngx_http_static_header_filter() is
not.

Where is my mistake ? :)

thanks in advance
Lucas Brasilino


[1] https://github.com/lucasbrasilino/ngx_http_simple_example


From wandenberg at gmail.com  Thu Mar  1 18:04:15 2012
From: wandenberg at gmail.com (Wandenberg Peixoto)
Date: Thu, 1 Mar 2012 15:04:15 -0300
Subject: Filter module callback not been called
In-Reply-To: <CAF19W4UJZ8Dj-ePoEa0uxUtAoN3--yecxdqyNGG9rX3u6s7z1Q@mail.gmail.com>
References: <CAF19W4UJZ8Dj-ePoEa0uxUtAoN3--yecxdqyNGG9rX3u6s7z1Q@mail.gmail.com>
Message-ID: <CAFXmt0XfyHFPSBx7404RSjC++aGYPrQ3QZW8AUTdyEz57idHVw@mail.gmail.com>

I think the problem is in config file.

Try to set this
HTTP_AUX_FILTER_MODULES="$HTTP_AUX_FILTER_MODULES
ngx_http_simple_example_module "

instead of this

HTTP_MODULES="$HTTP_MODULES ngx_http_simple_example_module"

To enable your module as a filter module

Regards

On Thu, Mar 1, 2012 at 2:22 PM, Lucas Brasilino
<lucas.brasilino at gmail.com>wrote:

> Hi!
>
> I gonna write a filter module so I'm starting to do some experiments.
> I did a simple handler module that can be seen at [1] and is working fine.
>
> I'm facing a little problem. It should be some detail I can't see right
> now. My filter module callback is not been called.
>
> The code is here: http://pastebin.com/wUB5H0A3
>
> I registered the filter to the 'filters chain' at
> 'ngx_http_static_header_init()'.
>
> ngx_http_static_header_init() is called, but
> ngx_http_static_header_filter() is
> not.
>
> Where is my mistake ? :)
>
> thanks in advance
> Lucas Brasilino
>
>
> [1] https://github.com/lucasbrasilino/ngx_http_simple_example
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120301/ba62ca88/attachment.html>

From lucas.brasilino at gmail.com  Thu Mar  1 18:42:13 2012
From: lucas.brasilino at gmail.com (Lucas Brasilino)
Date: Thu, 1 Mar 2012 15:42:13 -0300
Subject: Filter module callback not been called
In-Reply-To: <CAFXmt0XfyHFPSBx7404RSjC++aGYPrQ3QZW8AUTdyEz57idHVw@mail.gmail.com>
References: <CAF19W4UJZ8Dj-ePoEa0uxUtAoN3--yecxdqyNGG9rX3u6s7z1Q@mail.gmail.com>
	<CAFXmt0XfyHFPSBx7404RSjC++aGYPrQ3QZW8AUTdyEz57idHVw@mail.gmail.com>
Message-ID: <CAF19W4WptYe2rsUoog5_0LX0c1y3Tb1dGTiF9os-jAOANVKpEw@mail.gmail.com>

Hi!

> Try to set this
> HTTP_AUX_FILTER_MODULES="$HTTP_AUX_FILTER_MODULES
> ngx_http_simple_example_module "
>
> instead of this
>
> HTTP_MODULES="$HTTP_MODULES ngx_http_simple_example_module"
>
> To enable your module as a filter module

The ngx_http_simple_example_module works right :)

The problem is at ngx_http_static_header_module, which code I've
pasted at pastebin.

But, anyway, you are right. I've changed config from my filter module and now it
is been called.

Thanks a lot!
regards
Lucas Brasilino


From mdounin at mdounin.ru  Fri Mar  2 18:52:36 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 2 Mar 2012 22:52:36 +0400
Subject: [PATCH] add xslt_stylesheet_param directive
In-Reply-To: <CALb=60a2Dsr3GT8x_paOa35W8v3M+W9PR0LfbpH7NaXik7spZA@mail.gmail.com>
References: <CALb=60Zx-n0fFn=DVLDuyrJavgDHvX8Ej0H2xGQj3VQAEVJDaw@mail.gmail.com>
	<20120209231221.GR67687@mdounin.ru>
	<CALb=60a2Dsr3GT8x_paOa35W8v3M+W9PR0LfbpH7NaXik7spZA@mail.gmail.com>
Message-ID: <20120302185236.GT67687@mdounin.ru>

Hello!

On Sun, Feb 26, 2012 at 04:15:33PM +0100, SamB wrote:

> Hi,
> 
>   sorry for late response,  had no spare time for this. Here is new patch,
> please review it.
>   I've modified the string_param-enabled - it seems to better fit my
> usage...

Please check the attached patch.  It cleans up some style issues 
and slightly changes parameter processing to be more uniform with 
parameters from xslt_stylesheet directives.

There is also a tiny patch included which fixes long-standing 
off-by-one bug in parsing parameters from xslt_stylesheet 
directives.

Tests are here:
http://mdounin.ru/hg/nginx-tests/rev/6bac00bba8d4

Maxim Dounin
-------------- next part --------------
# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1330697803 -14400
# Node ID 4df0e70d82dd2555b2aa076197947f744c4aea44
# Parent  88e257b4bd3d61e30d1421ab6f2e417a8baedb96
Fixed off-by-one in xslt parameter parsing.

The problem was introduced in 0.7.44 (r2589) during conversion to complex
values.  Previously string.len included space for terminating NUL, but
with complex values it doesn't.

diff --git a/src/http/modules/ngx_http_xslt_filter_module.c b/src/http/modules/ngx_http_xslt_filter_module.c
--- a/src/http/modules/ngx_http_xslt_filter_module.c
+++ b/src/http/modules/ngx_http_xslt_filter_module.c
@@ -585,7 +585,7 @@ ngx_http_xslt_params(ngx_http_request_t 
                        "xslt filter param: \"%s\"", string.data);
 
         p = string.data;
-        last = string.data + string.len - 1;
+        last = string.data + string.len;
 
         while (p && *p) {
 
# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1330707634 -14400
# Node ID ceb911a15613cf81f6a3669c8d39aef596ad36e7
# Parent  4df0e70d82dd2555b2aa076197947f744c4aea44
Added xslt_param and xslt_string_param directives.

Based on patch by Samuel Behan.

diff --git a/src/http/modules/ngx_http_xslt_filter_module.c b/src/http/modules/ngx_http_xslt_filter_module.c
--- a/src/http/modules/ngx_http_xslt_filter_module.c
+++ b/src/http/modules/ngx_http_xslt_filter_module.c
@@ -14,6 +14,7 @@
 #include <libxslt/xslt.h>
 #include <libxslt/xsltInternals.h>
 #include <libxslt/transform.h>
+#include <libxslt/variables.h>
 #include <libxslt/xsltutils.h>
 
 #if (NGX_HAVE_EXSLT)
@@ -27,38 +28,47 @@
 
 
 typedef struct {
-    u_char              *name;
-    void                *data;
+    u_char                    *name;
+    void                      *data;
 } ngx_http_xslt_file_t;
 
 
 typedef struct {
-    ngx_array_t          dtd_files;    /* ngx_http_xslt_file_t */
-    ngx_array_t          sheet_files;  /* ngx_http_xslt_file_t */
+    ngx_array_t                dtd_files;    /* ngx_http_xslt_file_t */
+    ngx_array_t                sheet_files;  /* ngx_http_xslt_file_t */
 } ngx_http_xslt_filter_main_conf_t;
 
 
 typedef struct {
-    xsltStylesheetPtr    stylesheet;
-    ngx_array_t          params;       /* ngx_http_complex_value_t */
+    u_char                    *name;
+    ngx_http_complex_value_t   value;
+    ngx_uint_t                 quote;        /* unsigned  quote:1; */
+} ngx_http_xslt_param_t;
+
+
+typedef struct {
+    xsltStylesheetPtr          stylesheet;
+    ngx_array_t                params;       /* ngx_http_xslt_param_t */
 } ngx_http_xslt_sheet_t;
 
 
 typedef struct {
-    xmlDtdPtr            dtd;
-    ngx_array_t          sheets;       /* ngx_http_xslt_sheet_t */
-    ngx_hash_t           types;
-    ngx_array_t         *types_keys;
+    xmlDtdPtr                  dtd;
+    ngx_array_t                sheets;       /* ngx_http_xslt_sheet_t */
+    ngx_hash_t                 types;
+    ngx_array_t               *types_keys;
+    ngx_array_t               *params;       /* ngx_http_xslt_param_t */
 } ngx_http_xslt_filter_loc_conf_t;
 
 
 typedef struct {
-    xmlDocPtr            doc;
-    xmlParserCtxtPtr     ctxt;
-    ngx_http_request_t  *request;
-    ngx_array_t          params;
+    xmlDocPtr                  doc;
+    xmlParserCtxtPtr           ctxt;
+    xsltTransformContextPtr    transform;
+    ngx_http_request_t        *request;
+    ngx_array_t                params;
 
-    ngx_uint_t           done;         /* unsigned  done:1; */
+    ngx_uint_t                 done;         /* unsigned  done:1; */
 } ngx_http_xslt_filter_ctx_t;
 
 
@@ -76,7 +86,7 @@ static void ngx_cdecl ngx_http_xslt_sax_
 static ngx_buf_t *ngx_http_xslt_apply_stylesheet(ngx_http_request_t *r,
     ngx_http_xslt_filter_ctx_t *ctx);
 static ngx_int_t ngx_http_xslt_params(ngx_http_request_t *r,
-    ngx_http_xslt_filter_ctx_t *ctx, ngx_array_t *params);
+    ngx_http_xslt_filter_ctx_t *ctx, ngx_array_t *params, ngx_uint_t final);
 static u_char *ngx_http_xslt_content_type(xsltStylesheetPtr s);
 static u_char *ngx_http_xslt_encoding(xsltStylesheetPtr s);
 static void ngx_http_xslt_cleanup(void *data);
@@ -85,6 +95,8 @@ static char *ngx_http_xslt_entities(ngx_
     void *conf);
 static char *ngx_http_xslt_stylesheet(ngx_conf_t *cf, ngx_command_t *cmd,
     void *conf);
+static char *ngx_http_xslt_param(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf);
 static void ngx_http_xslt_cleanup_dtd(void *data);
 static void ngx_http_xslt_cleanup_stylesheet(void *data);
 static void *ngx_http_xslt_filter_create_main_conf(ngx_conf_t *cf);
@@ -117,6 +129,20 @@ static ngx_command_t  ngx_http_xslt_filt
       0,
       NULL },
 
+    { ngx_string("xslt_param"),
+      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
+      ngx_http_xslt_param,
+      NGX_HTTP_LOC_CONF_OFFSET,
+      0,
+      NULL },
+
+    { ngx_string("xslt_string_param"),
+      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
+      ngx_http_xslt_param,
+      NGX_HTTP_LOC_CONF_OFFSET,
+      0,
+      (void *) 1 },
+
     { ngx_string("xslt_types"),
       NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_1MORE,
       ngx_http_types_slot,
@@ -469,13 +495,31 @@ ngx_http_xslt_apply_stylesheet(ngx_http_
 
     for (i = 0; i < conf->sheets.nelts; i++) {
 
-        if (ngx_http_xslt_params(r, ctx, &sheet[i].params) != NGX_OK) {
+        ctx->transform = xsltNewTransformContext(sheet[i].stylesheet, doc);
+        if (ctx->transform == NULL) {
             xmlFreeDoc(doc);
             return NULL;
         }
 
-        res = xsltApplyStylesheet(sheet[i].stylesheet, doc, ctx->params.elts);
+        if (conf->params
+            && ngx_http_xslt_params(r, ctx, conf->params, 0) != NGX_OK)
+        {
+            xsltFreeTransformContext(ctx->transform);
+            xmlFreeDoc(doc);
+            return NULL;
+        }
 
+        if (ngx_http_xslt_params(r, ctx, &sheet[i].params, 1) != NGX_OK) {
+            xsltFreeTransformContext(ctx->transform);
+            xmlFreeDoc(doc);
+            return NULL;
+        }
+
+        res = xsltApplyStylesheetUser(sheet[i].stylesheet, doc,
+                                      ctx->params.elts, NULL, NULL,
+                                      ctx->transform);
+
+        xsltFreeTransformContext(ctx->transform);
         xmlFreeDoc(doc);
 
         if (res == NULL) {
@@ -565,25 +609,66 @@ ngx_http_xslt_apply_stylesheet(ngx_http_
 
 static ngx_int_t
 ngx_http_xslt_params(ngx_http_request_t *r, ngx_http_xslt_filter_ctx_t *ctx,
-    ngx_array_t *params)
+    ngx_array_t *params, ngx_uint_t final)
 {
-    u_char                    *p, *last, *value, *dst, *src, **s;
-    size_t                     len;
-    ngx_uint_t                 i;
-    ngx_str_t                  string;
-    ngx_http_complex_value_t  *param;
+    u_char                 *p, *last, *value, *dst, *src, **s;
+    size_t                  len;
+    ngx_uint_t              i;
+    ngx_str_t               string;
+    ngx_http_xslt_param_t  *param;
 
     param = params->elts;
 
     for (i = 0; i < params->nelts; i++) {
 
-        if (ngx_http_complex_value(r, &param[i], &string) != NGX_OK) {
+        if (ngx_http_complex_value(r, &param[i].value, &string) != NGX_OK) {
             return NGX_ERROR;
         }
 
         ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                        "xslt filter param: \"%s\"", string.data);
 
+        if (param[i].name) {
+
+            ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
+                           "xslt filter param name: \"%s\"", param[i].name);
+
+            if (param[i].quote) {
+                if (xsltQuoteOneUserParam(ctx->transform, param[i].name,
+                                          string.data)
+                    != 0)
+                {
+                    ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                                "xsltQuoteOneUserParam(\"%s\", \"%s\") failed",
+                                param[i].name, string.data);
+                    return NGX_ERROR;
+                }
+
+                continue;
+            }
+
+            s = ngx_array_push(&ctx->params);
+            if (s == NULL) {
+                return NGX_ERROR;
+            }
+
+            *s = param[i].name;
+
+            s = ngx_array_push(&ctx->params);
+            if (s == NULL) {
+                return NGX_ERROR;
+            }
+
+            *s = string.data;
+
+            continue;
+        }
+
+        /*
+         * parse param1=value1:param2=value2 syntax as used by parameters
+         * specified in xslt_stylesheet directives
+         */
+
         p = string.data;
         last = string.data + string.len;
 
@@ -641,13 +726,15 @@ ngx_http_xslt_params(ngx_http_request_t 
         }
     }
 
-    s = ngx_array_push(&ctx->params);
-    if (s == NULL) {
-        return NGX_ERROR;
+    if (final) {
+        s = ngx_array_push(&ctx->params);
+        if (s == NULL) {
+            return NGX_ERROR;
+        }
+
+        *s = NULL;
     }
 
-    *s = NULL;
-
     return NGX_OK;
 }
 
@@ -768,7 +855,7 @@ ngx_http_xslt_stylesheet(ngx_conf_t *cf,
     ngx_pool_cleanup_t                *cln;
     ngx_http_xslt_file_t              *file;
     ngx_http_xslt_sheet_t             *sheet;
-    ngx_http_complex_value_t          *param;
+    ngx_http_xslt_param_t             *param;
     ngx_http_compile_complex_value_t   ccv;
     ngx_http_xslt_filter_main_conf_t  *xmcf;
 
@@ -837,7 +924,7 @@ found:
     }
 
     if (ngx_array_init(&sheet->params, cf->pool, n - 2,
-                       sizeof(ngx_http_complex_value_t))
+                       sizeof(ngx_http_xslt_param_t))
         != NGX_OK)
     {
         return NGX_CONF_ERROR;
@@ -850,11 +937,12 @@ found:
             return NGX_CONF_ERROR;
         }
 
+        ngx_memzero(param, sizeof(ngx_http_xslt_param_t));
         ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
 
         ccv.cf = cf;
         ccv.value = &value[i];
-        ccv.complex_value = param;
+        ccv.complex_value = &param->value;
         ccv.zero = 1;
 
         if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
@@ -866,6 +954,48 @@ found:
 }
 
 
+static char *
+ngx_http_xslt_param(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
+{
+    ngx_http_xslt_filter_loc_conf_t  *xlcf = conf;
+
+    ngx_http_xslt_param_t            *param;
+    ngx_http_compile_complex_value_t  ccv;
+    ngx_str_t                        *value;
+
+    value = cf->args->elts;
+
+    if (xlcf->params == NULL) {
+        xlcf->params = ngx_array_create(cf->pool, 2,
+                                        sizeof(ngx_http_xslt_param_t));
+        if (xlcf->params == NULL) {
+            return NGX_CONF_ERROR;
+        }
+    }
+
+    param = ngx_array_push(xlcf->params);
+    if (param == NULL) {
+        return NGX_CONF_ERROR;
+    }
+
+    param->name = value[1].data;
+    param->quote = (cmd->post == NULL) ? 0 : 1;
+
+    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
+
+    ccv.cf = cf;
+    ccv.value = &value[2];
+    ccv.complex_value = &param->value;
+    ccv.zero = 1;
+
+    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
+        return NGX_CONF_ERROR;
+    }
+
+    return NGX_CONF_OK;
+}
+
+
 static void
 ngx_http_xslt_cleanup_dtd(void *data)
 {
@@ -925,6 +1055,7 @@ ngx_http_xslt_filter_create_conf(ngx_con
      *     conf->sheets = { NULL };
      *     conf->types = { NULL };
      *     conf->types_keys = NULL;
+     *     conf->params = NULL;
      */
 
     return conf;
@@ -945,6 +1076,10 @@ ngx_http_xslt_filter_merge_conf(ngx_conf
         conf->sheets = prev->sheets;
     }
 
+    if (conf->params == NULL) {
+        conf->params = prev->params;
+    }
+
     if (ngx_http_merge_types(cf, &conf->types_keys, &conf->types,
                              &prev->types_keys, &prev->types,
                              ngx_http_xslt_default_types)

From mdounin at mdounin.ru  Mon Mar  5 11:36:22 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 11:36:22 +0000
Subject: [nginx] svn commit: r4505 - in branches/stable-1.0/src: core
	http/modules/perl
Message-ID: <20120305113622.29FE73F9CFB@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 11:36:21 +0000 (Mon, 05 Mar 2012)
New Revision: 4505

Log:
Version bump.


Modified:
   branches/stable-1.0/src/core/nginx.h
   branches/stable-1.0/src/http/modules/perl/nginx.pm

Modified: branches/stable-1.0/src/core/nginx.h
===================================================================
--- branches/stable-1.0/src/core/nginx.h	2012-02-29 13:45:39 UTC (rev 4504)
+++ branches/stable-1.0/src/core/nginx.h	2012-03-05 11:36:21 UTC (rev 4505)
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1000012
-#define NGINX_VERSION      "1.0.12"
+#define nginx_version      1000013
+#define NGINX_VERSION      "1.0.13"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #define NGINX_VAR          "NGINX"

Modified: branches/stable-1.0/src/http/modules/perl/nginx.pm
===================================================================
--- branches/stable-1.0/src/http/modules/perl/nginx.pm	2012-02-29 13:45:39 UTC (rev 4504)
+++ branches/stable-1.0/src/http/modules/perl/nginx.pm	2012-03-05 11:36:21 UTC (rev 4505)
@@ -48,7 +48,7 @@
     HTTP_INSUFFICIENT_STORAGE
 );
 
-our $VERSION = '1.0.12';
+our $VERSION = '1.0.13';
 
 require XSLoader;
 XSLoader::load('nginx', $VERSION);


From mdounin at mdounin.ru  Mon Mar  5 11:47:25 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 11:47:25 +0000
Subject: [nginx] svn commit: r4506 - in branches/stable-1.0: . src/http/modules
Message-ID: <20120305114726.471E23F9C1F@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 11:47:25 +0000 (Mon, 05 Mar 2012)
New Revision: 4506

Log:
Merge of r4423: fixed proxy_redirect off inheritance.


Modified:
   branches/stable-1.0/
   branches/stable-1.0/src/http/modules/ngx_http_proxy_module.c


Property changes on: branches/stable-1.0
___________________________________________________________________
Modified: svn:mergeinfo
   - /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423

Modified: branches/stable-1.0/src/http/modules/ngx_http_proxy_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_proxy_module.c	2012-03-05 11:36:21 UTC (rev 4505)
+++ branches/stable-1.0/src/http/modules/ngx_http_proxy_module.c	2012-03-05 11:47:25 UTC (rev 4506)
@@ -2496,6 +2496,8 @@
         return NGX_CONF_OK;
     }
 
+    plcf->redirect = 1;
+
     value = cf->args->elts;
 
     if (cf->args->nelts == 2) {


From mdounin at mdounin.ru  Mon Mar  5 12:10:10 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 12:10:10 +0000
Subject: [nginx] svn commit: r4507 - in branches/stable-1.0: . auto src/os/unix
Message-ID: <20120305121010.701A23F9C1F@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 12:10:09 +0000 (Mon, 05 Mar 2012)
New Revision: 4507

Log:
Merge of r4460: ngx_ncpu detection for most *nix platforms.

This inaccurate detection by using sysconf(_SC_NPROCESSORS_ONLN) can improve
usage of the mutex lock optimization on multicore systems.


Modified:
   branches/stable-1.0/
   branches/stable-1.0/auto/unix
   branches/stable-1.0/src/os/unix/ngx_posix_init.c


Property changes on: branches/stable-1.0
___________________________________________________________________
Modified: svn:mergeinfo
   - /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460

Modified: branches/stable-1.0/auto/unix
===================================================================
--- branches/stable-1.0/auto/unix	2012-03-05 11:47:25 UTC (rev 4506)
+++ branches/stable-1.0/auto/unix	2012-03-05 12:10:09 UTC (rev 4507)
@@ -707,3 +707,13 @@
 ngx_feature_libs=
 ngx_feature_test="struct dirent  dir; dir.d_type = DT_REG"
 . auto/feature
+
+
+ngx_feature="sysconf(_SC_NPROCESSORS_ONLN)"
+ngx_feature_name="NGX_HAVE_SC_NPROCESSORS_ONLN"
+ngx_feature_run=no
+ngx_feature_incs=
+ngx_feature_path=
+ngx_feature_libs=
+ngx_feature_test="sysconf(_SC_NPROCESSORS_ONLN)"
+. auto/feature

Modified: branches/stable-1.0/src/os/unix/ngx_posix_init.c
===================================================================
--- branches/stable-1.0/src/os/unix/ngx_posix_init.c	2012-03-05 11:47:25 UTC (rev 4506)
+++ branches/stable-1.0/src/os/unix/ngx_posix_init.c	2012-03-05 12:10:09 UTC (rev 4507)
@@ -47,7 +47,13 @@
 
     for (n = ngx_pagesize; n >>= 1; ngx_pagesize_shift++) { /* void */ }
 
+#if (NGX_HAVE_SC_NPROCESSORS_ONLN)
     if (ngx_ncpu == 0) {
+        ngx_ncpu = sysconf(_SC_NPROCESSORS_ONLN);
+    }
+#endif
+
+    if (ngx_ncpu < 1) {
         ngx_ncpu = 1;
     }
 


From mdounin at mdounin.ru  Mon Mar  5 12:15:03 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 12:15:03 +0000
Subject: [nginx] svn commit:  r4508 - in branches/stable-1.0: . src/http
Message-ID: <20120305121503.6ADE43F9F16@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 12:15:02 +0000 (Mon, 05 Mar 2012)
New Revision: 4508

Log:
Merge of r4461: upstream: fixed "too big header" check.

If header filter postponed processing of a header by returning NGX_AGAIN 
and not moved u->buffer->pos, previous check incorrectly assumed there
is additional space and did another recv() with zero-size buffer.  This
resulted in "upstream prematurely closed connection" error instead
of correct "upstream sent too big header" one.

Patch by Feibo Li.


Modified:
   branches/stable-1.0/
   branches/stable-1.0/src/http/ngx_http_upstream.c


Property changes on: branches/stable-1.0
___________________________________________________________________
Modified: svn:mergeinfo
   - /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461

Modified: branches/stable-1.0/src/http/ngx_http_upstream.c
===================================================================
--- branches/stable-1.0/src/http/ngx_http_upstream.c	2012-03-05 12:10:09 UTC (rev 4507)
+++ branches/stable-1.0/src/http/ngx_http_upstream.c	2012-03-05 12:15:02 UTC (rev 4508)
@@ -1561,7 +1561,7 @@
 
         if (rc == NGX_AGAIN) {
 
-            if (u->buffer.pos == u->buffer.end) {
+            if (u->buffer.last == u->buffer.end) {
                 ngx_log_error(NGX_LOG_ERR, c->log, 0,
                               "upstream sent too big header");
 


From mdounin at mdounin.ru  Mon Mar  5 12:24:19 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 12:24:19 +0000
Subject: [nginx] svn commit: r4509 - in branches/stable-1.0: . src/http/modules
Message-ID: <20120305122419.270A23F9CB1@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 12:24:18 +0000 (Mon, 05 Mar 2012)
New Revision: 4509

Log:
Merge of r4468:

Removed r->cache/r->cached dependencies in range filter.  This is
a layering violation, use correct offset calculations instead.


Modified:
   branches/stable-1.0/
   branches/stable-1.0/src/http/modules/ngx_http_range_filter_module.c


Property changes on: branches/stable-1.0
___________________________________________________________________
Modified: svn:mergeinfo
   - /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468

Modified: branches/stable-1.0/src/http/modules/ngx_http_range_filter_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_range_filter_module.c	2012-03-05 12:15:02 UTC (rev 4508)
+++ branches/stable-1.0/src/http/modules/ngx_http_range_filter_module.c	2012-03-05 12:24:18 UTC (rev 4509)
@@ -595,16 +595,9 @@
     buf = in->buf;
 
     if (!buf->last_buf) {
+        start = ctx->offset;
+        last = ctx->offset + ngx_buf_size(buf);
 
-        if (buf->in_file) {
-            start = buf->file_pos + ctx->offset;
-            last = buf->file_last + ctx->offset;
-
-        } else {
-            start = buf->pos - buf->start + ctx->offset;
-            last = buf->last - buf->start + ctx->offset;
-        }
-
         range = ctx->ranges.elts;
         for (i = 0; i < ctx->ranges.nelts; i++) {
             if (start > range[i].start || last < range[i].end) {
@@ -716,7 +709,6 @@
 ngx_http_range_multipart_body(ngx_http_request_t *r,
     ngx_http_range_filter_ctx_t *ctx, ngx_chain_t *in)
 {
-    off_t              body_start;
     ngx_buf_t         *b, *buf;
     ngx_uint_t         i;
     ngx_chain_t       *out, *hcl, *rcl, *dcl, **ll;
@@ -726,12 +718,6 @@
     buf = in->buf;
     range = ctx->ranges.elts;
 
-#if (NGX_HTTP_CACHE)
-    body_start = r->cached ? r->cache->body_start : 0;
-#else
-    body_start = 0;
-#endif
-
     for (i = 0; i < ctx->ranges.nelts; i++) {
 
         /*
@@ -792,13 +778,13 @@
         b->file = buf->file;
 
         if (buf->in_file) {
-            b->file_pos = body_start + range[i].start;
-            b->file_last = body_start + range[i].end;
+            b->file_pos = buf->file_pos + range[i].start;
+            b->file_last = buf->file_pos + range[i].end;
         }
 
         if (ngx_buf_in_memory(buf)) {
-            b->pos = buf->start + (size_t) range[i].start;
-            b->last = buf->start + (size_t) range[i].end;
+            b->pos = buf->pos + (size_t) range[i].start;
+            b->last = buf->pos + (size_t) range[i].end;
         }
 
         dcl = ngx_alloc_chain_link(r->pool);


From mdounin at mdounin.ru  Mon Mar  5 12:33:07 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 12:33:07 +0000
Subject: [nginx] svn commit:  r4510 - in branches/stable-1.0: . src/http
Message-ID: <20120305123307.27C9C3F9D93@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 12:33:06 +0000 (Mon, 05 Mar 2012)
New Revision: 4510

Log:
Merge of r4470:

Fix for proxy_store leaving temporary files for subrequests.

Temporary files might not be removed if the "proxy_store" or "fastcgi_store"
directives were used for subrequests (e.g. ssi includes) and client closed
connection prematurely.

Non-active subrequests are finalized out of the control of the upstream
module when client closes a connection.  As a result, the code to remove
unfinished temporary files in ngx_http_upstream_process_request() wasn't
executed.

Fix is to move relevant code into ngx_http_upstream_finalize_request() which
is called in all cases, either directly or via the cleanup handler.


Modified:
   branches/stable-1.0/
   branches/stable-1.0/src/http/ngx_http_upstream.c


Property changes on: branches/stable-1.0
___________________________________________________________________
Modified: svn:mergeinfo
   - /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470

Modified: branches/stable-1.0/src/http/ngx_http_upstream.c
===================================================================
--- branches/stable-1.0/src/http/ngx_http_upstream.c	2012-03-05 12:24:18 UTC (rev 4509)
+++ branches/stable-1.0/src/http/ngx_http_upstream.c	2012-03-05 12:33:06 UTC (rev 4510)
@@ -2616,7 +2616,6 @@
 static void
 ngx_http_upstream_process_request(ngx_http_request_t *r)
 {
-    ngx_uint_t            del;
     ngx_temp_file_t      *tf;
     ngx_event_pipe_t     *p;
     ngx_http_upstream_t  *u;
@@ -2628,32 +2627,18 @@
 
         if (u->store) {
 
-            del = p->upstream_error;
-
-            tf = u->pipe->temp_file;
-
             if (p->upstream_eof || p->upstream_done) {
 
+                tf = u->pipe->temp_file;
+
                 if (u->headers_in.status_n == NGX_HTTP_OK
                     && (u->headers_in.content_length_n == -1
                         || (u->headers_in.content_length_n == tf->offset)))
                 {
                     ngx_http_upstream_store(r, u);
-
-                } else {
-                    del = 1;
+                    u->store = 0;
                 }
             }
-
-            if (del && tf->file.fd != NGX_INVALID_FILE) {
-
-                if (ngx_delete_file(tf->file.name.data) == NGX_FILE_ERROR) {
-
-                    ngx_log_error(NGX_LOG_CRIT, r->connection->log, ngx_errno,
-                                  ngx_delete_file_n " \"%s\" failed",
-                                  u->pipe->temp_file->file.name.data);
-                }
-            }
         }
 
 #if (NGX_HTTP_CACHE)
@@ -2994,6 +2979,18 @@
                        u->pipe->temp_file->file.fd);
     }
 
+    if (u->store && u->pipe && u->pipe->temp_file
+        && u->pipe->temp_file->file.fd != NGX_INVALID_FILE)
+    {
+        if (ngx_delete_file(u->pipe->temp_file->file.name.data)
+            == NGX_FILE_ERROR)
+        {
+            ngx_log_error(NGX_LOG_CRIT, r->connection->log, ngx_errno,
+                          ngx_delete_file_n " \"%s\" failed",
+                          u->pipe->temp_file->file.name.data);
+        }
+    }
+
 #if (NGX_HTTP_CACHE)
 
     if (r->cache) {


From mdounin at mdounin.ru  Mon Mar  5 12:36:51 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 12:36:51 +0000
Subject: [nginx] svn commit:  r4511 - in branches/stable-1.0: . src/http
Message-ID: <20120305123651.809743F9F3E@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 12:36:51 +0000 (Mon, 05 Mar 2012)
New Revision: 4511

Log:
Merge of r4471:

Variables: honor no_cacheable for not_found variables.

Variables with the "not_found" flag set follow the same rules as ones with
the "valid" flag set.  Make sure ngx_http_get_flushed_variable() will flush
non-cacheable variables with the "not_found" flag set.

This fixes at least one known problem with $args not available in a subrequest
(with args) when there were no args in the main request and $args variable was
queried in the main request (reported by Laurence Rowe aka elro on irc).

Also this eliminates unneeded call to ngx_http_get_indexed_variable() in
cacheable case (as it will return cached value anyway).


Modified:
   branches/stable-1.0/
   branches/stable-1.0/src/http/ngx_http_variables.c


Property changes on: branches/stable-1.0
___________________________________________________________________
Modified: svn:mergeinfo
   - /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471

Modified: branches/stable-1.0/src/http/ngx_http_variables.c
===================================================================
--- branches/stable-1.0/src/http/ngx_http_variables.c	2012-03-05 12:33:06 UTC (rev 4510)
+++ branches/stable-1.0/src/http/ngx_http_variables.c	2012-03-05 12:36:51 UTC (rev 4511)
@@ -428,7 +428,7 @@
 
     v = &r->variables[index];
 
-    if (v->valid) {
+    if (v->valid || v->not_found) {
         if (!v->no_cacheable) {
             return v;
         }


From mdounin at mdounin.ru  Mon Mar  5 12:49:32 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 12:49:32 +0000
Subject: [nginx] svn commit:  r4512 - in branches/stable-1.0: . src/http
Message-ID: <20120305124932.B2F323F9CB1@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 12:49:32 +0000 (Mon, 05 Mar 2012)
New Revision: 4512

Log:
Merge of r4473:

Core: protection from cycles with named locations and post_action.

Now redirects to named locations are counted against normal uri changes
limit, and post_action respects this limit as well.  As a result at least
the following (bad) configurations no longer trigger infinite cycles:

1. Post action which recursively triggers post action:

    location / {
        post_action /index.html;
    }

2. Post action pointing to nonexistent named location:

    location / {
        post_action @nonexistent;
    }

3. Recursive error page for 500 (Internal Server Error) pointing to
   a nonexistent named location:

    location / {
        recursive_error_pages on;
        error_page 500 @nonexistent;
        return 500;
    }


Modified:
   branches/stable-1.0/
   branches/stable-1.0/src/http/ngx_http_core_module.c
   branches/stable-1.0/src/http/ngx_http_request.c


Property changes on: branches/stable-1.0
___________________________________________________________________
Modified: svn:mergeinfo
   - /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473

Modified: branches/stable-1.0/src/http/ngx_http_core_module.c
===================================================================
--- branches/stable-1.0/src/http/ngx_http_core_module.c	2012-03-05 12:36:51 UTC (rev 4511)
+++ branches/stable-1.0/src/http/ngx_http_core_module.c	2012-03-05 12:49:32 UTC (rev 4512)
@@ -2524,7 +2524,17 @@
     ngx_http_core_main_conf_t   *cmcf;
 
     r->main->count++;
+    r->uri_changes--;
 
+    if (r->uri_changes == 0) {
+        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                      "rewrite or internal redirection cycle "
+                      "while redirect to named location \"%V\"", name);
+
+        ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
+        return NGX_DONE;
+    }
+
     cscf = ngx_http_get_module_srv_conf(r, ngx_http_core_module);
 
     if (cscf->named_locations) {

Modified: branches/stable-1.0/src/http/ngx_http_request.c
===================================================================
--- branches/stable-1.0/src/http/ngx_http_request.c	2012-03-05 12:36:51 UTC (rev 4511)
+++ branches/stable-1.0/src/http/ngx_http_request.c	2012-03-05 12:49:32 UTC (rev 4512)
@@ -2898,6 +2898,10 @@
         return NGX_DECLINED;
     }
 
+    if (r->post_action && r->uri_changes == 0) {
+        return NGX_DECLINED;
+    }
+
     ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                    "post action: \"%V\"", &clcf->post_action);
 


From mdounin at mdounin.ru  Mon Mar  5 12:58:10 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 12:58:10 +0000
Subject: [nginx] svn commit: r4513 - in branches/stable-1.0: . auto/cc
	auto/lib/perl src/http/modules/perl
Message-ID: <20120305125810.BA32E3F9DAC@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 12:58:10 +0000 (Mon, 05 Mar 2012)
New Revision: 4513

Log:
Merge of r4474, r4493: configure/build fixes.

*) Fixed build with embedded perl and --with-openssl.

*) Configure: moved icc detection before gcc.  New versions of icc
   confuse auto/cc/name due to introduced handling of a "icc -v".


Modified:
   branches/stable-1.0/
   branches/stable-1.0/auto/cc/name
   branches/stable-1.0/auto/lib/perl/make
   branches/stable-1.0/src/http/modules/perl/Makefile.PL


Property changes on: branches/stable-1.0
___________________________________________________________________
Modified: svn:mergeinfo
   - /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4493

Modified: branches/stable-1.0/auto/cc/name
===================================================================
--- branches/stable-1.0/auto/cc/name	2012-03-05 12:49:32 UTC (rev 4512)
+++ branches/stable-1.0/auto/cc/name	2012-03-05 12:58:10 UTC (rev 4513)
@@ -64,16 +64,16 @@
     echo " + using Borland C++ compiler"
 
 else
+if `$CC -V 2>&1 | grep '^Intel(R) C' >/dev/null 2>&1`; then
+    NGX_CC_NAME=icc
+    echo " + using Intel C++ compiler"
+
+else
 if `$CC -v 2>&1 | grep 'gcc version' >/dev/null 2>&1`; then
     NGX_CC_NAME=gcc
     echo " + using GNU C compiler"
 
 else
-if `$CC -V 2>&1 | grep '^Intel(R) C' >/dev/null 2>&1`; then
-    NGX_CC_NAME=icc
-    echo " + using Intel C++ compiler"
-
-else
 if `$CC -V 2>&1 | grep 'Sun C' >/dev/null 2>&1`; then
     NGX_CC_NAME=sunc
     echo " + using Sun C compiler"

Modified: branches/stable-1.0/auto/lib/perl/make
===================================================================
--- branches/stable-1.0/auto/lib/perl/make	2012-03-05 12:49:32 UTC (rev 4512)
+++ branches/stable-1.0/auto/lib/perl/make	2012-03-05 12:58:10 UTC (rev 4513)
@@ -28,6 +28,7 @@
 		&& NGX_PM_CFLAGS="\$(NGX_PM_CFLAGS) -g $NGX_CC_OPT"	\
 			NGX_PCRE=$PCRE					\
 			NGX_OBJS=$NGX_OBJS				\
+			NGX_OPENSSL=$OPENSSL				\
 		$NGX_PERL Makefile.PL					\
 			LIB=$NGX_PERL_MODULES				\
 			INSTALLSITEMAN3DIR=$NGX_PERL_MODULES_MAN

Modified: branches/stable-1.0/src/http/modules/perl/Makefile.PL
===================================================================
--- branches/stable-1.0/src/http/modules/perl/Makefile.PL	2012-03-05 12:49:32 UTC (rev 4512)
+++ branches/stable-1.0/src/http/modules/perl/Makefile.PL	2012-03-05 12:58:10 UTC (rev 4513)
@@ -25,7 +25,11 @@
                          "-I ../../../../../$ENV{NGX_OBJS} " .
                          ($ENV{NGX_PCRE} =~ /^(YES|NO)/ ? "" :
                              ($ENV{NGX_PCRE} =~ m#^/# ? "-I $ENV{NGX_PCRE} " :
-                                  "-I ../../../../../$ENV{NGX_PCRE} ")),
+                                  "-I ../../../../../$ENV{NGX_PCRE} ")) .
+                         ($ENV{NGX_OPENSSL} =~ /^(YES|NO)/ ? "" :
+                             ($ENV{NGX_OPENSSL} =~ m#^/# ?
+                                  "-I $ENV{NGX_OPENSSL}/.openssl/include " :
+                      "-I ../../../../../$ENV{NGX_OPENSSL}/.openssl/include ")),
 
     depend => {
         'nginx.c'     =>


From mdounin at mdounin.ru  Mon Mar  5 13:03:40 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 13:03:40 +0000
Subject: [nginx] svn commit: r4514 - in branches/stable-1.0: . src/http
	src/http/modules src/http/modules/perl
Message-ID: <20120305130340.D9DB83FA356@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 13:03:39 +0000 (Mon, 05 Mar 2012)
New Revision: 4514

Log:
Merge of r4491, r4492:

*) Renamed constants and fixed off-by-one error in "msie_padding on"
   handling.

*) Added support for the 307 Temporary Redirect.


Modified:
   branches/stable-1.0/
   branches/stable-1.0/src/http/modules/ngx_http_headers_filter_module.c
   branches/stable-1.0/src/http/modules/perl/nginx.pm
   branches/stable-1.0/src/http/ngx_http_core_module.c
   branches/stable-1.0/src/http/ngx_http_header_filter_module.c
   branches/stable-1.0/src/http/ngx_http_request.h
   branches/stable-1.0/src/http/ngx_http_special_response.c


Property changes on: branches/stable-1.0
___________________________________________________________________
Modified: svn:mergeinfo
   - /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4493
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4491-4493

Modified: branches/stable-1.0/src/http/modules/ngx_http_headers_filter_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_headers_filter_module.c	2012-03-05 12:58:10 UTC (rev 4513)
+++ branches/stable-1.0/src/http/modules/ngx_http_headers_filter_module.c	2012-03-05 13:03:39 UTC (rev 4514)
@@ -149,7 +149,9 @@
             && r->headers_out.status != NGX_HTTP_PARTIAL_CONTENT
             && r->headers_out.status != NGX_HTTP_MOVED_PERMANENTLY
             && r->headers_out.status != NGX_HTTP_MOVED_TEMPORARILY
-            && r->headers_out.status != NGX_HTTP_NOT_MODIFIED))
+            && r->headers_out.status != NGX_HTTP_SEE_OTHER
+            && r->headers_out.status != NGX_HTTP_NOT_MODIFIED
+            && r->headers_out.status != NGX_HTTP_TEMPORARY_REDIRECT))
     {
         return ngx_http_next_header_filter(r);
     }

Modified: branches/stable-1.0/src/http/modules/perl/nginx.pm
===================================================================
--- branches/stable-1.0/src/http/modules/perl/nginx.pm	2012-03-05 12:58:10 UTC (rev 4513)
+++ branches/stable-1.0/src/http/modules/perl/nginx.pm	2012-03-05 13:03:39 UTC (rev 4514)
@@ -21,7 +21,9 @@
     HTTP_MOVED_PERMANENTLY
     HTTP_MOVED_TEMPORARILY
     HTTP_REDIRECT
+    HTTP_SEE_OTHER
     HTTP_NOT_MODIFIED
+    HTTP_TEMPORARY_REDIRECT
 
     HTTP_BAD_REQUEST
     HTTP_UNAUTHORIZED
@@ -67,7 +69,9 @@
 use constant HTTP_MOVED_PERMANENTLY         => 301;
 use constant HTTP_MOVED_TEMPORARILY         => 302;
 use constant HTTP_REDIRECT                  => 302;
+use constant HTTP_SEE_OTHER                 => 303;
 use constant HTTP_NOT_MODIFIED              => 304;
+use constant HTTP_TEMPORARY_REDIRECT        => 307;
 
 use constant HTTP_BAD_REQUEST               => 400;
 use constant HTTP_UNAUTHORIZED              => 401;

Modified: branches/stable-1.0/src/http/ngx_http_core_module.c
===================================================================
--- branches/stable-1.0/src/http/ngx_http_core_module.c	2012-03-05 12:58:10 UTC (rev 4513)
+++ branches/stable-1.0/src/http/ngx_http_core_module.c	2012-03-05 13:03:39 UTC (rev 4514)
@@ -1798,8 +1798,11 @@
         return NGX_HTTP_INTERNAL_SERVER_ERROR;
     }
 
-    if (status >= NGX_HTTP_MOVED_PERMANENTLY && status <= NGX_HTTP_SEE_OTHER) {
-
+    if (status == NGX_HTTP_MOVED_PERMANENTLY
+        || status == NGX_HTTP_MOVED_TEMPORARILY
+        || status == NGX_HTTP_SEE_OTHER
+        || status == NGX_HTTP_TEMPORARY_REDIRECT)
+    {
         ngx_http_clear_location(r);
 
         r->headers_out.location = ngx_list_push(&r->headers_out.headers);

Modified: branches/stable-1.0/src/http/ngx_http_header_filter_module.c
===================================================================
--- branches/stable-1.0/src/http/ngx_http_header_filter_module.c	2012-03-05 12:58:10 UTC (rev 4513)
+++ branches/stable-1.0/src/http/ngx_http_header_filter_module.c	2012-03-05 13:03:39 UTC (rev 4514)
@@ -71,12 +71,11 @@
     ngx_string("302 Moved Temporarily"),
     ngx_string("303 See Other"),
     ngx_string("304 Not Modified"),
+    ngx_null_string,  /* "305 Use Proxy" */
+    ngx_null_string,  /* "306 unused" */
+    ngx_string("307 Temporary Redirect"),
 
-    /* ngx_null_string, */  /* "305 Use Proxy" */
-    /* ngx_null_string, */  /* "306 unused" */
-    /* ngx_null_string, */  /* "307 Temporary Redirect" */
-
-#define NGX_HTTP_LAST_3XX  305
+#define NGX_HTTP_LAST_3XX  308
 #define NGX_HTTP_OFF_4XX   (NGX_HTTP_LAST_3XX - 301 + NGX_HTTP_OFF_3XX)
 
     ngx_string("400 Bad Request"),

Modified: branches/stable-1.0/src/http/ngx_http_request.h
===================================================================
--- branches/stable-1.0/src/http/ngx_http_request.h	2012-03-05 12:58:10 UTC (rev 4513)
+++ branches/stable-1.0/src/http/ngx_http_request.h	2012-03-05 13:03:39 UTC (rev 4514)
@@ -75,6 +75,7 @@
 #define NGX_HTTP_MOVED_TEMPORARILY         302
 #define NGX_HTTP_SEE_OTHER                 303
 #define NGX_HTTP_NOT_MODIFIED              304
+#define NGX_HTTP_TEMPORARY_REDIRECT        307
 
 #define NGX_HTTP_BAD_REQUEST               400
 #define NGX_HTTP_UNAUTHORIZED              401

Modified: branches/stable-1.0/src/http/ngx_http_special_response.c
===================================================================
--- branches/stable-1.0/src/http/ngx_http_special_response.c	2012-03-05 12:58:10 UTC (rev 4513)
+++ branches/stable-1.0/src/http/ngx_http_special_response.c	2012-03-05 13:03:39 UTC (rev 4514)
@@ -74,6 +74,14 @@
 ;
 
 
+static char ngx_http_error_307_page[] =
+"<html>" CRLF
+"<head><title>307 Temporary Redirect</title></head>" CRLF
+"<body bgcolor=\"white\">" CRLF
+"<center><h1>307 Temporary Redirect</h1></center>" CRLF
+;
+
+
 static char ngx_http_error_400_page[] =
 "<html>" CRLF
 "<head><title>400 Bad Request</title></head>" CRLF
@@ -294,16 +302,20 @@
 
     ngx_null_string,                     /* 201, 204 */
 
-#define NGX_HTTP_LAST_LEVEL_200  202
-#define NGX_HTTP_LEVEL_200       (NGX_HTTP_LAST_LEVEL_200 - 201)
+#define NGX_HTTP_LAST_2XX  202
+#define NGX_HTTP_OFF_3XX   (NGX_HTTP_LAST_2XX - 201)
 
     /* ngx_null_string, */               /* 300 */
     ngx_string(ngx_http_error_301_page),
     ngx_string(ngx_http_error_302_page),
     ngx_string(ngx_http_error_303_page),
+    ngx_null_string,                     /* 304 */
+    ngx_null_string,                     /* 305 */
+    ngx_null_string,                     /* 306 */
+    ngx_string(ngx_http_error_307_page),
 
-#define NGX_HTTP_LAST_LEVEL_300  304
-#define NGX_HTTP_LEVEL_300       (NGX_HTTP_LAST_LEVEL_300 - 301)
+#define NGX_HTTP_LAST_3XX  308
+#define NGX_HTTP_OFF_4XX   (NGX_HTTP_LAST_3XX - 301 + NGX_HTTP_OFF_3XX)
 
     ngx_string(ngx_http_error_400_page),
     ngx_string(ngx_http_error_401_page),
@@ -323,8 +335,8 @@
     ngx_string(ngx_http_error_415_page),
     ngx_string(ngx_http_error_416_page),
 
-#define NGX_HTTP_LAST_LEVEL_400  417
-#define NGX_HTTP_LEVEL_400       (NGX_HTTP_LAST_LEVEL_400 - 400)
+#define NGX_HTTP_LAST_4XX  417
+#define NGX_HTTP_OFF_5XX   (NGX_HTTP_LAST_4XX - 400 + NGX_HTTP_OFF_4XX)
 
     ngx_string(ngx_http_error_494_page), /* 494, request header too large */
     ngx_string(ngx_http_error_495_page), /* 495, https certificate error */
@@ -342,7 +354,7 @@
     ngx_null_string,                     /* 506 */
     ngx_string(ngx_http_error_507_page)
 
-#define NGX_HTTP_LAST_LEVEL_500  508
+#define NGX_HTTP_LAST_5XX  508
 
 };
 
@@ -428,25 +440,22 @@
         err = 0;
 
     } else if (error >= NGX_HTTP_MOVED_PERMANENTLY
-               && error < NGX_HTTP_LAST_LEVEL_300)
+               && error < NGX_HTTP_LAST_3XX)
     {
         /* 3XX */
-        err = error - NGX_HTTP_MOVED_PERMANENTLY + NGX_HTTP_LEVEL_200;
+        err = error - NGX_HTTP_MOVED_PERMANENTLY + NGX_HTTP_OFF_3XX;
 
     } else if (error >= NGX_HTTP_BAD_REQUEST
-               && error < NGX_HTTP_LAST_LEVEL_400)
+               && error < NGX_HTTP_LAST_4XX)
     {
         /* 4XX */
-        err = error - NGX_HTTP_BAD_REQUEST + NGX_HTTP_LEVEL_200
-                                           + NGX_HTTP_LEVEL_300;
+        err = error - NGX_HTTP_BAD_REQUEST + NGX_HTTP_OFF_4XX;
 
     } else if (error >= NGX_HTTP_NGINX_CODES
-               && error < NGX_HTTP_LAST_LEVEL_500)
+               && error < NGX_HTTP_LAST_5XX)
     {
         /* 49X, 5XX */
-        err = error - NGX_HTTP_NGINX_CODES + NGX_HTTP_LEVEL_200
-                                           + NGX_HTTP_LEVEL_300
-                                           + NGX_HTTP_LEVEL_400;
+        err = error - NGX_HTTP_NGINX_CODES + NGX_HTTP_OFF_5XX;
         switch (error) {
             case NGX_HTTP_TO_HTTPS:
             case NGX_HTTPS_CERT_ERROR:
@@ -570,12 +579,11 @@
         return NGX_ERROR;
     }
 
-    if (overwrite >= NGX_HTTP_MOVED_PERMANENTLY
-        && overwrite <= NGX_HTTP_SEE_OTHER)
+    if (overwrite != NGX_HTTP_MOVED_PERMANENTLY
+        && overwrite != NGX_HTTP_MOVED_TEMPORARILY
+        && overwrite != NGX_HTTP_SEE_OTHER
+        && overwrite != NGX_HTTP_TEMPORARY_REDIRECT)
     {
-        r->err_status = overwrite;
-
-    } else {
         r->err_status = NGX_HTTP_MOVED_TEMPORARILY;
     }
 
@@ -595,7 +603,7 @@
 
     return ngx_http_send_special_response(r, clcf, r->err_status
                                                    - NGX_HTTP_MOVED_PERMANENTLY
-                                                   + NGX_HTTP_LEVEL_200);
+                                                   + NGX_HTTP_OFF_3XX);
 }
 
 
@@ -626,7 +634,7 @@
         if (clcf->msie_padding
             && (r->headers_in.msie || r->headers_in.chrome)
             && r->http_version >= NGX_HTTP_VERSION_10
-            && err >= NGX_HTTP_LEVEL_300)
+            && err >= NGX_HTTP_OFF_4XX)
         {
             r->headers_out.content_length_n +=
                                          sizeof(ngx_http_msie_padding) - 1;


From mdounin at mdounin.ru  Mon Mar  5 13:06:30 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 13:06:30 +0000
Subject: [nginx] svn commit:  r4515 - in branches/stable-1.0: . src/core
Message-ID: <20120305130630.3DA443F9DCA@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 13:06:29 +0000 (Mon, 05 Mar 2012)
New Revision: 4515

Log:
Merge of r4497:

Fixed null pointer dereference in resolver (ticket #91).

The cycle->new_log.file may not be set before config parsing finished if
there are no error_log directive defined at global level.  Fix is to
copy it after config parsing.

Patch by Roman Arutyunyan.


Modified:
   branches/stable-1.0/
   branches/stable-1.0/src/core/ngx_resolver.c


Property changes on: branches/stable-1.0
___________________________________________________________________
Modified: svn:mergeinfo
   - /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4491-4493
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4491-4493,4497

Modified: branches/stable-1.0/src/core/ngx_resolver.c
===================================================================
--- branches/stable-1.0/src/core/ngx_resolver.c	2012-03-05 13:03:39 UTC (rev 4514)
+++ branches/stable-1.0/src/core/ngx_resolver.c	2012-03-05 13:06:29 UTC (rev 4515)
@@ -152,11 +152,6 @@
         uc->sockaddr = addr->sockaddr;
         uc->socklen = addr->socklen;
         uc->server = addr->name;
-
-        uc->log = cf->cycle->new_log;
-        uc->log.handler = ngx_resolver_log_error;
-        uc->log.data = uc;
-        uc->log.action = "resolving";
     }
 
     return r;
@@ -830,6 +825,12 @@
     uc = r->udp_connection;
 
     if (uc->connection == NULL) {
+
+        uc->log = *r->log;
+        uc->log.handler = ngx_resolver_log_error;
+        uc->log.data = uc;
+        uc->log.action = "resolving";
+
         if (ngx_udp_connect(uc) != NGX_OK) {
             return NGX_ERROR;
         }


From mdounin at mdounin.ru  Mon Mar  5 13:17:56 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 13:17:56 +0000
Subject: [nginx] svn commit: r4516 - in branches/stable-1.0: . src/core
	src/event src/http src/http/modules
Message-ID: <20120305131757.231313F9CFB@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 13:17:56 +0000 (Mon, 05 Mar 2012)
New Revision: 4516

Log:
Merge of r4498:

Fix of rbtree lookup on hash collisions.

Previous code incorrectly assumed that nodes with identical keys are linked
together.  This might not be true after tree rebalance.

Patch by Lanshun Zhou.


Modified:
   branches/stable-1.0/
   branches/stable-1.0/src/core/ngx_open_file_cache.c
   branches/stable-1.0/src/core/ngx_resolver.c
   branches/stable-1.0/src/event/ngx_event_openssl.c
   branches/stable-1.0/src/http/modules/ngx_http_limit_req_module.c
   branches/stable-1.0/src/http/modules/ngx_http_limit_zone_module.c
   branches/stable-1.0/src/http/ngx_http_file_cache.c


Property changes on: branches/stable-1.0
___________________________________________________________________
Modified: svn:mergeinfo
   - /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4491-4493,4497
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4491-4493,4497-4498

Modified: branches/stable-1.0/src/core/ngx_open_file_cache.c
===================================================================
--- branches/stable-1.0/src/core/ngx_open_file_cache.c	2012-03-05 13:06:29 UTC (rev 4515)
+++ branches/stable-1.0/src/core/ngx_open_file_cache.c	2012-03-05 13:17:56 UTC (rev 4516)
@@ -837,20 +837,15 @@
 
         /* hash == node->key */
 
-        do {
-            file = (ngx_cached_open_file_t *) node;
+        file = (ngx_cached_open_file_t *) node;
 
-            rc = ngx_strcmp(name->data, file->name);
+        rc = ngx_strcmp(name->data, file->name);
 
-            if (rc == 0) {
-                return file;
-            }
+        if (rc == 0) {
+            return file;
+        }
 
-            node = (rc < 0) ? node->left : node->right;
-
-        } while (node != sentinel && hash == node->key);
-
-        break;
+        node = (rc < 0) ? node->left : node->right;
     }
 
     return NULL;

Modified: branches/stable-1.0/src/core/ngx_resolver.c
===================================================================
--- branches/stable-1.0/src/core/ngx_resolver.c	2012-03-05 13:06:29 UTC (rev 4515)
+++ branches/stable-1.0/src/core/ngx_resolver.c	2012-03-05 13:17:56 UTC (rev 4516)
@@ -1626,20 +1626,15 @@
 
         /* hash == node->key */
 
-        do {
-            rn = (ngx_resolver_node_t *) node;
+        rn = (ngx_resolver_node_t *) node;
 
-            rc = ngx_memn2cmp(name->data, rn->name, name->len, rn->nlen);
+        rc = ngx_memn2cmp(name->data, rn->name, name->len, rn->nlen);
 
-            if (rc == 0) {
-                return rn;
-            }
+        if (rc == 0) {
+            return rn;
+        }
 
-            node = (rc < 0) ? node->left : node->right;
-
-        } while (node != sentinel && hash == node->key);
-
-        break;
+        node = (rc < 0) ? node->left : node->right;
     }
 
     /* not found */

Modified: branches/stable-1.0/src/event/ngx_event_openssl.c
===================================================================
--- branches/stable-1.0/src/event/ngx_event_openssl.c	2012-03-05 13:06:29 UTC (rev 4515)
+++ branches/stable-1.0/src/event/ngx_event_openssl.c	2012-03-05 13:17:56 UTC (rev 4516)
@@ -1801,44 +1801,39 @@
 
         /* hash == node->key */
 
-        do {
-            sess_id = (ngx_ssl_sess_id_t *) node;
+        sess_id = (ngx_ssl_sess_id_t *) node;
 
-            rc = ngx_memn2cmp(id, sess_id->id,
-                              (size_t) len, (size_t) node->data);
-            if (rc == 0) {
+        rc = ngx_memn2cmp(id, sess_id->id, (size_t) len, (size_t) node->data);
 
-                if (sess_id->expire > ngx_time()) {
-                    ngx_memcpy(buf, sess_id->session, sess_id->len);
+        if (rc == 0) {
 
-                    ngx_shmtx_unlock(&shpool->mutex);
+            if (sess_id->expire > ngx_time()) {
+                ngx_memcpy(buf, sess_id->session, sess_id->len);
 
-                    p = buf;
-                    sess = d2i_SSL_SESSION(NULL, &p, sess_id->len);
+                ngx_shmtx_unlock(&shpool->mutex);
 
-                    return sess;
-                }
+                p = buf;
+                sess = d2i_SSL_SESSION(NULL, &p, sess_id->len);
 
-                ngx_queue_remove(&sess_id->queue);
+                return sess;
+            }
 
-                ngx_rbtree_delete(&cache->session_rbtree, node);
+            ngx_queue_remove(&sess_id->queue);
 
-                ngx_slab_free_locked(shpool, sess_id->session);
+            ngx_rbtree_delete(&cache->session_rbtree, node);
+
+            ngx_slab_free_locked(shpool, sess_id->session);
 #if (NGX_PTR_SIZE == 4)
-                ngx_slab_free_locked(shpool, sess_id->id);
+            ngx_slab_free_locked(shpool, sess_id->id);
 #endif
-                ngx_slab_free_locked(shpool, sess_id);
+            ngx_slab_free_locked(shpool, sess_id);
 
-                sess = NULL;
+            sess = NULL;
 
-                goto done;
-            }
+            goto done;
+        }
 
-            node = (rc < 0) ? node->left : node->right;
-
-        } while (node != sentinel && hash == node->key);
-
-        break;
+        node = (rc < 0) ? node->left : node->right;
     }
 
 done:
@@ -1908,31 +1903,26 @@
 
         /* hash == node->key */
 
-        do {
-            sess_id = (ngx_ssl_sess_id_t *) node;
+        sess_id = (ngx_ssl_sess_id_t *) node;
 
-            rc = ngx_memn2cmp(id, sess_id->id, len, (size_t) node->data);
+        rc = ngx_memn2cmp(id, sess_id->id, len, (size_t) node->data);
 
-            if (rc == 0) {
+        if (rc == 0) {
 
-                ngx_queue_remove(&sess_id->queue);
+            ngx_queue_remove(&sess_id->queue);
 
-                ngx_rbtree_delete(&cache->session_rbtree, node);
+            ngx_rbtree_delete(&cache->session_rbtree, node);
 
-                ngx_slab_free_locked(shpool, sess_id->session);
+            ngx_slab_free_locked(shpool, sess_id->session);
 #if (NGX_PTR_SIZE == 4)
-                ngx_slab_free_locked(shpool, sess_id->id);
+            ngx_slab_free_locked(shpool, sess_id->id);
 #endif
-                ngx_slab_free_locked(shpool, sess_id);
+            ngx_slab_free_locked(shpool, sess_id);
 
-                goto done;
-            }
+            goto done;
+        }
 
-            node = (rc < 0) ? node->left : node->right;
-
-        } while (node != sentinel && hash == node->key);
-
-        break;
+        node = (rc < 0) ? node->left : node->right;
     }
 
 done:

Modified: branches/stable-1.0/src/http/modules/ngx_http_limit_req_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_limit_req_module.c	2012-03-05 13:06:29 UTC (rev 4515)
+++ branches/stable-1.0/src/http/modules/ngx_http_limit_req_module.c	2012-03-05 13:17:56 UTC (rev 4516)
@@ -372,47 +372,42 @@
 
         /* hash == node->key */
 
-        do {
-            lr = (ngx_http_limit_req_node_t *) &node->color;
+        lr = (ngx_http_limit_req_node_t *) &node->color;
 
-            rc = ngx_memn2cmp(data, lr->data, len, (size_t) lr->len);
+        rc = ngx_memn2cmp(data, lr->data, len, (size_t) lr->len);
 
-            if (rc == 0) {
-                ngx_queue_remove(&lr->queue);
-                ngx_queue_insert_head(&ctx->sh->queue, &lr->queue);
+        if (rc == 0) {
+            ngx_queue_remove(&lr->queue);
+            ngx_queue_insert_head(&ctx->sh->queue, &lr->queue);
 
-                tp = ngx_timeofday();
+            tp = ngx_timeofday();
 
-                now = (ngx_msec_t) (tp->sec * 1000 + tp->msec);
-                ms = (ngx_msec_int_t) (now - lr->last);
+            now = (ngx_msec_t) (tp->sec * 1000 + tp->msec);
+            ms = (ngx_msec_int_t) (now - lr->last);
 
-                excess = lr->excess - ctx->rate * ngx_abs(ms) / 1000 + 1000;
+            excess = lr->excess - ctx->rate * ngx_abs(ms) / 1000 + 1000;
 
-                if (excess < 0) {
-                    excess = 0;
-                }
+            if (excess < 0) {
+                excess = 0;
+            }
 
-                *ep = excess;
+            *ep = excess;
 
-                if ((ngx_uint_t) excess > lrcf->burst) {
-                    return NGX_BUSY;
-                }
+            if ((ngx_uint_t) excess > lrcf->burst) {
+                return NGX_BUSY;
+            }
 
-                lr->excess = excess;
-                lr->last = now;
+            lr->excess = excess;
+            lr->last = now;
 
-                if (excess) {
-                    return NGX_AGAIN;
-                }
-
-                return NGX_OK;
+            if (excess) {
+                return NGX_AGAIN;
             }
 
-            node = (rc < 0) ? node->left : node->right;
+            return NGX_OK;
+        }
 
-        } while (node != sentinel && hash == node->key);
-
-        break;
+        node = (rc < 0) ? node->left : node->right;
     }
 
     *ep = 0;

Modified: branches/stable-1.0/src/http/modules/ngx_http_limit_zone_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_limit_zone_module.c	2012-03-05 13:06:29 UTC (rev 4515)
+++ branches/stable-1.0/src/http/modules/ngx_http_limit_zone_module.c	2012-03-05 13:17:56 UTC (rev 4516)
@@ -194,31 +194,26 @@
 
         /* hash == node->key */
 
-        do {
-            lz = (ngx_http_limit_zone_node_t *) &node->color;
+        lz = (ngx_http_limit_zone_node_t *) &node->color;
 
-            rc = ngx_memn2cmp(vv->data, lz->data, len, (size_t) lz->len);
+        rc = ngx_memn2cmp(vv->data, lz->data, len, (size_t) lz->len);
 
-            if (rc == 0) {
-                if ((ngx_uint_t) lz->conn < lzcf->conn) {
-                    lz->conn++;
-                    goto done;
-                }
-
-                ngx_shmtx_unlock(&shpool->mutex);
-
-                ngx_log_error(lzcf->log_level, r->connection->log, 0,
-                              "limiting connections by zone \"%V\"",
-                              &lzcf->shm_zone->shm.name);
-
-                return NGX_HTTP_SERVICE_UNAVAILABLE;
+        if (rc == 0) {
+            if ((ngx_uint_t) lz->conn < lzcf->conn) {
+                lz->conn++;
+                goto done;
             }
 
-            node = (rc < 0) ? node->left : node->right;
+            ngx_shmtx_unlock(&shpool->mutex);
 
-        } while (node != sentinel && hash == node->key);
+            ngx_log_error(lzcf->log_level, r->connection->log, 0,
+                          "limiting connections by zone \"%V\"",
+                          &lzcf->shm_zone->shm.name);
 
-        break;
+            return NGX_HTTP_SERVICE_UNAVAILABLE;
+        }
+
+        node = (rc < 0) ? node->left : node->right;
     }
 
     n = offsetof(ngx_rbtree_node_t, color)

Modified: branches/stable-1.0/src/http/ngx_http_file_cache.c
===================================================================
--- branches/stable-1.0/src/http/ngx_http_file_cache.c	2012-03-05 13:06:29 UTC (rev 4515)
+++ branches/stable-1.0/src/http/ngx_http_file_cache.c	2012-03-05 13:17:56 UTC (rev 4516)
@@ -673,21 +673,16 @@
 
         /* node_key == node->key */
 
-        do {
-            fcn = (ngx_http_file_cache_node_t *) node;
+        fcn = (ngx_http_file_cache_node_t *) node;
 
-            rc = ngx_memcmp(&key[sizeof(ngx_rbtree_key_t)], fcn->key,
-                            NGX_HTTP_CACHE_KEY_LEN - sizeof(ngx_rbtree_key_t));
+        rc = ngx_memcmp(&key[sizeof(ngx_rbtree_key_t)], fcn->key,
+                        NGX_HTTP_CACHE_KEY_LEN - sizeof(ngx_rbtree_key_t));
 
-            if (rc == 0) {
-                return fcn;
-            }
+        if (rc == 0) {
+            return fcn;
+        }
 
-            node = (rc < 0) ? node->left : node->right;
-
-        } while (node != sentinel && node_key == node->key);
-
-        break;
+        node = (rc < 0) ? node->left : node->right;
     }
 
     /* not found */


From mdounin at mdounin.ru  Mon Mar  5 13:20:41 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 13:20:41 +0000
Subject: [nginx] svn commit:  r4517 - in branches/stable-1.0: . src/os/unix
Message-ID: <20120305132041.4930D3F9C1F@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 13:20:40 +0000 (Mon, 05 Mar 2012)
New Revision: 4517

Log:
Merge of r4499: workaround for fs_size on ZFS (ticket #46).

ZFS reports incorrect st_blocks until file settles on disk, and this
may take a while (i.e. just after creation of a file the st_blocks value
is incorrect).  As a workaround we now use st_blocks only if
st_blocks * 512 > st_size, this should fix ZFS problems while still
preserving accuracy for other filesystems.

The problem had appeared in r3900 (1.0.1).


Modified:
   branches/stable-1.0/
   branches/stable-1.0/src/os/unix/ngx_files.h


Property changes on: branches/stable-1.0
___________________________________________________________________
Modified: svn:mergeinfo
   - /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4491-4493,4497-4498
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4491-4493,4497-4499

Modified: branches/stable-1.0/src/os/unix/ngx_files.h
===================================================================
--- branches/stable-1.0/src/os/unix/ngx_files.h	2012-03-05 13:17:56 UTC (rev 4516)
+++ branches/stable-1.0/src/os/unix/ngx_files.h	2012-03-05 13:20:40 UTC (rev 4517)
@@ -159,7 +159,7 @@
 #define ngx_is_exec(sb)          (((sb)->st_mode & S_IXUSR) == S_IXUSR)
 #define ngx_file_access(sb)      ((sb)->st_mode & 0777)
 #define ngx_file_size(sb)        (sb)->st_size
-#define ngx_file_fs_size(sb)     ((sb)->st_blocks * 512)
+#define ngx_file_fs_size(sb)     ngx_max((sb)->st_size, (sb)->st_blocks * 512)
 #define ngx_file_mtime(sb)       (sb)->st_mtime
 #define ngx_file_uniq(sb)        (sb)->st_ino
 
@@ -255,7 +255,8 @@
 
 #define ngx_de_access(dir)       (((dir)->info.st_mode) & 0777)
 #define ngx_de_size(dir)         (dir)->info.st_size
-#define ngx_de_fs_size(dir)      ((dir)->info.st_blocks * 512)
+#define ngx_de_fs_size(dir)                                                  \
+    ngx_max((dir)->info.st_size, (dir)->info.st_blocks * 512)
 #define ngx_de_mtime(dir)        (dir)->info.st_mtime
 
 


From mdounin at mdounin.ru  Mon Mar  5 13:26:40 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 13:26:40 +0000
Subject: [nginx] svn commit: r4518 - in branches/stable-1.0: . src/core
	src/event src/http src/http/modules src/mail src/os/unix src/os/win32
Message-ID: <20120305132641.04DF43F9E2A@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 13:26:40 +0000 (Mon, 05 Mar 2012)
New Revision: 4518

Log:
Merge of r4500: fixed spelling in single-line comments.


Modified:
   branches/stable-1.0/
   branches/stable-1.0/src/core/ngx_times.c
   branches/stable-1.0/src/event/ngx_event.h
   branches/stable-1.0/src/event/ngx_event_openssl.c
   branches/stable-1.0/src/event/ngx_event_pipe.c
   branches/stable-1.0/src/http/modules/ngx_http_autoindex_module.c
   branches/stable-1.0/src/http/modules/ngx_http_index_module.c
   branches/stable-1.0/src/http/modules/ngx_http_limit_req_module.c
   branches/stable-1.0/src/http/modules/ngx_http_limit_zone_module.c
   branches/stable-1.0/src/http/modules/ngx_http_log_module.c
   branches/stable-1.0/src/http/modules/ngx_http_memcached_module.c
   branches/stable-1.0/src/http/modules/ngx_http_proxy_module.c
   branches/stable-1.0/src/http/modules/ngx_http_random_index_module.c
   branches/stable-1.0/src/http/modules/ngx_http_rewrite_module.c
   branches/stable-1.0/src/http/modules/ngx_http_userid_filter_module.c
   branches/stable-1.0/src/http/ngx_http_parse_time.c
   branches/stable-1.0/src/http/ngx_http_upstream.c
   branches/stable-1.0/src/mail/ngx_mail_pop3_handler.c
   branches/stable-1.0/src/os/unix/ngx_darwin_sendfile_chain.c
   branches/stable-1.0/src/os/unix/ngx_freebsd_sendfile_chain.c
   branches/stable-1.0/src/os/unix/ngx_user.c
   branches/stable-1.0/src/os/win32/ngx_service.c


Property changes on: branches/stable-1.0
___________________________________________________________________
Modified: svn:mergeinfo
   - /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4491-4493,4497-4499
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4491-4493,4497-4500

Modified: branches/stable-1.0/src/core/ngx_times.c
===================================================================
--- branches/stable-1.0/src/core/ngx_times.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/core/ngx_times.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -287,7 +287,7 @@
 
     days = n / 86400;
 
-    /* Jaunary 1, 1970 was Thursday */
+    /* January 1, 1970 was Thursday */
 
     wday = (4 + days) % 7;
 

Modified: branches/stable-1.0/src/event/ngx_event.h
===================================================================
--- branches/stable-1.0/src/event/ngx_event.h	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/event/ngx_event.h	2012-03-05 13:26:40 UTC (rev 4518)
@@ -83,7 +83,7 @@
 #endif
 
 #if (NGX_WIN32)
-    /* setsockopt(SO_UPDATE_ACCEPT_CONTEXT) was succesfull */
+    /* setsockopt(SO_UPDATE_ACCEPT_CONTEXT) was successful */
     unsigned         accept_context_updated:1;
 #endif
 

Modified: branches/stable-1.0/src/event/ngx_event_openssl.c
===================================================================
--- branches/stable-1.0/src/event/ngx_event_openssl.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/event/ngx_event_openssl.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -842,7 +842,7 @@
         case NGX_ERROR:
             c->read->error = 1;
 
-            /* fall thruogh */
+            /* fall through */
 
         case NGX_AGAIN:
             return c->ssl->last;

Modified: branches/stable-1.0/src/event/ngx_event_pipe.c
===================================================================
--- branches/stable-1.0/src/event/ngx_event_pipe.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/event/ngx_event_pipe.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -946,7 +946,7 @@
         return NGX_OK;
     }
 
-    /* the first free buf is partialy filled, thus add the free buf after it */
+    /* the first free buf is partially filled, thus add the free buf after it */
 
     cl->next = p->free_raw_bufs->next;
     p->free_raw_bufs->next = cl;

Modified: branches/stable-1.0/src/http/modules/ngx_http_autoindex_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_autoindex_module.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/http/modules/ngx_http_autoindex_module.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -95,8 +95,8 @@
     NULL,                                  /* create server configuration */
     NULL,                                  /* merge server configuration */
 
-    ngx_http_autoindex_create_loc_conf,    /* create location configration */
-    ngx_http_autoindex_merge_loc_conf      /* merge location configration */
+    ngx_http_autoindex_create_loc_conf,    /* create location configuration */
+    ngx_http_autoindex_merge_loc_conf      /* merge location configuration */
 };
 
 

Modified: branches/stable-1.0/src/http/modules/ngx_http_index_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_index_module.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/http/modules/ngx_http_index_module.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -62,8 +62,8 @@
     NULL,                                  /* create server configuration */
     NULL,                                  /* merge server configuration */
 
-    ngx_http_index_create_loc_conf,        /* create location configration */
-    ngx_http_index_merge_loc_conf          /* merge location configration */
+    ngx_http_index_create_loc_conf,        /* create location configuration */
+    ngx_http_index_merge_loc_conf          /* merge location configuration */
 };
 
 

Modified: branches/stable-1.0/src/http/modules/ngx_http_limit_req_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_limit_req_module.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/http/modules/ngx_http_limit_req_module.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -112,8 +112,8 @@
     NULL,                                  /* create server configuration */
     NULL,                                  /* merge server configuration */
 
-    ngx_http_limit_req_create_conf,        /* create location configration */
-    ngx_http_limit_req_merge_conf          /* merge location configration */
+    ngx_http_limit_req_create_conf,        /* create location configuration */
+    ngx_http_limit_req_merge_conf          /* merge location configuration */
 };
 
 

Modified: branches/stable-1.0/src/http/modules/ngx_http_limit_zone_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_limit_zone_module.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/http/modules/ngx_http_limit_zone_module.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -96,8 +96,8 @@
     NULL,                                  /* create server configuration */
     NULL,                                  /* merge server configuration */
 
-    ngx_http_limit_zone_create_conf,       /* create location configration */
-    ngx_http_limit_zone_merge_conf         /* merge location configration */
+    ngx_http_limit_zone_create_conf,       /* create location configuration */
+    ngx_http_limit_zone_merge_conf         /* merge location configuration */
 };
 
 

Modified: branches/stable-1.0/src/http/modules/ngx_http_log_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_log_module.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/http/modules/ngx_http_log_module.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -161,8 +161,8 @@
     NULL,                                  /* create server configuration */
     NULL,                                  /* merge server configuration */
 
-    ngx_http_log_create_loc_conf,          /* create location configration */
-    ngx_http_log_merge_loc_conf            /* merge location configration */
+    ngx_http_log_create_loc_conf,          /* create location configuration */
+    ngx_http_log_merge_loc_conf            /* merge location configuration */
 };
 
 
@@ -375,10 +375,10 @@
 
     if (!r->root_tested) {
 
-        /* test root directory existance */
+        /* test root directory existence */
 
         if (ngx_http_map_uri_to_path(r, &path, &root, 0) == NULL) {
-            /* simulate successfull logging */
+            /* simulate successful logging */
             return len;
         }
 
@@ -399,14 +399,14 @@
             != NGX_OK)
         {
             if (of.err == 0) {
-                /* simulate successfull logging */
+                /* simulate successful logging */
                 return len;
             }
 
             ngx_log_error(NGX_LOG_ERR, r->connection->log, of.err,
                           "testing \"%s\" existence failed", path.data);
 
-            /* simulate successfull logging */
+            /* simulate successful logging */
             return len;
         }
 
@@ -414,7 +414,7 @@
             ngx_log_error(NGX_LOG_ERR, r->connection->log, NGX_ENOTDIR,
                           "testing \"%s\" existence failed", path.data);
 
-            /* simulate successfull logging */
+            /* simulate successful logging */
             return len;
         }
     }
@@ -423,7 +423,7 @@
                             script->values->elts)
         == NULL)
     {
-        /* simulate successfull logging */
+        /* simulate successful logging */
         return len;
     }
 
@@ -447,7 +447,7 @@
     {
         ngx_log_error(NGX_LOG_CRIT, r->connection->log, ngx_errno,
                       "%s \"%s\" failed", of.failed, log.data);
-        /* simulate successfull logging */
+        /* simulate successful logging */
         return len;
     }
 

Modified: branches/stable-1.0/src/http/modules/ngx_http_memcached_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_memcached_module.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/http/modules/ngx_http_memcached_module.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -115,8 +115,8 @@
     NULL,                                  /* create server configuration */
     NULL,                                  /* merge server configuration */
 
-    ngx_http_memcached_create_loc_conf,    /* create location configration */
-    ngx_http_memcached_merge_loc_conf      /* merge location configration */
+    ngx_http_memcached_create_loc_conf,    /* create location configuration */
+    ngx_http_memcached_merge_loc_conf      /* merge location configuration */
 };
 
 

Modified: branches/stable-1.0/src/http/modules/ngx_http_proxy_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_proxy_module.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/http/modules/ngx_http_proxy_module.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -458,8 +458,8 @@
     NULL,                                  /* create server configuration */
     NULL,                                  /* merge server configuration */
 
-    ngx_http_proxy_create_loc_conf,        /* create location configration */
-    ngx_http_proxy_merge_loc_conf          /* merge location configration */
+    ngx_http_proxy_create_loc_conf,        /* create location configuration */
+    ngx_http_proxy_merge_loc_conf          /* merge location configuration */
 };
 
 

Modified: branches/stable-1.0/src/http/modules/ngx_http_random_index_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_random_index_module.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/http/modules/ngx_http_random_index_module.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -49,8 +49,8 @@
     NULL,                                  /* create server configuration */
     NULL,                                  /* merge server configuration */
 
-    ngx_http_random_index_create_loc_conf, /* create location configration */
-    ngx_http_random_index_merge_loc_conf   /* merge location configration */
+    ngx_http_random_index_create_loc_conf, /* create location configuration */
+    ngx_http_random_index_merge_loc_conf   /* merge location configuration */
 };
 
 

Modified: branches/stable-1.0/src/http/modules/ngx_http_rewrite_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_rewrite_module.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/http/modules/ngx_http_rewrite_module.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -112,8 +112,8 @@
     NULL,                                  /* create server configuration */
     NULL,                                  /* merge server configuration */
 
-    ngx_http_rewrite_create_loc_conf,      /* create location configration */
-    ngx_http_rewrite_merge_loc_conf        /* merge location configration */
+    ngx_http_rewrite_create_loc_conf,      /* create location configuration */
+    ngx_http_rewrite_merge_loc_conf        /* merge location configuration */
 };
 
 

Modified: branches/stable-1.0/src/http/modules/ngx_http_userid_filter_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_userid_filter_module.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/http/modules/ngx_http_userid_filter_module.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -166,8 +166,8 @@
     NULL,                                  /* create server configuration */
     NULL,                                  /* merge server configuration */
 
-    ngx_http_userid_create_conf,           /* create location configration */
-    ngx_http_userid_merge_conf             /* merge location configration */
+    ngx_http_userid_create_conf,           /* create location configuration */
+    ngx_http_userid_merge_conf             /* merge location configuration */
 };
 
 

Modified: branches/stable-1.0/src/http/ngx_http_parse_time.c
===================================================================
--- branches/stable-1.0/src/http/ngx_http_parse_time.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/http/ngx_http_parse_time.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -242,7 +242,7 @@
         year -= 1;
     }
 
-    /* Gauss' formula for Grigorian days since March 1, 1 BC */
+    /* Gauss' formula for Gregorian days since March 1, 1 BC */
 
     time = (uint64_t) (
             /* days in years including leap years since March 1, 1 BC */

Modified: branches/stable-1.0/src/http/ngx_http_upstream.c
===================================================================
--- branches/stable-1.0/src/http/ngx_http_upstream.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/http/ngx_http_upstream.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -2263,7 +2263,7 @@
     }
 
     if (ngx_event_flags & NGX_USE_AIO_EVENT) {
-        /* the posted aio operation may currupt a shadow buffer */
+        /* the posted aio operation may corrupt a shadow buffer */
         p->single_buf = 1;
     }
 

Modified: branches/stable-1.0/src/mail/ngx_mail_pop3_handler.c
===================================================================
--- branches/stable-1.0/src/mail/ngx_mail_pop3_handler.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/mail/ngx_mail_pop3_handler.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -218,7 +218,7 @@
 
             break;
 
-        /* suppress warinings */
+        /* suppress warnings */
         case ngx_pop3_passwd:
             break;
 

Modified: branches/stable-1.0/src/os/unix/ngx_darwin_sendfile_chain.c
===================================================================
--- branches/stable-1.0/src/os/unix/ngx_darwin_sendfile_chain.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/os/unix/ngx_darwin_sendfile_chain.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -173,7 +173,7 @@
 
         if (file && header.nelts == 0) {
 
-            /* create the tailer iovec and coalesce the neighbouring bufs */
+            /* create the trailer iovec and coalesce the neighbouring bufs */
 
             prev = NULL;
             iov = NULL;

Modified: branches/stable-1.0/src/os/unix/ngx_freebsd_sendfile_chain.c
===================================================================
--- branches/stable-1.0/src/os/unix/ngx_freebsd_sendfile_chain.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/os/unix/ngx_freebsd_sendfile_chain.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -178,7 +178,7 @@
 
         if (file) {
 
-            /* create the tailer iovec and coalesce the neighbouring bufs */
+            /* create the trailer iovec and coalesce the neighbouring bufs */
 
             prev = NULL;
             iov = NULL;

Modified: branches/stable-1.0/src/os/unix/ngx_user.c
===================================================================
--- branches/stable-1.0/src/os/unix/ngx_user.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/os/unix/ngx_user.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -67,7 +67,7 @@
 
 #if (NGX_THREADS && NGX_NONREENTRANT_CRYPT)
 
-    /* crypt() is a time consuming funtion, so we only try to lock */
+    /* crypt() is a time consuming function, so we only try to lock */
 
     if (ngx_mutex_trylock(ngx_crypt_mutex) != NGX_OK) {
         return NGX_AGAIN;

Modified: branches/stable-1.0/src/os/win32/ngx_service.c
===================================================================
--- branches/stable-1.0/src/os/win32/ngx_service.c	2012-03-05 13:20:40 UTC (rev 4517)
+++ branches/stable-1.0/src/os/win32/ngx_service.c	2012-03-05 13:26:40 UTC (rev 4518)
@@ -20,7 +20,7 @@
 {
     /* primary thread */
 
-    /* StartServiceCtrlDispatcher() shouxpdl be called within 30 seconds */
+    /* StartServiceCtrlDispatcher() should be called within 30 seconds */
 
     if (StartServiceCtrlDispatcher(st) == 0) {
         ngx_log_error(NGX_LOG_EMERG, log, ngx_errno,


From mdounin at mdounin.ru  Mon Mar  5 15:19:49 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 15:19:49 +0000
Subject: [nginx] svn commit:  r4519 - branches/stable-1.0/docs/xml/nginx
Message-ID: <20120305151949.E7F203F9E08@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 15:19:49 +0000 (Mon, 05 Mar 2012)
New Revision: 4519

Log:
nginx-1.0.13-RELEASE


Modified:
   branches/stable-1.0/docs/xml/nginx/changes.xml

Modified: branches/stable-1.0/docs/xml/nginx/changes.xml
===================================================================
--- branches/stable-1.0/docs/xml/nginx/changes.xml	2012-03-05 13:26:40 UTC (rev 4518)
+++ branches/stable-1.0/docs/xml/nginx/changes.xml	2012-03-05 15:19:49 UTC (rev 4519)
@@ -9,6 +9,133 @@
 <title lang="en">nginx changelog</title>
 
 
+<changes ver="1.0.13" date="05.03.2012">
+
+<change type="feature">
+<para lang="ru">
+????????? return ? error_page ?????? ????? ?????????????? ??? ????????
+??????????????? ? ????? 307.
+</para>
+<para lang="en">
+the "return" and "error_page" directives can now be used to return 307
+redirections.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ??????? ???????? ??? ????????? segmentation fault,
+???? ?????????????? ????????? resolver
+? ?? ?????????? ?????? ?? ???? ?????? ????????? error_log.<br/>
+??????? ?????? ?????????.
+</para>
+<para lang="en">
+a segmentation fault might occur in a worker process
+if the "resolver" directive was used
+and there was no "error_log" directive specified at global level.<br/>
+Thanks to Roman Arutyunyan.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+?????? ??????.<br/>
+??????? Lanshun Zhou.
+</para>
+<para lang="en">
+memory leaks.<br/>
+Thanks to Lanshun Zhou.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+nginx ??? ??????????? ???????? ?? ?????? "upstream prematurely closed
+connection" ?????? "upstream sent too big header".<br/>
+??????? Feibo Li.
+</para>
+<para lang="en">
+nginx might log incorrect error "upstream prematurely closed connection"
+instead of correct "upstream sent too big header" one.<br/>
+Thanks to Feibo Li.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ????????????? ZFS ?????? ???? ?? ????? ??? ????????? ???????????;
+?????? ????????? ? 1.0.1.
+</para>
+<para lang="en">
+on ZFS filesystem disk cache size might be calculated incorrectly;
+the bug had appeared in 1.0.1.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+?????????? ?????????? ??????????????? ? ??????????? location'?
+?? ??????????????.
+</para>
+<para lang="en">
+the number of internal redirects to named locations was not limited.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ????????????? ????????? proxy_store ? SSI-????????????
+????????? ????? ????? ?? ?????????.
+</para>
+<para lang="en">
+temporary files might be not removed
+if the "proxy_store" directive was used with SSI includes.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ????????? ??????? ???????????? ?????????? (?????, ??? $args)
+?????????? ?????? ?????? ?????????????? ????????.
+</para>
+<para lang="en">
+in some cases non-cacheable variables (such as the $args variable)
+returned old empty cached value.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+????????? proxy_redirect ????? ????????????? ???????????.
+</para>
+<para lang="en">
+the "proxy_redirect" directives might be inherited incorrectly.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+nginx ?? ????????? ? ??????? ngx_http_perl_module,
+???? ????????????? ???????? --with-openssl.
+</para>
+<para lang="en">
+nginx could not be built with the ngx_http_perl_module
+if the --with-openssl option was used.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+nginx ?? ????????? ???????????? icc 12.1.
+</para>
+<para lang="en">
+nginx could not be built by the icc 12.1 compiler.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.0.12" date="06.02.2012">
 
 <change type="feature">


From mdounin at mdounin.ru  Mon Mar  5 15:20:15 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 15:20:15 +0000
Subject: [nginx] svn commit:  r4520 - in tags: . release-1.0.13
Message-ID: <20120305152016.03B1A3F9C6D@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 15:20:15 +0000 (Mon, 05 Mar 2012)
New Revision: 4520

Log:
release-1.0.13 tag


Added:
   tags/release-1.0.13/


Property changes on: tags/release-1.0.13
___________________________________________________________________
Added: svn:ignore
   + access.log
client_body_temp
fastcgi_temp
proxy_temp
GNUmakefile
Makefile
makefile
nginx
nginx.conf
nginx-*.tar.gz
objs*
tmp

Added: svn:mergeinfo
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4491-4493,4497-4500


From mdounin at mdounin.ru  Mon Mar  5 18:06:15 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 18:06:15 +0000
Subject: [nginx] svn commit:  r4521 - in trunk/src: core http/modules/perl
Message-ID: <20120305180615.EE1EB3F9DB0@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 18:06:15 +0000 (Mon, 05 Mar 2012)
New Revision: 4521

Log:
Version bump.


Modified:
   trunk/src/core/nginx.h
   trunk/src/http/modules/perl/nginx.pm

Modified: trunk/src/core/nginx.h
===================================================================
--- trunk/src/core/nginx.h	2012-03-05 15:20:15 UTC (rev 4520)
+++ trunk/src/core/nginx.h	2012-03-05 18:06:15 UTC (rev 4521)
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1001016
-#define NGINX_VERSION      "1.1.16"
+#define nginx_version      1001017
+#define NGINX_VERSION      "1.1.17"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #define NGINX_VAR          "NGINX"

Modified: trunk/src/http/modules/perl/nginx.pm
===================================================================
--- trunk/src/http/modules/perl/nginx.pm	2012-03-05 15:20:15 UTC (rev 4520)
+++ trunk/src/http/modules/perl/nginx.pm	2012-03-05 18:06:15 UTC (rev 4521)
@@ -50,7 +50,7 @@
     HTTP_INSUFFICIENT_STORAGE
 );
 
-our $VERSION = '1.1.16';
+our $VERSION = '1.1.17';
 
 require XSLoader;
 XSLoader::load('nginx', $VERSION);


From mdounin at mdounin.ru  Mon Mar  5 18:08:23 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 18:08:23 +0000
Subject: [nginx] svn commit:  r4522 - trunk/docs/xml/nginx
Message-ID: <20120305180823.C175F3F9DB0@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 18:08:23 +0000 (Mon, 05 Mar 2012)
New Revision: 4522

Log:
Grammar and wording fixes in CHANGES.


Modified:
   trunk/docs/xml/nginx/changes.xml

Modified: trunk/docs/xml/nginx/changes.xml
===================================================================
--- trunk/docs/xml/nginx/changes.xml	2012-03-05 18:06:15 UTC (rev 4521)
+++ trunk/docs/xml/nginx/changes.xml	2012-03-05 18:08:23 UTC (rev 4522)
@@ -35,7 +35,7 @@
 ??????????????? ? ????? 307.
 </para>
 <para lang="en">
-the "return" and "error_page" directives can be used to return 307
+the "return" and "error_page" directives can now be used to return 307
 redirections.
 </para>
 </change>
@@ -172,7 +172,7 @@
 ?? ??????????????.
 </para>
 <para lang="en">
-internal redirects to named locations were not limited.
+the number of internal redirects to named locations was not limited.
 </para>
 </change>
 
@@ -194,7 +194,7 @@
 </para>
 <para lang="en">
 temporary files might be not removed
-if the "proxy_store" directive were used with SSI includes.
+if the "proxy_store" directive was used with SSI includes.
 </para>
 </change>
 
@@ -270,7 +270,7 @@
 ????????? proxy_redirect ????? ????????????? ???????????.
 </para>
 <para lang="en">
-the "proxy_redirect" directives might not be correctly inherited.
+the "proxy_redirect" directives might be inherited incorrectly.
 </para>
 </change>
 


From mdounin at mdounin.ru  Mon Mar  5 18:09:07 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Mon,  5 Mar 2012 18:09:07 +0000
Subject: [nginx] svn commit: r4523 - in trunk/src: core event http
	http/modules mail os/unix
Message-ID: <20120305180907.4FDC53F9DB0@mail.nginx.com>

Author: mdounin
Date: 2012-03-05 18:09:06 +0000 (Mon, 05 Mar 2012)
New Revision: 4523

Log:
Whitespace fixes.


Modified:
   trunk/src/core/ngx_connection.c
   trunk/src/event/ngx_event_openssl.c
   trunk/src/http/modules/ngx_http_fastcgi_module.c
   trunk/src/http/modules/ngx_http_scgi_module.c
   trunk/src/http/modules/ngx_http_uwsgi_module.c
   trunk/src/http/ngx_http_busy_lock.c
   trunk/src/http/ngx_http_core_module.c
   trunk/src/mail/ngx_mail_core_module.c
   trunk/src/os/unix/ngx_process.c

Modified: trunk/src/core/ngx_connection.c
===================================================================
--- trunk/src/core/ngx_connection.c	2012-03-05 18:08:23 UTC (rev 4522)
+++ trunk/src/core/ngx_connection.c	2012-03-05 18:09:06 UTC (rev 4523)
@@ -514,7 +514,7 @@
             }
         }
 
-#if (NGX_HAVE_KEEPALIVE_TUNABLE) 
+#if (NGX_HAVE_KEEPALIVE_TUNABLE)
 
         if (ls[i].keepidle) {
             if (setsockopt(ls[i].fd, IPPROTO_TCP, TCP_KEEPIDLE,

Modified: trunk/src/event/ngx_event_openssl.c
===================================================================
--- trunk/src/event/ngx_event_openssl.c	2012-03-05 18:08:23 UTC (rev 4522)
+++ trunk/src/event/ngx_event_openssl.c	2012-03-05 18:09:06 UTC (rev 4523)
@@ -478,6 +478,7 @@
     return NGX_OK;
 }
 
+
 ngx_int_t
 ngx_ssl_ecdh_curve(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *name)
 {
@@ -518,6 +519,7 @@
     return NGX_OK;
 }
 
+
 ngx_int_t
 ngx_ssl_create_connection(ngx_ssl_t *ssl, ngx_connection_t *c, ngx_uint_t flags)
 {

Modified: trunk/src/http/modules/ngx_http_fastcgi_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_fastcgi_module.c	2012-03-05 18:08:23 UTC (rev 4522)
+++ trunk/src/http/modules/ngx_http_fastcgi_module.c	2012-03-05 18:09:06 UTC (rev 4523)
@@ -2432,7 +2432,8 @@
 
         if (prev->headers_hash.buckets
 #if (NGX_HTTP_CACHE)
-            && ((conf->upstream.cache == NULL) == (prev->upstream.cache == NULL))
+            && ((conf->upstream.cache == NULL)
+                == (prev->upstream.cache == NULL))
 #endif
            )
         {

Modified: trunk/src/http/modules/ngx_http_scgi_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_scgi_module.c	2012-03-05 18:08:23 UTC (rev 4522)
+++ trunk/src/http/modules/ngx_http_scgi_module.c	2012-03-05 18:09:06 UTC (rev 4523)
@@ -1384,7 +1384,8 @@
 
         if (prev->headers_hash.buckets
 #if (NGX_HTTP_CACHE)
-            && ((conf->upstream.cache == NULL) == (prev->upstream.cache == NULL))
+            && ((conf->upstream.cache == NULL)
+                == (prev->upstream.cache == NULL))
 #endif
            )
         {

Modified: trunk/src/http/modules/ngx_http_uwsgi_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_uwsgi_module.c	2012-03-05 18:08:23 UTC (rev 4522)
+++ trunk/src/http/modules/ngx_http_uwsgi_module.c	2012-03-05 18:09:06 UTC (rev 4523)
@@ -1441,7 +1441,8 @@
 
         if (prev->headers_hash.buckets
 #if (NGX_HTTP_CACHE)
-            && ((conf->upstream.cache == NULL) == (prev->upstream.cache == NULL))
+            && ((conf->upstream.cache == NULL)
+                == (prev->upstream.cache == NULL))
 #endif
            )
         {

Modified: trunk/src/http/ngx_http_busy_lock.c
===================================================================
--- trunk/src/http/ngx_http_busy_lock.c	2012-03-05 18:08:23 UTC (rev 4522)
+++ trunk/src/http/ngx_http_busy_lock.c	2012-03-05 18:09:06 UTC (rev 4523)
@@ -300,7 +300,7 @@
 
     if (bl->timeout == 0 && bl->max_waiting) {
         ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
-                           "busy lock waiting is useless with zero timeout, ignoring");
+                   "busy lock waiting is useless with zero timeout, ignoring");
     }
 
     return NGX_CONF_OK;

Modified: trunk/src/http/ngx_http_core_module.c
===================================================================
--- trunk/src/http/ngx_http_core_module.c	2012-03-05 18:08:23 UTC (rev 4522)
+++ trunk/src/http/ngx_http_core_module.c	2012-03-05 18:09:06 UTC (rev 4523)
@@ -3970,7 +3970,7 @@
                     }
                 }
 
-                if (lsopt.tcp_keepidle == 0 && lsopt.tcp_keepintvl == 0 
+                if (lsopt.tcp_keepidle == 0 && lsopt.tcp_keepintvl == 0
                     && lsopt.tcp_keepcnt == 0)
                 {
                     goto invalid_so_keepalive;
@@ -4629,7 +4629,7 @@
 
     if (max == 0) {
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
-                           "\"open_file_cache\" must have the \"max\" parameter");
+                        "\"open_file_cache\" must have the \"max\" parameter");
         return NGX_CONF_ERROR;
     }
 

Modified: trunk/src/mail/ngx_mail_core_module.c
===================================================================
--- trunk/src/mail/ngx_mail_core_module.c	2012-03-05 18:08:23 UTC (rev 4522)
+++ trunk/src/mail/ngx_mail_core_module.c	2012-03-05 18:09:06 UTC (rev 4523)
@@ -26,7 +26,7 @@
 
 
 static ngx_conf_deprecated_t  ngx_conf_deprecated_so_keepalive = {
-    ngx_conf_deprecated, "so_keepalive", 
+    ngx_conf_deprecated, "so_keepalive",
     "so_keepalive\" parameter of the \"listen"
 };
 

Modified: trunk/src/os/unix/ngx_process.c
===================================================================
--- trunk/src/os/unix/ngx_process.c	2012-03-05 18:08:23 UTC (rev 4522)
+++ trunk/src/os/unix/ngx_process.c	2012-03-05 18:09:06 UTC (rev 4523)
@@ -544,7 +544,7 @@
 
 static void
 ngx_unlock_mutexes(ngx_pid_t pid)
-{ 
+{
     ngx_uint_t        i;
     ngx_shm_zone_t   *shm_zone;
     ngx_list_part_t  *part;


From RBarnett at trustwave.com  Mon Mar  5 19:24:23 2012
From: RBarnett at trustwave.com (Ryan Barnett)
Date: Mon, 5 Mar 2012 13:24:23 -0600
Subject: Seeking a port of ModSecurity WAF to Nginx
Message-ID: <CB7A7D17.45101%rbarnett@trustwave.com>

Hello everyone,
If you have heard of the ModSecurity open source WAF module for Apache (www.modsecurity.org), we are actively seeking to port the module to other web server platforms (https://modsecurity.org/projects/).  We are seeking Nginx expertise to assist with a port so that Nginx users can directly leverage ModSecurity's rules such as the OWASP ModSecurity Core Rule Set - https://www.owasp.org/index.php/Category:OWASP_ModSecurity_Core_Rule_Set_Project.

If you are interested in helping with this effort ? please sign-up for the ModSecurity developer mail-list: https://lists.sourceforge.net/lists/listinfo/mod-security-developers ? and email me directly so we can add you to the dev team contact list.

Thanks for your help.

--
Ryan Barnett
Trustwave SpiderLabs
ModSecurity Project Leader
OWASP ModSecurity CRS Project Leader

________________________________
This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120305/430e7280/attachment.html>

From piotr.sikora at frickle.com  Mon Mar  5 21:26:35 2012
From: piotr.sikora at frickle.com (Piotr Sikora)
Date: Mon, 5 Mar 2012 22:26:35 +0100
Subject: [PATCH] Fix ./configure script for less error-forgiving compilers
In-Reply-To: <50DFC9CF41D24C87AF89E02E62904937@Desktop>
References: <6D1F240BFA024B0192CAE2E659741B61@Desktop>
	<20101111143328.GW44164@mdounin.ru>
	<50DFC9CF41D24C87AF89E02E62904937@Desktop>
Message-ID: <C7CD1B1B4BA64AE489638EC91D23E2BD@Desktop>

Maxim,
any chances this will ever get committed?

Best regards,
Piotr Sikora < piotr.sikora at frickle.com >


--- auto/types/sizeof.orig      Thu Aug 12 04:08:09 2010
+++ auto/types/sizeof   Thu Aug 12 04:17:26 2010
@@ -19,12 +19,13 @@
#include <sys/time.h>
$NGX_INCLUDE_UNISTD_H
#include <signal.h>
+#include <stdio.h>
#include <sys/resource.h>
$NGX_INCLUDE_INTTYPES_H
$NGX_INCLUDE_AUTO_CONFIG_H

int main() {
-    printf("%d", sizeof($ngx_type));
+    printf("%zu", sizeof($ngx_type));
     return 0;
}

--- auto/types/typedef.orig     Wed Aug 30 10:39:17 2006
+++ auto/types/typedef  Tue Aug 24 19:38:06 2010
@@ -27,7 +27,8 @@
$NGX_INCLUDE_INTTYPES_H

int main() {
-    $ngx_try i = 0;
+    $ngx_try i;
+    i = 0;
     return 0;
}

--- auto/types/uintptr_t.orig   Sun Jul 29 18:24:53 2007
+++ auto/types/uintptr_t        Tue Aug 24 19:38:17 2010
@@ -14,7 +14,8 @@
$NGX_INTTYPES_H

int main() {
-    uintptr_t i = 0;
+    uintptr_t i;
+    i = 0;
     return 0;
}

--- auto/unix.orig      Thu Aug 12 04:33:51 2010
+++ auto/unix   Thu Aug 12 04:34:20 2010
@@ -82,7 +82,7 @@
ngx_feature="setproctitle()"
ngx_feature_name="NGX_HAVE_SETPROCTITLE"
ngx_feature_run=no
-ngx_feature_incs=
+ngx_feature_incs="#include <stdlib.h>"
ngx_feature_path=
ngx_feature_libs=$NGX_SETPROCTITLE_LIB
ngx_feature_test="setproctitle(\"test\");"


From piotr.sikora at frickle.com  Mon Mar  5 21:37:31 2012
From: piotr.sikora at frickle.com (Piotr Sikora)
Date: Mon, 5 Mar 2012 22:37:31 +0100
Subject: [PATCH] Fix ./configure script for less error-forgiving compilers
In-Reply-To: <C7CD1B1B4BA64AE489638EC91D23E2BD@Desktop>
References: <6D1F240BFA024B0192CAE2E659741B61@Desktop>
	<20101111143328.GW44164@mdounin.ru>
	<50DFC9CF41D24C87AF89E02E62904937@Desktop>
	<C7CD1B1B4BA64AE489638EC91D23E2BD@Desktop>
Message-ID: <40F392827F72402D819905F1EF31043E@Desktop>

Attached, my MUA went completely crazy on this diff, sorry.

Best regards,
Piotr Sikora < piotr.sikora at frickle.com >
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nginx__clang_Werror_Wall_fixes_zu.patch
Type: application/octet-stream
Size: 1212 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120305/157c5e1d/attachment.obj>

From piotr.sikora at frickle.com  Mon Mar  5 21:56:27 2012
From: piotr.sikora at frickle.com (Piotr Sikora)
Date: Mon, 5 Mar 2012 22:56:27 +0100
Subject: [PATCH] manpage cleanup from OpenBSD
Message-ID: <06F853F696DD486189D18E681D5E6C38@Desktop>

Hi,
attached manpage cleanup from OpenBSD.

Best regards,
Piotr Sikora < piotr.sikora at frickle.com >
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nginx__OpenBSD_nginx_8_cleanup.patch
Type: application/octet-stream
Size: 4540 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120305/4cc7bde9/attachment-0001.obj>

From ja.nginx at mailnull.com  Mon Mar  5 22:32:51 2012
From: ja.nginx at mailnull.com (SamB)
Date: Mon, 5 Mar 2012 23:32:51 +0100
Subject: [PATCH] add xslt_stylesheet_param directive
In-Reply-To: <20120302185236.GT67687@mdounin.ru>
References: <CALb=60Zx-n0fFn=DVLDuyrJavgDHvX8Ej0H2xGQj3VQAEVJDaw@mail.gmail.com>
	<20120209231221.GR67687@mdounin.ru>
	<CALb=60a2Dsr3GT8x_paOa35W8v3M+W9PR0LfbpH7NaXik7spZA@mail.gmail.com>
	<20120302185236.GT67687@mdounin.ru>
Message-ID: <CALb=60ZtJ9+E3aLhXrof0Ay4=Og1Yd+PsM9iVKNYzTczNT2qWg@mail.gmail.com>

Works for me, thanks a lot :)

On Fri, Mar 2, 2012 at 7:52 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Sun, Feb 26, 2012 at 04:15:33PM +0100, SamB wrote:
>
> > Hi,
> >
> >   sorry for late response,  had no spare time for this. Here is new
> patch,
> > please review it.
> >   I've modified the string_param-enabled - it seems to better fit my
> > usage...
>
> Please check the attached patch.  It cleans up some style issues
> and slightly changes parameter processing to be more uniform with
> parameters from xslt_stylesheet directives.
>
> There is also a tiny patch included which fixes long-standing
> off-by-one bug in parsing parameters from xslt_stylesheet
> directives.
>
> Tests are here:
> http://mdounin.ru/hg/nginx-tests/rev/6bac00bba8d4
>
> Maxim Dounin
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120305/d9e07056/attachment.html>

From ru at nginx.com  Tue Mar  6 06:54:49 2012
From: ru at nginx.com (ru at nginx.com)
Date: Tue,  6 Mar 2012 06:54:49 +0000
Subject: [nginx] svn commit:  r4524 - trunk/docs/man
Message-ID: <20120306065449.8CDB83F9C1E@mail.nginx.com>

Author: ru
Date: 2012-03-06 06:54:48 +0000 (Tue, 06 Mar 2012)
New Revision: 4524

Log:
- Applied some of the OpenBSD changes.
- Expanded contractions.
- Fixed some markup.
- Updated URL of official documentation.


Modified:
   trunk/docs/man/nginx.8

Modified: trunk/docs/man/nginx.8
===================================================================
--- trunk/docs/man/nginx.8	2012-03-05 18:09:06 UTC (rev 4523)
+++ trunk/docs/man/nginx.8	2012-03-06 06:54:48 UTC (rev 4524)
@@ -25,7 +25,7 @@
 .\" SUCH DAMAGE.
 .\"
 .\"
-.Dd August 10, 2011
+.Dd March 6, 2012
 .Dt NGINX 8
 .Os
 .Sh NAME
@@ -33,25 +33,22 @@
 .Nd "HTTP and reverse proxy server, mail proxy server"
 .Sh SYNOPSIS
 .Nm
-.Op Fl hqtvV?
+.Op Fl ?hqtVv
 .Op Fl c Ar file
 .Op Fl g Ar directives
 .Op Fl p Ar prefix
 .Op Fl s Ar signal
 .Sh DESCRIPTION
-The
 .Nm
-(spelled
+(pronounced
 .Dq engine x )
 is an HTTP and reverse proxy server, as well as a mail proxy server.
-The
-.Nm
-is known for its high performance, stability, rich feature set, simple
+It is known for its high performance, stability, rich feature set, simple
 configuration, and low resource consumption.
 .Pp
 The options are as follows:
 .Bl -tag -width ".Fl d Ar directives"
-.It Fl ?\& | h
+.It Fl ?\& , h
 Print help.
 .It Fl c Ar file
 Use an alternative configuration
@@ -62,20 +59,20 @@
 .Sx EXAMPLES
 for details.
 .It Fl p Ar prefix
-Set prefix path.
-Default value is
+Set the prefix path.
+The default value is
 .Pa %%PREFIX%% .
 .It Fl q
 Suppress non-error messages during configuration testing.
 .It Fl s Ar signal
-Send signal to the master process.
+Send a signal to the master process.
 The argument
 .Ar signal
 can be one of:
 .Cm stop , quit , reopen , reload .
-The following table shows the corresponding system signals.
+The following table shows the corresponding system signals:
 .Pp
-.Bl -tag -width ".It Cm reopen" -compact
+.Bl -tag -width ".Cm reopen" -compact
 .It Cm stop
 .Dv SIGTERM
 .It Cm quit
@@ -86,49 +83,48 @@
 .Dv SIGHUP
 .El
 .It Fl t
-Don't run, just test the configuration file.
-The
+Do not run, just test the configuration file.
 .Nm
-checks configuration for correct syntax and then tries to open files
-referred in configuration.
-.It Fl v
-Print
-.Nm
-version.
+checks the configuration file syntax and then tries to open files
+referenced in the configuration file.
 .It Fl V
-Print
+Print the
 .Nm
-version, compiler version and
+version, compiler version, and
 .Pa configure
 script parameters.
+.It Fl v
+Print the
+.Nm
+version.
 .El
 .Sh SIGNALS
 The master process of
 .Nm
-can handle the following signals.
+can handle the following signals:
 .Pp
-.Bl -tag -width ".It Dv SIGINT , SIGTERM" -compact
+.Bl -tag -width ".Dv SIGINT , SIGTERM" -compact
 .It Dv SIGINT , SIGTERM
 Shut down quickly.
 .It Dv SIGHUP
 Reload configuration, start the new worker process with a new
-configuration, gracefully shut down old worker processes.
+configuration, and gracefully shut down old worker processes.
 .It Dv SIGQUIT
 Shut down gracefully.
 .It Dv SIGUSR1
 Reopen log files.
 .It Dv SIGUSR2
-Upgrade
+Upgrade the
 .Nm
 executable on the fly.
 .It Dv SIGWINCH
-Shut down gracefully worker processes.
+Shut down worker processes gracefully.
 .El
 .Pp
-While there's no need to explicitly control worker processes normally,
-they support some signals, too:
+While there is no need to explicitly control worker processes normally,
+they support some signals too:
 .Pp
-.Bl -tag -width ".It Dv SIGINT , SIGTERM" -compact
+.Bl -tag -width ".Dv SIGINT , SIGTERM" -compact
 .It Dv SIGTERM
 Shut down quickly.
 .It Dv SIGQUIT
@@ -150,40 +146,38 @@
 .Pp
 .Dl "error_log /path/to/log debug;"
 .Pp
-It is also possible to enable the debugging for some IP address:
+It is also possible to enable the debugging for a particular IP address:
 .Bd -literal -offset indent
 events {
 	debug_connection 127.0.0.1;
 }
 .Ed
 .Sh FILES
-.Bl -tag -width indent -compact
+.Bl -tag -width indent
 .It Pa %%PID_PATH%%
-Contains the process ID of the
-.Nm
-listening for connections.
-The content of this file is not sensitive; it can be world-readable.
+Contains the process ID of
+.Nm .
+The contents of this file are not sensitive, so it can be world-readable.
 .It Pa %%CONF_PATH%%
-Main configuration file.
+The main configuration file.
 .It Pa %%ERROR_LOG_PATH%%
 Error log file.
 .El
 .Sh EXIT STATUS
 Exit status is 0 on success, or 1 if the command fails.
 .Sh EXAMPLES
-.Bd -literal
-nginx -t -c ~/mynginx.conf -g "pid /var/run/mynginx.pid; worker_processes 2;"
-.Ed
 Test configuration file
 .Pa ~/mynginx.conf
-with global directives for PID and quantity of worker processes.
+with global directives for PID and quantity of worker processes:
+.Bd -literal -offset indent
+nginx -t -c ~/mynginx.conf \e
+	-g "pid /var/run/mynginx.pid; worker_processes 2;"
+.Ed
 .Sh SEE ALSO
 .\"Xr nginx.conf 5
 .\"Pp
 Documentation at
-.Pa http://nginx.org/
-and
-.Pa http://sysoev.ru/nginx/ .
+.Pa http://nginx.org/en/docs/ .
 .Pp
 For questions and technical support, please refer to
 .Pa http://nginx.org/en/support.html .
@@ -193,10 +187,10 @@
 started in 2002, with the first public release on October 4, 2004.
 .Sh AUTHORS
 .An -nosplit
-.An Igor Sysoev Aq igor at sysoev.ru
+.An Igor Sysoev Aq igor at sysoev.ru .
 .Pp
 This manual page was written by
 .An Sergey A. Osokin Aq osa at FreeBSD.org.ru
-as a result of compilation of many
+as a result of compiling many
 .Nm
-documents all over the world.
+documents from all over the world.


From ru at nginx.com  Tue Mar  6 07:00:00 2012
From: ru at nginx.com (Ruslan Ermilov)
Date: Tue, 6 Mar 2012 11:00:00 +0400
Subject: [PATCH] manpage cleanup from OpenBSD
In-Reply-To: <06F853F696DD486189D18E681D5E6C38@Desktop>
References: <06F853F696DD486189D18E681D5E6C38@Desktop>
Message-ID: <20120306070000.GA90547@lo0.su>

On Mon, Mar 05, 2012 at 10:56:27PM +0100, Piotr Sikora wrote:
> Hi,
> attached manpage cleanup from OpenBSD.

I've applied most of the non-controversial changes, plus some more:

http://trac.nginx.org/nginx/changeset/4524/nginx


From ru at nginx.com  Tue Mar  6 08:45:03 2012
From: ru at nginx.com (Ruslan Ermilov)
Date: Tue, 6 Mar 2012 12:45:03 +0400
Subject: [PATCH] Fix ./configure script for less error-forgiving compilers
In-Reply-To: <40F392827F72402D819905F1EF31043E@Desktop>
References: <6D1F240BFA024B0192CAE2E659741B61@Desktop>
	<20101111143328.GW44164@mdounin.ru>
	<50DFC9CF41D24C87AF89E02E62904937@Desktop>
	<C7CD1B1B4BA64AE489638EC91D23E2BD@Desktop>
	<40F392827F72402D819905F1EF31043E@Desktop>
Message-ID: <20120306084503.GB90547@lo0.su>

On Mon, Mar 05, 2012 at 10:37:31PM +0100, Piotr Sikora wrote:
> --- auto/types/uintptr_t.orig	Sun Jul 29 18:24:53 2007
> +++ auto/types/uintptr_t	Tue Aug 24 19:38:17 2010
> @@ -14,7 +14,8 @@
>  $NGX_INTTYPES_H
>  
>  int main() {
> -    uintptr_t i = 0;
> +    uintptr_t i;
> +    i = 0;
>      return 0;
>  }

The way this script is written, -Wall -Werror were not passed
to the compiler.  I've fixed this, and it made your change
obviously required.

> --- auto/unix.orig	Thu Aug 12 04:33:51 2010
> +++ auto/unix	Thu Aug 12 04:34:20 2010
> @@ -82,7 +82,7 @@
>  ngx_feature="setproctitle()"
>  ngx_feature_name="NGX_HAVE_SETPROCTITLE"
>  ngx_feature_run=no
> -ngx_feature_incs=
> +ngx_feature_incs="#include <stdlib.h>"
>  ngx_feature_path=
>  ngx_feature_libs=$NGX_SETPROCTITLE_LIB
>  ngx_feature_test="setproctitle(\"test\");"

setproctitle() is in <unistd.h> on FreeBSD, and the latter
is always included when available.  This change isn't needed.

I've also fixed configure errors on Linux when configured
with --with-cc=gcc --with-cc-opt="-Wall -Werror".
-------------- next part --------------
Index: auto/os/linux
===================================================================
--- auto/os/linux	(revision 4504)
+++ auto/os/linux	(working copy)
@@ -52,7 +52,7 @@
 ngx_feature_incs="#include <sys/epoll.h>"
 ngx_feature_path=
 ngx_feature_libs=
-ngx_feature_test="int efd = 0, fd = 1, n;
+ngx_feature_test="int efd = 0;
                   struct epoll_event ee;
                   ee.events = EPOLLIN|EPOLLOUT|EPOLLET;
                   ee.data.ptr = NULL;
@@ -142,7 +142,7 @@
 ngx_feature_path=
 ngx_feature_libs=-lcrypt
 ngx_feature_test="struct crypt_data  cd;
-                  crypt_r(NULL, NULL, &cd);"
+                  crypt_r(\"key\", \"salt\", &cd);"
 . auto/feature
 
 
Index: auto/unix
===================================================================
--- auto/unix	(revision 4504)
+++ auto/unix	(working copy)
@@ -237,7 +237,7 @@
 ngx_feature_path=
 ngx_feature_libs=
 ngx_feature_test="struct statfs  fs;
-                  statfs(NULL, &fs);"
+                  statfs(\".\", &fs);"
 . auto/feature
 
 
@@ -249,7 +249,7 @@
 ngx_feature_path=
 ngx_feature_libs=
 ngx_feature_test="struct statvfs  fs;
-                  statvfs(NULL, &fs);"
+                  statvfs(\".\", &fs);"
 . auto/feature
 
 
@@ -585,7 +585,8 @@
 ngx_feature="memalign()"
 ngx_feature_name="NGX_HAVE_MEMALIGN"
 ngx_feature_run=no
-ngx_feature_incs="#include <stdlib.h>"
+ngx_feature_incs="#include <stdlib.h>
+                  #include <malloc.h>"
 ngx_feature_path=
 ngx_feature_libs=
 ngx_feature_test="void *p; p = memalign(4096, 4096)"
Index: auto/types/sizeof
===================================================================
--- auto/types/sizeof	(revision 4504)
+++ auto/types/sizeof	(working copy)
@@ -20,12 +20,13 @@
 #include <sys/time.h>
 $NGX_INCLUDE_UNISTD_H
 #include <signal.h>
+#include <stdio.h>
 #include <sys/resource.h>
 $NGX_INCLUDE_INTTYPES_H
 $NGX_INCLUDE_AUTO_CONFIG_H
 
 int main() {
-    printf("%d", sizeof($ngx_type));
+    printf("%zu", sizeof($ngx_type));
     return 0;
 }
 
Index: auto/types/typedef
===================================================================
--- auto/types/typedef	(revision 4504)
+++ auto/types/typedef	(working copy)
@@ -28,7 +28,8 @@
 $NGX_INCLUDE_INTTYPES_H
 
 int main() {
-    $ngx_try i = 0;
+    $ngx_try i;
+    i = 0;
     return 0;
 }
 
Index: auto/types/uintptr_t
===================================================================
--- auto/types/uintptr_t	(revision 4504)
+++ auto/types/uintptr_t	(working copy)
@@ -4,8 +4,8 @@
 
 
 echo $ngx_n "checking for uintptr_t ...$ngx_c"
-echo >> $NGX_ERR
-echo "checking for uintptr_t" >> $NGX_ERR
+echo >> $NGX_AUTOCONF_ERR
+echo "checking for uintptr_t" >> $NGX_AUTOCONF_ERR
 
 found=no
 
@@ -15,13 +15,17 @@
 $NGX_INTTYPES_H
 
 int main() {
-    uintptr_t i = 0;
+    uintptr_t i;
+    i = 0;
     return 0;
 }
 
 END
 
-eval "$CC -o $NGX_AUTOTEST $NGX_AUTOTEST.c >> $NGX_ERR 2>&1"
+ngx_test="$CC $CC_TEST_FLAGS $CC_AUX_FLAGS \
+          -o $NGX_AUTOTEST $NGX_AUTOTEST.c $NGX_LD_OPT"
+
+eval "$ngx_test >> $NGX_AUTOCONF_ERR 2>&1"
 
 if [ -x $NGX_AUTOTEST ]; then
     echo " uintptr_t found"

From piotr.sikora at frickle.com  Tue Mar  6 14:50:59 2012
From: piotr.sikora at frickle.com (Piotr Sikora)
Date: Tue, 6 Mar 2012 15:50:59 +0100
Subject: [PATCH] Fix ./configure script for less error-forgiving compilers
In-Reply-To: <20120306084503.GB90547@lo0.su>
References: <6D1F240BFA024B0192CAE2E659741B61@Desktop>
	<20101111143328.GW44164@mdounin.ru>
	<50DFC9CF41D24C87AF89E02E62904937@Desktop>
	<C7CD1B1B4BA64AE489638EC91D23E2BD@Desktop>
	<40F392827F72402D819905F1EF31043E@Desktop>
	<20120306084503.GB90547@lo0.su>
Message-ID: <35F0DEF66E7D4B49BEBA330CFE09FBC7@Desktop>

Hi Ruslan,

> setproctitle() is in <unistd.h> on FreeBSD, and the latter
> is always included when available.  This change isn't needed.

But it's in <stdlib.h> on OpenBSD and NetBSD.

> I've also fixed configure errors on Linux when configured
> with --with-cc=gcc --with-cc-opt="-Wall -Werror".

Looks OK to me, thanks.

Best regards,
Piotr Sikora < piotr.sikora at frickle.com >


From soares.chen at gmail.com  Tue Mar  6 16:04:15 2012
From: soares.chen at gmail.com (Soares Chen)
Date: Tue, 6 Mar 2012 17:04:15 +0100
Subject: Buffer Reuse in Nginx
Message-ID: <CACUDQOE_-1tbhUXFtyvz+iDX9732Q5J+aALo_JzRxbLrUsp7vw@mail.gmail.com>

Hi,

I am trying to write an Nginx module that receive input and produce
output in streaming fashion over long lasting connection. Due to the
nature of the connection, I can't wait for all the buffers to be
reclaimed at the end of the request, as that would potentially cause
high memory consumption to the server. As a result, I am looking for
ways to determine if buffers sent through ngx_http_output_filter() are
safe to be reused.

The problem:
When ngx_http_output_filter() is called frequent enough with large
buffers, it is possible that NGX_AGAIN would be returned, indicating
that some buffers are retained somewhere in the output chain waiting
to be sent later. This creates problem for buffer reuse because there
is no way to know which buffers are being retained, and when the
retained buffers will become free after being sent. I tried doing
experiment of sending large amount of dynamic data through
ngx_http_output_filter() and then immediately modifying the buffer
data. What I found is that the data would become corrupted when
NGX_AGAIN is returned.


There are lack of documentation and resources about buffer reuse when
I searched through the Internet. I also tried reading the source code
but find most parts confusing even after days of studying. So I'd like
to clarify a few clues that I found to determine if my understanding
is correct. Following are the data structures and functions that I
believe are related:


#define ngx_free_chain(pool, cl)
Unfortunately this function's name does not reflect that it does
exactly. From my understanding it only recycles the ngx_chain_t object
by placing the pointer into pool->chain. Even the ngx_buf_t object
that cl->buf points to is completely ignored.


ngx_chain_t* ngx_alloc_chain_link(ngx_pool_t *pool)
This function only allocate the ngx_chain_t object and will try to
reuse the chains that were freed by ngx_free_chain(). Because the buf
pointer in the chain is ignored it is not safe to assume that the
ngx_buf_t object and the buffer data can be reused.


ngx_int_t ngx_output_chain(ngx_output_chain_ctx_t *ctx, ngx_chain_t *in)
It seems to me that most of the buffer copying magic happens in this
function. I tried as hard to follow but still could not fully
understand what the function does. As I understand this function would
copy the actual buffer data if the ngx_buf_t object has some specific
flags set as determined by ngx_output_chain_as_is(). I am not sure
whether I should set any flags to instruct ngx_output_chain() to copy
all buffer data so that I can safely reuse the buffers that I own.


typedef void* ngx_buf_tag_t
This mysterious tag seems to be the way for me to claim ownership to a
buffer by assigning it a unique pointer value. However I could find
almost no explanation on how to use this tag field properly. I'd like
to know if setting this tag would guarantee that the buffers I created
would never be shared ownership with other modules?


void ngx_chain_update_chains(ngx_pool_t *p, ngx_chain_t **free,
ngx_chain_t **busy, ngx_chain_t **out, ngx_buf_tag_t tag)
I find that this function seems to be performing what I want, and it
seems to be called in other modules that has similar buffer reuse
mechanism. However I am really confused about the purpose of this
function and what it does exactly. From the signature it seems to be
determining which buffers are safe to reuse, and then reclaim the free
buffers into the **free chain. However on close inspection I found
that all it does is to move all tagged buffers at **busy and **out to
free, while calling ngx_free_chain() on buffer chains that do not
share the same tag. I don't know if the buffers freed by this function
is guaranteed safe to be reused, and I don't know what happen with the
buffers that have different tags.


ngx_chain_t * ngx_chain_get_free_buf(ngx_pool_t *p, ngx_chain_t **free)
I find that this function will return the buffers freed by
ngx_chain_update_chains(). Most modules seem to do overwrite the data
on the obtained buffer without any issue. That makes me wonder if
ngx_chain_update_chains() really works.


ngx_chain_t * ngx_connection_s::send_chain(ngx_connection_t *c,
ngx_chain_t *in, off_t limit)
The function pointer at r->connection->send_chain would return the
buffer chain that it has not yet sent. I also found that the returned
chain is then stored in r->out waiting to be sent next time. So it
seems like I can determine if my buffers are safe for reuse by
checking if the buffer chains at r->out point to the same buffer data.
However I am not sure if solely based on this method is really safe,
especially if there are filter modules that retain buffers in their
own context.


ngx_int_t ngx_http_output_filter(ngx_http_request_t *r, ngx_chain_t *in)
After so many clues that I stated above, I just wish to know what is
really the right way to determine if buffers are safe for reuse after
this function, ngx_http_output_filter(), is called. Can I just set
buf->tag? Or should I check r->out? Or should I call
ngx_chain_get_free_buf()?


I am sorry if I have any misunderstanding about the internals or used
the wrong term in describing my findings. I would love to spend more
time to study the Nginx source code, but it has been weeks and I
really need to proceed with my module development. I'd appreciate if
anyone can clarify my misunderstandings and explain to me what exactly
happens when buffers are being sent.

Thanks!


References:
http://mailman.nginx.org/pipermail/nginx-devel/2011-September/001180.html
http://mailman.nginx.org/pipermail/nginx/2010-April/019814.html


Regards,

Soares


From zzz at zzz.org.ua  Tue Mar  6 16:28:20 2012
From: zzz at zzz.org.ua (Alexandr Gomoliako)
Date: Tue, 6 Mar 2012 18:28:20 +0200
Subject: Buffer Reuse in Nginx
In-Reply-To: <CACUDQOE_-1tbhUXFtyvz+iDX9732Q5J+aALo_JzRxbLrUsp7vw@mail.gmail.com>
References: <CACUDQOE_-1tbhUXFtyvz+iDX9732Q5J+aALo_JzRxbLrUsp7vw@mail.gmail.com>
Message-ID: <CAKb9G9gW7tD17A0v4uuPKchk6NuhDohA7EBXRmcLqU08-jToiQ@mail.gmail.com>

On 3/6/12, Soares Chen <soares.chen at gmail.com> wrote:
> I am trying to write an Nginx module that receive input and produce
> output in streaming fashion over long lasting connection. Due to the
> nature of the connection, I can't wait for all the buffers to be
> reclaimed at the end of the request, as that would potentially cause
> high memory consumption to the server. As a result, I am looking for
> ways to determine if buffers sent through ngx_http_output_filter() are
> safe to be reused.

There is another way: you can skip ngx_http_output_filter() altogether
and use r->connection directly with c->send/c->recv and your own
handlers for c->read/c->write.

This will be much easier with perl though. I didn't document it very
well, but I'm willing to help you, if you are going to use this:
http://zzzcpan.github.com/nginx-perl/Nginx.html#CONNECTION_TAKEOVER


From mdounin at mdounin.ru  Tue Mar  6 17:26:44 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 6 Mar 2012 21:26:44 +0400
Subject: Buffer Reuse in Nginx
In-Reply-To: <CACUDQOE_-1tbhUXFtyvz+iDX9732Q5J+aALo_JzRxbLrUsp7vw@mail.gmail.com>
References: <CACUDQOE_-1tbhUXFtyvz+iDX9732Q5J+aALo_JzRxbLrUsp7vw@mail.gmail.com>
Message-ID: <20120306172644.GO67687@mdounin.ru>

Hello!

On Tue, Mar 06, 2012 at 05:04:15PM +0100, Soares Chen wrote:

> Hi,
> 
> I am trying to write an Nginx module that receive input and produce
> output in streaming fashion over long lasting connection. Due to the
> nature of the connection, I can't wait for all the buffers to be
> reclaimed at the end of the request, as that would potentially cause
> high memory consumption to the server. As a result, I am looking for
> ways to determine if buffers sent through ngx_http_output_filter() are
> safe to be reused.
> 
> The problem:
> When ngx_http_output_filter() is called frequent enough with large
> buffers, it is possible that NGX_AGAIN would be returned, indicating
> that some buffers are retained somewhere in the output chain waiting
> to be sent later. This creates problem for buffer reuse because there
> is no way to know which buffers are being retained, and when the
> retained buffers will become free after being sent. I tried doing
> experiment of sending large amount of dynamic data through
> ngx_http_output_filter() and then immediately modifying the buffer
> data. What I found is that the data would become corrupted when
> NGX_AGAIN is returned.
> 
> 
> There are lack of documentation and resources about buffer reuse when
> I searched through the Internet. I also tried reading the source code
> but find most parts confusing even after days of studying. So I'd like
> to clarify a few clues that I found to determine if my understanding
> is correct. Following are the data structures and functions that I
> believe are related:
> 
> 
> #define ngx_free_chain(pool, cl)
> Unfortunately this function's name does not reflect that it does
> exactly. From my understanding it only recycles the ngx_chain_t object
> by placing the pointer into pool->chain. Even the ngx_buf_t object
> that cl->buf points to is completely ignored.
> 
> 
> ngx_chain_t* ngx_alloc_chain_link(ngx_pool_t *pool)
> This function only allocate the ngx_chain_t object and will try to
> reuse the chains that were freed by ngx_free_chain(). Because the buf
> pointer in the chain is ignored it is not safe to assume that the
> ngx_buf_t object and the buffer data can be reused.

Correct, these two functions deal with chain links, and they 
completely ignore any possible content of the structures.  They 
are basically equivalent to

    ngx_palloc(pool, sizeof(ngx_chain_t));
    ngx_pfree(pool, cl);

> ngx_int_t ngx_output_chain(ngx_output_chain_ctx_t *ctx, ngx_chain_t *in)
> It seems to me that most of the buffer copying magic happens in this
> function. I tried as hard to follow but still could not fully
> understand what the function does. As I understand this function would
> copy the actual buffer data if the ngx_buf_t object has some specific
> flags set as determined by ngx_output_chain_as_is(). I am not sure
> whether I should set any flags to instruct ngx_output_chain() to copy
> all buffer data so that I can safely reuse the buffers that I own.

You shouldn't instruct it to copy anything.

Instead, you should reuse your own buffers as long as they are 
freed, via ngx_chain_update_chains() and friends.  See below.

> typedef void* ngx_buf_tag_t
> This mysterious tag seems to be the way for me to claim ownership to a
> buffer by assigning it a unique pointer value. However I could find
> almost no explanation on how to use this tag field properly. I'd like
> to know if setting this tag would guarantee that the buffers I created
> would never be shared ownership with other modules?

It is used by ngx_chain_update_chains() to match buffers allocated 
by your module.

> void ngx_chain_update_chains(ngx_pool_t *p, ngx_chain_t **free,
> ngx_chain_t **busy, ngx_chain_t **out, ngx_buf_tag_t tag)
> I find that this function seems to be performing what I want, and it
> seems to be called in other modules that has similar buffer reuse
> mechanism. However I am really confused about the purpose of this
> function and what it does exactly. From the signature it seems to be
> determining which buffers are safe to reuse, and then reclaim the free
> buffers into the **free chain. However on close inspection I found
> that all it does is to move all tagged buffers at **busy and **out to
> free, while calling ngx_free_chain() on buffer chains that do not
> share the same tag. I don't know if the buffers freed by this function
> is guaranteed safe to be reused, and I don't know what happen with the
> buffers that have different tags.

Buffers only moved to **free if they are indeed free, i.e. when 
ngx_buf_size(cl->buf) == 0.  Buffers from **free will be then 
reused either with ngx_chain_get_free_buf() or with your own code.

> ngx_chain_t * ngx_chain_get_free_buf(ngx_pool_t *p, ngx_chain_t **free)
> I find that this function will return the buffers freed by
> ngx_chain_update_chains(). Most modules seem to do overwrite the data
> on the obtained buffer without any issue. That makes me wonder if
> ngx_chain_update_chains() really works.

See above.

> ngx_chain_t * ngx_connection_s::send_chain(ngx_connection_t *c,
> ngx_chain_t *in, off_t limit)
> The function pointer at r->connection->send_chain would return the
> buffer chain that it has not yet sent. I also found that the returned
> chain is then stored in r->out waiting to be sent next time. So it
> seems like I can determine if my buffers are safe for reuse by
> checking if the buffer chains at r->out point to the same buffer data.
> However I am not sure if solely based on this method is really safe,
> especially if there are filter modules that retain buffers in their
> own context.

No, this isn't correct aproach.  Don't touch r->out, use 
ngx_chain_update_chains() instead, it will do needed work for you.

> ngx_int_t ngx_http_output_filter(ngx_http_request_t *r, ngx_chain_t *in)
> After so many clues that I stated above, I just wish to know what is
> really the right way to determine if buffers are safe for reuse after
> this function, ngx_http_output_filter(), is called. Can I just set
> buf->tag? Or should I check r->out? Or should I call
> ngx_chain_get_free_buf()?

You should really use ngx_chain_update_chains().  The basic 
aproach is to do something like this (partially stolen from 
chunked filter):

    cl = ngx_chain_get_free_buf(r->pool, &ctx->free);
    if (cl == NULL) {
        return NGX_ERROR;
    }

    if (cl->buf->start == NULL) {
        /*
         * allocate memory for a buffer, if you really need one 
         * with associated memory region
         */

        ...
    }

    cl->buf->tag = (ngx_buf_tag_t) &ngx_http_chunked_filter_module;

    ...

    rc = ngx_http_output_filter(r, out);

    ngx_chain_update_chains(r->pool, &ctx->free, &ctx->busy, &out,
                            (ngx_buf_tag_t) &ngx_http_chunked_filter_module);

    ...

You may also want to add some extra processing to ensure that no 
more than specified number of buffers will be allocated, handle 
case when you can't allocate more buffers and so on.  Note that 
chunked filter doesn't do this as it doesn't really care (it just 
want to reuse it's own buffers, but doesn't cap number of them), 
you may want to take a look at gzip filter and/or upstream module 
and event pipe code for more complex examples.

Maxim Dounin


From soares.chen at gmail.com  Wed Mar  7 13:41:55 2012
From: soares.chen at gmail.com (Soares Chen)
Date: Wed, 7 Mar 2012 14:41:55 +0100
Subject: Buffer Reuse in Nginx
In-Reply-To: <20120306172644.GO67687@mdounin.ru>
References: <CACUDQOE_-1tbhUXFtyvz+iDX9732Q5J+aALo_JzRxbLrUsp7vw@mail.gmail.com>
	<20120306172644.GO67687@mdounin.ru>
Message-ID: <CACUDQOGeL-SyLpir0QcMmgVi4rUgpaD8Q__Q=6nSMf+KkMM_7Q@mail.gmail.com>

On Tue, Mar 6, 2012 at 5:28 PM, Alexandr Gomoliako <zzz at zzz.org.ua> wrote:
> There is another way: you can skip ngx_http_output_filter() altogether
> and use r->connection directly with c->send/c->recv and your own
> handlers for c->read/c->write.

I believe that would be a last resort for me as that would disable the
whole benefits of Nginx modules and create incompatibility. The
solution should work though.

> This will be much easier with perl though. I didn't document it very
> well, but I'm willing to help you, if you are going to use this:
> http://zzzcpan.github.com/nginx-perl/Nginx.html#CONNECTION_TAKEOVER

Thanks for the offer, but my project is written in C/C++. Though I'll
look into it and see if I can apply this design pattern in my module.


On Tue, Mar 6, 2012 at 6:26 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
>> #define ngx_free_chain(pool, cl)
>> Unfortunately this function's name does not reflect that it does
>> exactly. From my understanding it only recycles the ngx_chain_t object
>> by placing the pointer into pool->chain. Even the ngx_buf_t object
>> that cl->buf points to is completely ignored.
>>
>>
>> ngx_chain_t* ngx_alloc_chain_link(ngx_pool_t *pool)
>> This function only allocate the ngx_chain_t object and will try to
>> reuse the chains that were freed by ngx_free_chain(). Because the buf
>> pointer in the chain is ignored it is not safe to assume that the
>> ngx_buf_t object and the buffer data can be reused.
>
> Correct, these two functions deal with chain links, and they
> completely ignore any possible content of the structures. ?They
> are basically equivalent to
>
> ? ?ngx_palloc(pool, sizeof(ngx_chain_t));
> ? ?ngx_pfree(pool, cl);

Ok got it. So my interpretation is correct.

>> ngx_int_t ngx_output_chain(ngx_output_chain_ctx_t *ctx, ngx_chain_t *in)
>> It seems to me that most of the buffer copying magic happens in this
>> function. I tried as hard to follow but still could not fully
>> understand what the function does. As I understand this function would
>> copy the actual buffer data if the ngx_buf_t object has some specific
>> flags set as determined by ngx_output_chain_as_is(). I am not sure
>> whether I should set any flags to instruct ngx_output_chain() to copy
>> all buffer data so that I can safely reuse the buffers that I own.
>
> You shouldn't instruct it to copy anything.
>
> Instead, you should reuse your own buffers as long as they are
> freed, via ngx_chain_update_chains() and friends. ?See below.

Alright.

>> typedef void* ngx_buf_tag_t
>> This mysterious tag seems to be the way for me to claim ownership to a
>> buffer by assigning it a unique pointer value. However I could find
>> almost no explanation on how to use this tag field properly. I'd like
>> to know if setting this tag would guarantee that the buffers I created
>> would never be shared ownership with other modules?
>
> It is used by ngx_chain_update_chains() to match buffers allocated
> by your module.

Ok then this is my interpretation for the buffer tag: There is always
exactly one ngx_buf_t object that points to a particular memory area,
and it is owned by the module that set the tag field. The modules down
the output chain would change the b->pos and b->last pointers and that
change is used by the buffer owner to determine if the buffer is free
to reuse again.

>> void ngx_chain_update_chains(ngx_pool_t *p, ngx_chain_t **free,
>> ngx_chain_t **busy, ngx_chain_t **out, ngx_buf_tag_t tag)
>> I find that this function seems to be performing what I want, and it
>> seems to be called in other modules that has similar buffer reuse
>> mechanism. However I am really confused about the purpose of this
>> function and what it does exactly. From the signature it seems to be
>> determining which buffers are safe to reuse, and then reclaim the free
>> buffers into the **free chain. However on close inspection I found
>> that all it does is to move all tagged buffers at **busy and **out to
>> free, while calling ngx_free_chain() on buffer chains that do not
>> share the same tag. I don't know if the buffers freed by this function
>> is guaranteed safe to be reused, and I don't know what happen with the
>> buffers that have different tags.
>
> Buffers only moved to **free if they are indeed free, i.e. when
> ngx_buf_size(cl->buf) == 0. ?Buffers from **free will be then
> reused either with ngx_chain_get_free_buf() or with your own code.

Ah now I realized I missed that line of code. So what you mean is that
it is also possible to manually determine if a buffer is free for
reuse by checking if ngx_buf_size(b) is 0?

To make sure I get the idea correct I am showing here my
interpretation of this function:

void
ngx_chain_update_chains(ngx_pool_t *p, ngx_chain_t **free, ngx_chain_t **busy,
    ngx_chain_t **out, ngx_buf_tag_t tag)
{
    ngx_chain_t  *cl;

    // Append *out to the end of *busy chain, and empty the *out chain
    if (*busy == NULL) {
        *busy = *out;

    } else {
        for (cl = *busy; cl->next; cl = cl->next) { /* void */ }

        cl->next = *out;
    }

    *out = NULL;

    // For each buffer in the busy chain
    while (*busy) {
        cl = *busy;

        // If the size is not zero, then the buffer is still being used.
        // We assume that the rest of buffers in the chain are still
        // in use as well, so we break and return.
        if (ngx_buf_size(cl->buf) != 0) {
            break;
        }

        // If the buffer is not owned by current module, proceed to
        // the next buffer and free the chain link while ignoring the
        // buffer pointer.
        if (cl->buf->tag != tag) {
            *busy = cl->next;
            ngx_free_chain(p, cl);
            continue;
        }

        // else, the buffer is owned by the current module
        // and the buffer is free for reuse.
        // reset the pos and last pointer.
        cl->buf->pos = cl->buf->start;
        cl->buf->last = cl->buf->start;

        // Put the chain together with the buffer into the *free chain
        // The buffer pointers at this *free chain link is guaranteed to
        // be valid and it's memory region is safe for reuse.
        *busy = cl->next;
        cl->next = *free;
        *free = cl;
    }
}

>> ngx_chain_t * ngx_chain_get_free_buf(ngx_pool_t *p, ngx_chain_t **free)
>> I find that this function will return the buffers freed by
>> ngx_chain_update_chains(). Most modules seem to do overwrite the data
>> on the obtained buffer without any issue. That makes me wonder if
>> ngx_chain_update_chains() really works.
>
> See above.

>From what I understand ngx_chain_get_free_buf() will always return a
ngx_chain_t object with valid pointer to a ngx_buf_t object. On the
opposite, ngx_alloc_chain_link() only returns the ngx_chain_t object
and the buf pointer must be reseted before any use.

Also, if the *free chain contains any chain link, that chain link will
be returned and the buffer attached would have b->start and b->end
pointing to memory region that is safe to write. But on the other hand
of *free is empty, then ngx_chain_get_free_buf would only allocate new
ngx_chain_t and ngx_buf_t objects but leave the b->start and b->end
pointers as NULL. As a result the caller of ngx_chain_get_free_buf()
must ensure that cl->buf->start points to some memory region, or the
caller itself have to manually allocate that memory region and assign
them to cl->buf.

>> ngx_chain_t * ngx_connection_s::send_chain(ngx_connection_t *c,
>> ngx_chain_t *in, off_t limit)
>> The function pointer at r->connection->send_chain would return the
>> buffer chain that it has not yet sent. I also found that the returned
>> chain is then stored in r->out waiting to be sent next time. So it
>> seems like I can determine if my buffers are safe for reuse by
>> checking if the buffer chains at r->out point to the same buffer data.
>> However I am not sure if solely based on this method is really safe,
>> especially if there are filter modules that retain buffers in their
>> own context.
>
> No, this isn't correct aproach. ?Don't touch r->out, use
> ngx_chain_update_chains() instead, it will do needed work for you.

Ok thanks for the warning.

>> ngx_int_t ngx_http_output_filter(ngx_http_request_t *r, ngx_chain_t *in)
>> After so many clues that I stated above, I just wish to know what is
>> really the right way to determine if buffers are safe for reuse after
>> this function, ngx_http_output_filter(), is called. Can I just set
>> buf->tag? Or should I check r->out? Or should I call
>> ngx_chain_get_free_buf()?
>
> You should really use ngx_chain_update_chains(). ?The basic
> aproach is to do something like this (partially stolen from
> chunked filter):
>
> ? ?cl = ngx_chain_get_free_buf(r->pool, &ctx->free);
> ? ?if (cl == NULL) {
> ? ? ? ?return NGX_ERROR;
> ? ?}
>
> ? ?if (cl->buf->start == NULL) {
> ? ? ? ?/*
> ? ? ? ? * allocate memory for a buffer, if you really need one
> ? ? ? ? * with associated memory region
> ? ? ? ? */
>
> ? ? ? ?...
> ? ?}
>
> ? ?cl->buf->tag = (ngx_buf_tag_t) &ngx_http_chunked_filter_module;
>
> ? ?...
>
> ? ?rc = ngx_http_output_filter(r, out);
>
> ? ?ngx_chain_update_chains(r->pool, &ctx->free, &ctx->busy, &out,
> ? ? ? ? ? ? ? ? ? ? ? ? ? ?(ngx_buf_tag_t) &ngx_http_chunked_filter_module);
>
> ? ?...

Thanks for the boilerplate. Now I am more or less understand how to
write this part of code.

> You may also want to add some extra processing to ensure that no
> more than specified number of buffers will be allocated, handle
> case when you can't allocate more buffers and so on. ?Note that
> chunked filter doesn't do this as it doesn't really care (it just
> want to reuse it's own buffers, but doesn't cap number of them),
> you may want to take a look at gzip filter and/or upstream module
> and event pipe code for more complex examples.

I'll take note of that. Thank you so much for your explanation!


Soares


From mdounin at mdounin.ru  Wed Mar  7 14:44:02 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 7 Mar 2012 18:44:02 +0400
Subject: Buffer Reuse in Nginx
In-Reply-To: <CACUDQOGeL-SyLpir0QcMmgVi4rUgpaD8Q__Q=6nSMf+KkMM_7Q@mail.gmail.com>
References: <CACUDQOE_-1tbhUXFtyvz+iDX9732Q5J+aALo_JzRxbLrUsp7vw@mail.gmail.com>
	<20120306172644.GO67687@mdounin.ru>
	<CACUDQOGeL-SyLpir0QcMmgVi4rUgpaD8Q__Q=6nSMf+KkMM_7Q@mail.gmail.com>
Message-ID: <20120307144401.GB67687@mdounin.ru>

Hello!

On Wed, Mar 07, 2012 at 02:41:55PM +0100, Soares Chen wrote:

[...]

> >> typedef void* ngx_buf_tag_t
> >> This mysterious tag seems to be the way for me to claim ownership to a
> >> buffer by assigning it a unique pointer value. However I could find
> >> almost no explanation on how to use this tag field properly. I'd like
> >> to know if setting this tag would guarantee that the buffers I created
> >> would never be shared ownership with other modules?
> >
> > It is used by ngx_chain_update_chains() to match buffers allocated
> > by your module.
> 
> Ok then this is my interpretation for the buffer tag: There is always
> exactly one ngx_buf_t object that points to a particular memory area,
> and it is owned by the module that set the tag field. The modules down
> the output chain would change the b->pos and b->last pointers and that
> change is used by the buffer owner to determine if the buffer is free
> to reuse again.

Mostly yes.

Though the "exactly one ngx_buf_t" isn't really correct, it's 
module responsibility to ensure that it's either true or handled 
somehow.  E.g. event pipe (src/event/ngx_event_pipe.c) uses 
multiple buffers with the same memory region (i.e. with identical 
b->start) and additionally handles this.

> >> void ngx_chain_update_chains(ngx_pool_t *p, ngx_chain_t **free,
> >> ngx_chain_t **busy, ngx_chain_t **out, ngx_buf_tag_t tag)
> >> I find that this function seems to be performing what I want, and it
> >> seems to be called in other modules that has similar buffer reuse
> >> mechanism. However I am really confused about the purpose of this
> >> function and what it does exactly. From the signature it seems to be
> >> determining which buffers are safe to reuse, and then reclaim the free
> >> buffers into the **free chain. However on close inspection I found
> >> that all it does is to move all tagged buffers at **busy and **out to
> >> free, while calling ngx_free_chain() on buffer chains that do not
> >> share the same tag. I don't know if the buffers freed by this function
> >> is guaranteed safe to be reused, and I don't know what happen with the
> >> buffers that have different tags.
> >
> > Buffers only moved to **free if they are indeed free, i.e. when
> > ngx_buf_size(cl->buf) == 0. ?Buffers from **free will be then
> > reused either with ngx_chain_get_free_buf() or with your own code.
> 
> Ah now I realized I missed that line of code. So what you mean is that
> it is also possible to manually determine if a buffer is free for
> reuse by checking if ngx_buf_size(b) is 0?

Yes.

> To make sure I get the idea correct I am showing here my
> interpretation of this function:
> 
> void
> ngx_chain_update_chains(ngx_pool_t *p, ngx_chain_t **free, ngx_chain_t **busy,
>     ngx_chain_t **out, ngx_buf_tag_t tag)
> {
>     ngx_chain_t  *cl;
> 
>     // Append *out to the end of *busy chain, and empty the *out chain
>     if (*busy == NULL) {
>         *busy = *out;
> 
>     } else {
>         for (cl = *busy; cl->next; cl = cl->next) { /* void */ }
> 
>         cl->next = *out;
>     }
> 
>     *out = NULL;
> 
>     // For each buffer in the busy chain
>     while (*busy) {
>         cl = *busy;
> 
>         // If the size is not zero, then the buffer is still being used.
>         // We assume that the rest of buffers in the chain are still
>         // in use as well, so we break and return.
>         if (ngx_buf_size(cl->buf) != 0) {
>             break;
>         }
> 
>         // If the buffer is not owned by current module, proceed to
>         // the next buffer and free the chain link while ignoring the
>         // buffer pointer.
>         if (cl->buf->tag != tag) {
>             *busy = cl->next;
>             ngx_free_chain(p, cl);
>             continue;
>         }

Note well: all chain links for your out/free/busy chains are 
expected to be allocated by your module, either directly or via 
e.g. ngx_chain_add_copy().

> 
>         // else, the buffer is owned by the current module
>         // and the buffer is free for reuse.
>         // reset the pos and last pointer.
>         cl->buf->pos = cl->buf->start;
>         cl->buf->last = cl->buf->start;
> 
>         // Put the chain together with the buffer into the *free chain
>         // The buffer pointers at this *free chain link is guaranteed to
>         // be valid and it's memory region is safe for reuse.
>         *busy = cl->next;
>         cl->next = *free;
>         *free = cl;
>     }
> }

Looks correct.

> >> ngx_chain_t * ngx_chain_get_free_buf(ngx_pool_t *p, ngx_chain_t **free)
> >> I find that this function will return the buffers freed by
> >> ngx_chain_update_chains(). Most modules seem to do overwrite the data
> >> on the obtained buffer without any issue. That makes me wonder if
> >> ngx_chain_update_chains() really works.
> >
> > See above.
> 
> From what I understand ngx_chain_get_free_buf() will always return a
> ngx_chain_t object with valid pointer to a ngx_buf_t object. On the
> opposite, ngx_alloc_chain_link() only returns the ngx_chain_t object
> and the buf pointer must be reseted before any use.
> 
> Also, if the *free chain contains any chain link, that chain link will
> be returned and the buffer attached would have b->start and b->end
> pointing to memory region that is safe to write. But on the other hand
> of *free is empty, then ngx_chain_get_free_buf would only allocate new
> ngx_chain_t and ngx_buf_t objects but leave the b->start and b->end
> pointers as NULL. As a result the caller of ngx_chain_get_free_buf()
> must ensure that cl->buf->start points to some memory region, or the
> caller itself have to manually allocate that memory region and assign
> them to cl->buf.

I would rather suggest to interpret this a bit differently:

The ngx_alloc_chain_link() function returns ngx_chain_t structure 
with undefined contents.

The ngx_chain_get_free_buf() function returns ngx_chain_t 
structure and associated ngx_buf_t structure.  If returned chain + 
buffer pair is reused, it's guaranteed to have the same 
cl->buf->start as before; else it will have cl->buf->start set to 
NULL.

Net effect is the same though.

[...]

Maxim Dounin


From matthieu.tourne at gmail.com  Thu Mar  8 01:35:56 2012
From: matthieu.tourne at gmail.com (Matthieu Tourne)
Date: Wed, 7 Mar 2012 17:35:56 -0800
Subject: [PATCH] ipv6 support for proxy_pass
Message-ID: <CAAGYmFf5esix3wDky9-h5SkH2HH_7EzaH+zXBL1p3OorTHe=Hw@mail.gmail.com>

Hi,

Attached is a simple patch to proxy_pass to an IPv6 upstream.
It currently works only with a variable. proxy_pass ::1; wont' work for
instance.

but this will :

   set $addr "[::1]:8080";
   proxy_pass http://$addr;

You also need to add --with-ipv6 to your configure line.

Happy hacking,
Matthieu.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120307/9b030bda/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ipv6.patch
Type: application/octet-stream
Size: 1866 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120307/9b030bda/attachment.obj>

From mdounin at mdounin.ru  Thu Mar  8 10:14:52 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 8 Mar 2012 14:14:52 +0400
Subject: [PATCH] ipv6 support for proxy_pass
In-Reply-To: <CAAGYmFf5esix3wDky9-h5SkH2HH_7EzaH+zXBL1p3OorTHe=Hw@mail.gmail.com>
References: <CAAGYmFf5esix3wDky9-h5SkH2HH_7EzaH+zXBL1p3OorTHe=Hw@mail.gmail.com>
Message-ID: <20120308101452.GE67687@mdounin.ru>

Hello!

On Wed, Mar 07, 2012 at 05:35:56PM -0800, Matthieu Tourne wrote:

> Hi,
> 
> Attached is a simple patch to proxy_pass to an IPv6 upstream.
> It currently works only with a variable. proxy_pass ::1; wont' work for
> instance.
> 
> but this will :
> 
>    set $addr "[::1]:8080";
>    proxy_pass http://$addr;
> 
> You also need to add --with-ipv6 to your configure line.

This should be addressed in ngx_parse_url().  The ipv6 cleanup is 
planned somewhere after 1.1.x become stable.

Maxim Dounin


From ahh at one.com  Thu Mar  8 14:01:32 2012
From: ahh at one.com (Adam Hasselbalch Hansen)
Date: Thu, 08 Mar 2012 15:01:32 +0100
Subject: Manipulating the body of a PUT/POST
In-Reply-To: <20120203100450.GH67687@mdounin.ru>
References: <4F2B9FEE.3060601@one.com> <20120203100450.GH67687@mdounin.ru>
Message-ID: <4F58BBBC.9000903@one.com>

On 2012-02-03 11:04, Maxim Dounin wrote:

> I'm working on prototyping input body filtering, it's expected to
> appear somewhere in 1.2.x.  It will allow manipulation of request
> body, as well as other tasks like content inspection and so on.

Any ETA on that, btw?

Adam


From pasik at iki.fi  Thu Mar  8 20:10:59 2012
From: pasik at iki.fi (Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?=)
Date: Thu, 8 Mar 2012 22:10:59 +0200
Subject: Manipulating the body of a PUT/POST
In-Reply-To: <4F58BBBC.9000903@one.com>
References: <4F2B9FEE.3060601@one.com> <20120203100450.GH67687@mdounin.ru>
	<4F58BBBC.9000903@one.com>
Message-ID: <20120308201058.GZ12984@reaktio.net>

On Thu, Mar 08, 2012 at 03:01:32PM +0100, Adam Hasselbalch Hansen wrote:
> On 2012-02-03 11:04, Maxim Dounin wrote:
>
>> I'm working on prototyping input body filtering, it's expected to
>> appear somewhere in 1.2.x.  It will allow manipulation of request
>> body, as well as other tasks like content inspection and so on.
>
> Any ETA on that, btw?
>

also does this prototyping work allow to bypass nginx local saving of put/post requests to a temp file?
I'm working on disabling nginx local caching of uploads..

-- Pasi


From list at pengo.org  Fri Mar  9 06:20:10 2012
From: list at pengo.org (Peter Halasz)
Date: Fri, 9 Mar 2012 17:20:10 +1100
Subject: Please add HTML support for http_xslt_module (there's an nginx fork
	which has it already)
Message-ID: <CABLXxgF=z8X2JEJ3CP8rZ-UN_Hh1TM6R5PFQNE4tV2Q0NFZ6ig@mail.gmail.com>

Hi devs,

I work for an environmental not-for-profit organisation where we use
XSLT to theme our website. (The XSLT is generated by Diazo, and the
site largely runs on Plone).

Currently we use Nginx to do the XSLT transformation. There's a
problem though, that our un-themed site doesn't come out as perfect
XML, so we need an XSLT parser which can transform HTML (not just
XML). Nginx's http_xslt_module does NOT currently support HTML
parsing, and I'd really like to see this feature added.

The problem isn't specific to Diazo, but the Diazo manual explains the
need for HTML parsing:

> In theory, any XSLT processor will do. In practice, however, most websites do not produce 100% well-formed XML (i.e. they do not conform to the XHTML ?strict? doctype). For this reason, it is normally necessary to use an XSLT processor that will parse the content using a more lenient parser with some knowledge of HTML. libxml2, the most popular XML processing library on Linux and similar operating systems, contains such a parser.

Fortunately there's a fork of nginx which does use libxml2: the
xslt_html project <http://code.google.com/p/html-xslt/>.
Unfortunately, the project is not maintained, so it ties us to a
patched version of nginx 0.7.67 (circa June 2010). I'd like to upgrade
nginx -- I've hit nginx bugs that were fixed long ago. I'm sure there
are many other nginx users with the same needs, so I'm requesting the
fork's changes make their way into the mainline. I'm assuming it's
just been forgotten.

The Diazo documentation also explains deploying with this patched Nginx:

> To deploy an Diazo theme to the Nginx web server, you will need to compile Nginx with a special version of the XSLT module that can (optionally) use the HTML parser from libxml2.

> In the future, the necessary patches to enable HTML mode parsing will hopefully be part of the standard Nginx distribution. In the meantime, they are maintained in the html-xslt project.

We're using this html-xslt fork of nginx at my organisation. But
unfortunately, it's not maintained, and the functionality hasn't made
it into the standard Nginx distribution. Can we please include it?

The fork adds the directive: "xslt_html_parser on;" which causes the
http_xslt_module to parse in HTML mode.

I've just made a diff <http://pastebin.com/CP1P8Gzj> to see what the
fork changes, and it's 755 lines long. (That's a bit longer than I
expected)

The files modified by the html-xslt fork are:

       src/http/modules/ngx_http_xslt_filter_module.c
       src/http/ngx_http_variables.c
       auto/options
       auto/lib/libxslt/conf

The diff is against nginx 0.7.67. Since then the
ngx_http_xslt_filter_module.c has seen about 300 lines removed and 20
lines added or changed, so obviously the diff can't be used as a patch
against the current version of nginx.

Hopefully that's more than enough info to get started if developers
are interested in folding the fork into nginx.

I know the other solution to our problem here is to move the XSLT to
another layer of the stack -- such as Varnish or Apache -- but I want
to make sure nginx devs know about the feature they're missing first.

Thanks for listening and I hope HTML parsing for XSLT can make it to
the mainline of nginx,

Peter Halasz.


From mdounin at mdounin.ru  Fri Mar  9 13:18:39 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 9 Mar 2012 17:18:39 +0400
Subject: Manipulating the body of a PUT/POST
In-Reply-To: <4F58BBBC.9000903@one.com>
References: <4F2B9FEE.3060601@one.com> <20120203100450.GH67687@mdounin.ru>
	<4F58BBBC.9000903@one.com>
Message-ID: <20120309131839.GH67687@mdounin.ru>

Hello!

On Thu, Mar 08, 2012 at 03:01:32PM +0100, Adam Hasselbalch Hansen wrote:

> On 2012-02-03 11:04, Maxim Dounin wrote:
> 
> >I'm working on prototyping input body filtering, it's expected to
> >appear somewhere in 1.2.x.  It will allow manipulation of request
> >body, as well as other tasks like content inspection and so on.
> 
> Any ETA on that, btw?

No ETA yet.

Maxim Dounin


From mdounin at mdounin.ru  Fri Mar  9 14:23:08 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 9 Mar 2012 18:23:08 +0400
Subject: Manipulating the body of a PUT/POST
In-Reply-To: <20120308201058.GZ12984@reaktio.net>
References: <4F2B9FEE.3060601@one.com> <20120203100450.GH67687@mdounin.ru>
	<4F58BBBC.9000903@one.com> <20120308201058.GZ12984@reaktio.net>
Message-ID: <20120309142308.GI67687@mdounin.ru>

Hello!

On Thu, Mar 08, 2012 at 10:10:59PM +0200, Pasi K?rkk?inen wrote:

> On Thu, Mar 08, 2012 at 03:01:32PM +0100, Adam Hasselbalch Hansen wrote:
> > On 2012-02-03 11:04, Maxim Dounin wrote:
> >
> >> I'm working on prototyping input body filtering, it's expected to
> >> appear somewhere in 1.2.x.  It will allow manipulation of request
> >> body, as well as other tasks like content inspection and so on.
> >
> > Any ETA on that, btw?
> >
> 
> also does this prototyping work allow to bypass nginx local saving of put/post requests to a temp file?
> I'm working on disabling nginx local caching of uploads..

It's considered as a "nice to have somewhere in the future", but 
not as the goal of this particular work.

Maxim Dounin


From l at lrowe.co.uk  Fri Mar  9 23:21:52 2012
From: l at lrowe.co.uk (Laurence Rowe)
Date: Fri, 9 Mar 2012 23:21:52 +0000
Subject: Please add HTML support for http_xslt_module (there's an nginx
	fork which has it already)
In-Reply-To: <CABLXxgF=z8X2JEJ3CP8rZ-UN_Hh1TM6R5PFQNE4tV2Q0NFZ6ig@mail.gmail.com>
References: <CABLXxgF=z8X2JEJ3CP8rZ-UN_Hh1TM6R5PFQNE4tV2Q0NFZ6ig@mail.gmail.com>
Message-ID: <CAOycyLTNZRjpeCytNUY5aShDYwr3bWjBc_Mzs4t99OJVo1pZ-w@mail.gmail.com>

On 9 March 2012 06:20, Peter Halasz <list at pengo.org> wrote:
> Hi devs,
>
> I work for an environmental not-for-profit organisation where we use
> XSLT to theme our website. (The XSLT is generated by Diazo, and the
> site largely runs on Plone).
>
> Currently we use Nginx to do the XSLT transformation. There's a
> problem though, that our un-themed site doesn't come out as perfect
> XML, so we need an XSLT parser which can transform HTML (not just
> XML). Nginx's http_xslt_module does NOT currently support HTML
> parsing, and I'd really like to see this feature added.
>
> The problem isn't specific to Diazo, but the Diazo manual explains the
> need for HTML parsing:
>
>> In theory, any XSLT processor will do. In practice, however, most websites do not produce 100% well-formed XML (i.e. they do not conform to the XHTML ?strict? doctype). For this reason, it is normally necessary to use an XSLT processor that will parse the content using a more lenient parser with some knowledge of HTML. libxml2, the most popular XML processing library on Linux and similar operating systems, contains such a parser.
>
> Fortunately there's a fork of nginx which does use libxml2: the
> xslt_html project <http://code.google.com/p/html-xslt/>.
> Unfortunately, the project is not maintained, so it ties us to a
> patched version of nginx 0.7.67 (circa June 2010). I'd like to upgrade
> nginx -- I've hit nginx bugs that were fixed long ago. I'm sure there
> are many other nginx users with the same needs, so I'm requesting the
> fork's changes make their way into the mainline. I'm assuming it's
> just been forgotten.
>
> The Diazo documentation also explains deploying with this patched Nginx:
>
>> To deploy an Diazo theme to the Nginx web server, you will need to compile Nginx with a special version of the XSLT module that can (optionally) use the HTML parser from libxml2.
>
>> In the future, the necessary patches to enable HTML mode parsing will hopefully be part of the standard Nginx distribution. In the meantime, they are maintained in the html-xslt project.
>
> We're using this html-xslt fork of nginx at my organisation. But
> unfortunately, it's not maintained, and the functionality hasn't made
> it into the standard Nginx distribution. Can we please include it?
>
> The fork adds the directive: "xslt_html_parser on;" which causes the
> http_xslt_module to parse in HTML mode.
>
> I've just made a diff <http://pastebin.com/CP1P8Gzj> to see what the
> fork changes, and it's 755 lines long. (That's a bit longer than I
> expected)
>
> The files modified by the html-xslt fork are:
>
> ? ? ? src/http/modules/ngx_http_xslt_filter_module.c
> ? ? ? src/http/ngx_http_variables.c
> ? ? ? auto/options
> ? ? ? auto/lib/libxslt/conf
>
> The diff is against nginx 0.7.67. Since then the
> ngx_http_xslt_filter_module.c has seen about 300 lines removed and 20
> lines added or changed, so obviously the diff can't be used as a patch
> against the current version of nginx.
>
> Hopefully that's more than enough info to get started if developers
> are interested in folding the fork into nginx.
>
> I know the other solution to our problem here is to move the XSLT to
> another layer of the stack -- such as Varnish or Apache -- but I want
> to make sure nginx devs know about the feature they're missing first.
>
> Thanks for listening and I hope HTML parsing for XSLT can make it to
> the mainline of nginx,

Thanks for bringing this up, I needed a bit of encouragement to get
back to this!

Some of the foundational components of the patch were merged, see:
http://mailman.nginx.org/pipermail/nginx-devel/2010-July/000390.html

I've updated the patch to the current version here:
https://bitbucket.org/lrowe/nginx-xslt-html-parser. It includes the
recent xslt_param work from:
http://mailman.nginx.org/pipermail/nginx-devel/2012-March/001926.html.
I've left out the autoconf changes supporting custom libxml2/libxslt
builds, they didn't work terribly well anyway.

The only significant omissions/changes from a Diazo perspective (when
compared to the nginx 0.7 patch) are:

* The lack of automatic uri unescaping of xslt parameters. This was
always a hack. If you need the SSI filter_xpath stuff from
http://docs.diazo.org/en/latest/deployment.html#including-external-content-with-ssi
then you will need to unquote the parameter value using inline perl
(didn't build on my Mac) or set_uri_unescape from
https://github.com/agentzh/set-misc-nginx-module.

* As with the current with standard nginx xslt module, responses with
a text/xml mime type will always have the stylesheet applied (and
parsed with the HTMLParser if that is switched on.) I'm not sure what
the best way to handle this is without breaking backwards
compatibility. We need something like an `xslt_unregister_types`
directive, but I'm not sure how to go about implementing that. Or
maybe a variable to set to disable xslt application.

* HTML documents are now no longer assumed to always have a utf-8
charset. You should ensure the charset is set on the response if the
HTML is utf-8 encoded, otherwise it leaves it up to the autodetection
that normally assumed Latin-1.

Anyway, I'd appreciate some testing and assuming it works I'll submit
the patches to this list.

Laurence


From list at pengo.org  Sun Mar 11 03:45:35 2012
From: list at pengo.org (Peter Halasz)
Date: Sun, 11 Mar 2012 14:45:35 +1100
Subject: Please add HTML support for http_xslt_module (there's an nginx
	fork which has it already)
In-Reply-To: <CAOycyLTNZRjpeCytNUY5aShDYwr3bWjBc_Mzs4t99OJVo1pZ-w@mail.gmail.com>
References: <CABLXxgF=z8X2JEJ3CP8rZ-UN_Hh1TM6R5PFQNE4tV2Q0NFZ6ig@mail.gmail.com>
	<CAOycyLTNZRjpeCytNUY5aShDYwr3bWjBc_Mzs4t99OJVo1pZ-w@mail.gmail.com>
Message-ID: <CABLXxgHLVdWDNaFu_ZjL6C12vQQ-XNPhu9xR246=Ha8+Xj6HUA@mail.gmail.com>

Hi Laurence Rowe,

Thanks for the update! I'm using your newly patched version without
any problems. I've just set it up in our buildout as a drop in
replacement for the old version:

[nginx]
recipe = zc.recipe.cmmi
# url = http://html-xslt.googlecode.com/files/nginx-0.7.67-html-xslt-4.tar.gz
# old nginx
url = https://bitbucket.org/lrowe/nginx-xslt-html-parser/get/7145bd8cc1e2.tar.gz

and left everything else the same and it's working well on our staging
server right now.

We have a pretty straight forward setup (no SSI and our encoding is
utf8), so I guess those cases may still need testing, but it's working
well for me.

Thanks again,

Peter Halasz

P.S. I've started a rough Wikipedia page on Diazo. Hopefully it
doesn't have too many mistakes:
http://en.wikipedia.org/wiki/Diazo_%28software%29


From ru at nginx.com  Sun Mar 11 13:33:03 2012
From: ru at nginx.com (ru at nginx.com)
Date: Sun, 11 Mar 2012 13:33:03 +0000
Subject: [nginx] svn commit:  r4525 - trunk
Message-ID: <20120311133303.9638D3F9DAB@mail.nginx.com>

Author: ru
Date: 2012-03-11 13:33:03 +0000 (Sun, 11 Mar 2012)
New Revision: 4525

Log:
Added scgi_temp and uwsgi_temp to svn:ignore.


Modified:
   trunk/


Property changes on: trunk
___________________________________________________________________
Modified: svn:ignore
   - access.log
client_body_temp
fastcgi_temp
proxy_temp
GNUmakefile
Makefile
makefile
nginx
nginx.conf
nginx-*.tar.gz
objs*
tmp

   + access.log
client_body_temp
fastcgi_temp
proxy_temp
scgi_temp
uwsgi_temp
GNUmakefile
Makefile
makefile
nginx
nginx.conf
nginx-*.tar.gz
objs*
tmp



From rob.stradling at comodo.com  Mon Mar 12 10:12:02 2012
From: rob.stradling at comodo.com (Rob Stradling)
Date: Mon, 12 Mar 2012 10:12:02 +0000
Subject: Does anyone plan to develop the feature of openssl' OCSP stapling?
In-Reply-To: <20110616150212.GA38841@sysoev.ru>
References: <201106161430.55434.rob.stradling@comodo.com>
	<20110616150212.GA38841@sysoev.ru>
Message-ID: <4F5DCBF2.6020107@comodo.com>

"As to OCSP, I'm going to implement it in the next 2.0 version."

Igor, is this still your plan?
If so, do you have a (very rough) ETA for nginx v2.0?
(I don't see v2.0 mentioned here: http://trac.nginx.org/nginx/roadmap)

Or are there any other regular nginx developers who could work on OCSP 
Stapling?

Or is it worth me having a go at writing a patch?

Thanks!

On 16/06/11 16:02, Igor Sysoev wrote:
> On Thu, Jun 16, 2011 at 02:30:55PM +0100, Rob Stradling wrote:
>> On November 25, 2010 04:42AM Weibin Yao wrote:
>>> Hi everyone,
>>>
>>> I think the the feature of OCSP stapling[1] is very useful for the
>>> browser blocked by the OCSP request. And the feature has supported since
>>> openssl 0.9.8g. Apache's mod_ssl has also added this patch in the
>>> development branch[2].
>>>
>>> Does anyone have the plan to develop this feature?
>>
>> Hi.  The CAs and Browsers represented in the CA/Browser Forum
>> (http://cabforum.org/forum.html) are growing increasingly interested in
>> encouraging wider adoption of OCSP Stapling.
>>
>> Since nobody else has replied to this thread, I presume that OCSP Stapling is
>> not currently a priority for the core nginx developers.  So, I've started
>> having a go at writing a patch.  I'm basing it heavily on Dr Steve Henson's
>> OCSP Stapling code that was first included in Apache httpd 2.3.3 [3].  I'd like
>> to ask a few questions before I proceed any further:
>>
>>    1. If I am able to complete my patch, are you likely to review/commit it?
>> Or is OCSP Stapling the sort of feature that you'd prefer to only let a core
>> nginx developer work on?
>>
>>    2. I was under the impression that nginx started life as a fork of Apache
>> httpd, but I don't see any messages along the lines of "This product includes
>> software developed by the Apache Group..." in the source code.  Is nginx 100%
>> *not* a derivative work of Apache httpd?
>>
>>    3. Steve Henson's code is presumably licensed under ASL 2.0 [4], which
>> presumably means that my patch would be classed as a "Derivative Work" subject
>> to various conditions (see the "4. Redistribution" section in ASL 2.0).  Would
>> this prevent you from accepting it?
>>
>> (Since ASL 2.0 says "nothing herein shall supersede or modify the terms of any
>> separate license agreement you may have executed with Licensor regarding such
>> Contributions", perhaps I should ask Steve Henson if he would be willing to
>> contribute the same code to nginx under a different licence).
>
> nginx is not Apache fork. I know well enough Apache 1.3 and I've got some
> ideas from Apache such as memory pools, configuration methods, processing
> phases, etc., but there is no line of Apache code. As to OCSP, I'm going
> to implement it in the next 2.0 version.
>
>

-- 
Rob Stradling
Senior Research & Development Scientist
COMODO - Creating Trust Online
Office Tel: +44.(0)1274.730505
Office Fax: +44.(0)1274.730909
www.comodo.com

COMODO CA Limited, Registered in England No. 04058690
Registered Office:
   3rd Floor, 26 Office Village, Exchange Quay,
   Trafford Road, Salford, Manchester M5 3EQ

This e-mail and any files transmitted with it are confidential and 
intended solely for the use of the individual or entity to whom they are 
addressed.  If you have received this email in error please notify the 
sender by replying to the e-mail containing this attachment. Replies to 
this email may be monitored by COMODO for operational or business 
reasons. Whilst every endeavour is taken to ensure that e-mails are free 
from viruses, no liability can be accepted and the recipient is 
requested to use their own virus checking software.


From ru at nginx.com  Mon Mar 12 12:29:57 2012
From: ru at nginx.com (ru at nginx.com)
Date: Mon, 12 Mar 2012 12:29:57 +0000
Subject: [nginx] svn commit:  r4526 - trunk/docs/man
Message-ID: <20120312122957.CBA593FA44B@mail.nginx.com>

Author: ru
Date: 2012-03-12 12:29:56 +0000 (Mon, 12 Mar 2012)
New Revision: 4526

Log:
Mentioned the NGINX environment variable.


Modified:
   trunk/docs/man/nginx.8

Modified: trunk/docs/man/nginx.8
===================================================================
--- trunk/docs/man/nginx.8	2012-03-11 13:33:03 UTC (rev 4525)
+++ trunk/docs/man/nginx.8	2012-03-12 12:29:56 UTC (rev 4526)
@@ -152,6 +152,12 @@
 	debug_connection 127.0.0.1;
 }
 .Ed
+.Sh ENVIRONMENT
+The
+.Ev NGINX
+environment variable is used internally by
+.Nm
+and should not be set directly by the user.
 .Sh FILES
 .Bl -tag -width indent
 .It Pa %%PID_PATH%%
@@ -189,7 +195,7 @@
 .An -nosplit
 .An Igor Sysoev Aq igor at sysoev.ru .
 .Pp
-This manual page was written by
+This manual page was originally written by
 .An Sergey A. Osokin Aq osa at FreeBSD.org.ru
 as a result of compiling many
 .Nm


From goelvivek2011 at gmail.com  Tue Mar 13 07:11:04 2012
From: goelvivek2011 at gmail.com (vivek goel)
Date: Tue, 13 Mar 2012 12:41:04 +0530
Subject: Hashing a header in nginx
Message-ID: <CAB+eYNVQb0jQGu-++MtTfxvt8eMa7xX3ycNRSemo-A6XVAGg1w@mail.gmail.com>

I was reading article
http://wiki.nginx.org/HeadersManagement#headers_in_and_proxy_pass

I am developing a custom nginx module. What is the fastest way to look for
custom header ?
Can I add it to headers_in object ?
Or
A way to hash it for fast lookup


regards
Vivek Goel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120313/62481930/attachment.html>

From mattjd at gmail.com  Tue Mar 13 07:34:33 2012
From: mattjd at gmail.com (Matthew Daley)
Date: Tue, 13 Mar 2012 20:34:33 +1300
Subject: [PATCH] perl module: fix SSI parameter termination bug
Message-ID: <CA+nUz_+2LGCR-KS_uQgOtj7g610pxYOGzf7N+L1cohzY08Fg=A@mail.gmail.com>

Hi,

There is a small issue in the handling of the SSI command provided by
the Perl module.

It NULL-terminates the 'sub' parameter value before passing it to
ngx_http_perl_eval_anon_sub, but this has the problem that if the
parameter's length is the maximum allowable for SSI commands, it will
write the NULL byte just past the end of the allocated buffer:

==14669== Invalid write of size 1
==14669==    at 0x80A79E3: ngx_http_perl_ssi (ngx_http_perl_module.c:384)
==14669==    by 0x80904CF: ngx_http_ssi_body_filter
(ngx_http_ssi_filter_module.c:794)
==14669==    by 0x8092B73: ngx_http_charset_body_filter
(ngx_http_charset_filter_module.c:553)
==14669==    by 0x8055F3F: ngx_output_chain (ngx_output_chain.c:206)
==14669==    by 0x807D09E: ngx_http_copy_filter
(ngx_http_copy_filter_module.c:142)
==14669==    by 0x808A288: ngx_http_range_body_filter
(ngx_http_range_filter_module.c:559)
==14669==    by 0x8071758: ngx_http_output_filter (ngx_http_core_module.c:1903)
==14669==    by 0x8089459: ngx_http_static_handler
(ngx_http_static_module.c:266)
==14669==    by 0x807598C: ngx_http_core_content_phase
(ngx_http_core_module.c:1394)
==14669==    by 0x80713F4: ngx_http_core_run_phases (ngx_http_core_module.c:877)
==14669==    by 0x80714ED: ngx_http_handler (ngx_http_core_module.c:860)
==14669==    by 0x807988F: ngx_http_process_request (ngx_http_request.c:1668)
==14669==  Address 0x44bd120 is 0 bytes after a block of size 256 alloc'd
==14669==    at 0x4023F50: malloc (vg_replace_malloc.c:236)
==14669==    by 0x806979B: ngx_alloc (ngx_alloc.c:22)
==14669==    by 0x8053BDC: ngx_malloc (ngx_palloc.c:149)
==14669==    by 0x8053D0A: ngx_pnalloc (ngx_palloc.c:183)
==14669==    by 0x808F4A8: ngx_http_ssi_body_filter
(ngx_http_ssi_filter_module.c:1206)
==14669==    by 0x8092B73: ngx_http_charset_body_filter
(ngx_http_charset_filter_module.c:553)
==14669==    by 0x8055F3F: ngx_output_chain (ngx_output_chain.c:206)
==14669==    by 0x807D09E: ngx_http_copy_filter
(ngx_http_copy_filter_module.c:142)
==14669==    by 0x808A288: ngx_http_range_body_filter
(ngx_http_range_filter_module.c:559)
==14669==    by 0x8071758: ngx_http_output_filter (ngx_http_core_module.c:1903)
==14669==    by 0x8089459: ngx_http_static_handler
(ngx_http_static_module.c:266)
==14669==    by 0x807598C: ngx_http_core_content_phase
(ngx_http_core_module.c:1394)

I don't believe this to have any security impact, as if you already
have the ability to inject Perl SSI commands, you already have the
Perl runtime for any malicious intent.

I have attached a patch which attempts to fix the problem by creating
an appropriately-sized buffer and NULL-terminating a copy of the
string to eval.

- Matthew Daley
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nginx-perl.patch
Type: application/octet-stream
Size: 2953 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120313/81c2898d/attachment.obj>

From mdounin at mdounin.ru  Tue Mar 13 08:28:02 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 13 Mar 2012 12:28:02 +0400
Subject: [PATCH] perl module: fix SSI parameter termination bug
In-Reply-To: <CA+nUz_+2LGCR-KS_uQgOtj7g610pxYOGzf7N+L1cohzY08Fg=A@mail.gmail.com>
References: <CA+nUz_+2LGCR-KS_uQgOtj7g610pxYOGzf7N+L1cohzY08Fg=A@mail.gmail.com>
Message-ID: <20120313082801.GN67687@mdounin.ru>

Hello!

On Tue, Mar 13, 2012 at 08:34:33PM +1300, Matthew Daley wrote:

> Hi,
> 
> There is a small issue in the handling of the SSI command provided by
> the Perl module.
> 
> It NULL-terminates the 'sub' parameter value before passing it to
> ngx_http_perl_eval_anon_sub, but this has the problem that if the
> parameter's length is the maximum allowable for SSI commands, it will
> write the NULL byte just past the end of the allocated buffer:
> 
> ==14669== Invalid write of size 1
> ==14669==    at 0x80A79E3: ngx_http_perl_ssi (ngx_http_perl_module.c:384)
> ==14669==    by 0x80904CF: ngx_http_ssi_body_filter (ngx_http_ssi_filter_module.c:794)

[...]

> I have attached a patch which attempts to fix the problem by creating
> an appropriately-sized buffer and NULL-terminating a copy of the
> string to eval.

Nice catch, thanks.  The patch is wrong though, see below.

[...]

> @@ -381,7 +382,6 @@
>      ctx->ssi = ssi_ctx;
>  
>      handler = params[NGX_HTTP_PERL_SSI_SUB];
> -    handler->data[handler->len] = '\0';
>  
>      {
>  
> @@ -392,7 +392,7 @@
>  
>      /* the code is disabled to force the precompiled perl code using only */
>  
> -    ngx_http_perl_eval_anon_sub(aTHX_ handler, &sv);
> +    ngx_http_perl_eval_anon_sub(aTHX_ r->pool, handler, &sv);

This code is indeed disabled, as stated in the comment above, it's 
under #if 0.

The null-terminated string in handler is required for 

    sv = newSVpvn((char *) handler->data, handler->len);

below though, and your patch will break it.

Correct solution would be to reserve space for NUL character in 
ssi module instead.  Quick and dirty patch which should work:

--- a/src/http/modules/ngx_http_ssi_filter_module.c
+++ b/src/http/modules/ngx_http_ssi_filter_module.c
@@ -1204,7 +1204,7 @@ ngx_http_ssi_parse(ngx_http_request_t *r
 
                 if (ctx->value_buf == NULL) {
                     ctx->param->value.data = ngx_pnalloc(r->pool,
-                                                         ctx->value_len);
+                                                         ctx->value_len + 1);
                     if (ctx->param->value.data == NULL) {
                         return NGX_ERROR;
                     }

Maxim Dounin


From mattjd at gmail.com  Tue Mar 13 08:43:25 2012
From: mattjd at gmail.com (Matthew Daley)
Date: Tue, 13 Mar 2012 21:43:25 +1300
Subject: [PATCH] perl module: fix SSI parameter termination bug
In-Reply-To: <20120313082801.GN67687@mdounin.ru>
References: <CA+nUz_+2LGCR-KS_uQgOtj7g610pxYOGzf7N+L1cohzY08Fg=A@mail.gmail.com>
	<20120313082801.GN67687@mdounin.ru>
Message-ID: <CA+nUz_L2TbpYXMjGnENLydKaOTC51MmYS91vA=Fft2Azapf4Mg@mail.gmail.com>

On Tue, Mar 13, 2012 at 9:28 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> Hello!
>
> On Tue, Mar 13, 2012 at 08:34:33PM +1300, Matthew Daley wrote:
>
>> Hi,
>>
>> There is a small issue in the handling of the SSI command provided by
>> the Perl module.
>>
>> It NULL-terminates the 'sub' parameter value before passing it to
>> ngx_http_perl_eval_anon_sub, but this has the problem that if the
>> parameter's length is the maximum allowable for SSI commands, it will
>> write the NULL byte just past the end of the allocated buffer:
>>
>> ==14669== Invalid write of size 1
>> ==14669== ? ?at 0x80A79E3: ngx_http_perl_ssi (ngx_http_perl_module.c:384)
>> ==14669== ? ?by 0x80904CF: ngx_http_ssi_body_filter (ngx_http_ssi_filter_module.c:794)
>
> [...]
>
>> I have attached a patch which attempts to fix the problem by creating
>> an appropriately-sized buffer and NULL-terminating a copy of the
>> string to eval.
>
> Nice catch, thanks. ?The patch is wrong though, see below.
>
> [...]
>
>> @@ -381,7 +382,6 @@
>> ? ? ?ctx->ssi = ssi_ctx;
>>
>> ? ? ?handler = params[NGX_HTTP_PERL_SSI_SUB];
>> - ? ?handler->data[handler->len] = '\0';
>>
>> ? ? ?{
>>
>> @@ -392,7 +392,7 @@
>>
>> ? ? ?/* the code is disabled to force the precompiled perl code using only */
>>
>> - ? ?ngx_http_perl_eval_anon_sub(aTHX_ handler, &sv);
>> + ? ?ngx_http_perl_eval_anon_sub(aTHX_ r->pool, handler, &sv);
>
> This code is indeed disabled, as stated in the comment above, it's
> under #if 0.

Yeah, just wanted to make sure that it'd still work if re-enabled.

>
> The null-terminated string in handler is required for
>
> ? ?sv = newSVpvn((char *) handler->data, handler->len);
>
> below though, and your patch will break it.

Hmm, I was going by this info in the perlapi documentation, which made
me think that newSVpvn didn't need it:
"Notice that you can choose to specify the length of the string to be
assigned by using sv_setpvn , newSVpvn , or newSVpv , or you may allow
Perl to calculate the length by using sv_setpv or by specifying 0 as
the second argument to newSVpv . Be warned, though, that Perl will
determine the string's length by using strlen , which depends on the
string terminating with a NUL character."

However I see below that it says:
"All SVs that contain strings should be terminated with a NUL
character. If it is not NUL-terminated there is a risk of core dumps
and corruptions from code which passes the string to C functions or
system calls which expect a NUL-terminated string. Perl's own
functions typically add a trailing NUL for this reason."

So you are right.

>
> Correct solution would be to reserve space for NUL character in
> ssi module instead. ?Quick and dirty patch which should work:
>
> --- a/src/http/modules/ngx_http_ssi_filter_module.c
> +++ b/src/http/modules/ngx_http_ssi_filter_module.c
> @@ -1204,7 +1204,7 @@ ngx_http_ssi_parse(ngx_http_request_t *r
>
> ? ? ? ? ? ? ? ? if (ctx->value_buf == NULL) {
> ? ? ? ? ? ? ? ? ? ? ctx->param->value.data = ngx_pnalloc(r->pool,
> - ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ctx->value_len);
> + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ctx->value_len + 1);
> ? ? ? ? ? ? ? ? ? ? if (ctx->param->value.data == NULL) {
> ? ? ? ? ? ? ? ? ? ? ? ? return NGX_ERROR;
> ? ? ? ? ? ? ? ? ? ? }
>

I wasn't sure if changing the SSI allocation code was appropriate in
this case, but since newSVpvn really does need the NULL termination,
this is indeed a better idea.


From zzz at zzz.org.ua  Tue Mar 13 14:16:30 2012
From: zzz at zzz.org.ua (Alexandr Gomoliako)
Date: Tue, 13 Mar 2012 16:16:30 +0200
Subject: [PATCH] perl module: fix SSI parameter termination bug
In-Reply-To: <20120313082801.GN67687@mdounin.ru>
References: <CA+nUz_+2LGCR-KS_uQgOtj7g610pxYOGzf7N+L1cohzY08Fg=A@mail.gmail.com>
	<20120313082801.GN67687@mdounin.ru>
Message-ID: <CAKb9G9hODkWFAkL=qvciQp3P63RokymPqEsSqZ65CnxmjrN0sw@mail.gmail.com>

> The null-terminated string in handler is required for
>
>     sv = newSVpvn((char *) handler->data, handler->len);
>
> below though, and your patch will break it.

This is incorrect, newSVpvn creates a new SV and copies header->len
bytes from header->data into it. It never does strlen() on this data
and always allocates extra byte itself. So, something like this is
fine:

    newSVpvn("\0\0\0", 3);

> Hmm, I was going by this info in the perlapi documentation, which made
> me think that newSVpvn didn't need it:

> "Notice that you can choose to specify the length of the string to be
> assigned by using sv_setpvn , newSVpvn , or newSVpv , or you may allow
> Perl to calculate the length by using sv_setpv or by specifying 0 as
> the second argument to newSVpv . Be warned, though, that Perl will
> determine the string's length by using strlen , which depends on the
> string terminating with a NUL character."

> However I see below that it says:
> "All SVs that contain strings should be terminated with a NUL
> character. If it is not NUL-terminated there is a risk of core dumps
> and corruptions from code which passes the string to C functions or
> system calls which expect a NUL-terminated string. Perl's own
> functions typically add a trailing NUL for this reason."

Notice how it says, that "Perl's own functions typically add a
trailing NUL for this reason.", this includes newSVpvn.

>> Correct solution would be to reserve space for NUL character in
>> ssi module instead.  Quick and dirty patch which should work:

>> --- a/src/http/modules/ngx_http_ssi_filter_module.c
>> +++ b/src/http/modules/ngx_http_ssi_filter_module.c
>> @@ -1204,7 +1204,7 @@ ngx_http_ssi_parse(ngx_http_request_t *r
>>
>>                 if (ctx->value_buf == NULL) {
>>                     ctx->param->value.data = ngx_pnalloc(r->pool,
>> -                                                         ctx->value_len);
>> +                                                         ctx->value_len +
>> 1);
>>                     if (ctx->param->value.data == NULL) {
>>                         return NGX_ERROR;
>>                     }
>>
>
> I wasn't sure if changing the SSI allocation code was appropriate in
> this case, but since newSVpvn really does need the NULL termination,
> this is indeed a better idea.

No, it's not. The only solution is to fix it in ssi module, as
proposed by Maxim.


From zzz at zzz.org.ua  Tue Mar 13 14:29:21 2012
From: zzz at zzz.org.ua (Alexandr Gomoliako)
Date: Tue, 13 Mar 2012 16:29:21 +0200
Subject: [PATCH] perl module: fix SSI parameter termination bug
In-Reply-To: <CAKb9G9hODkWFAkL=qvciQp3P63RokymPqEsSqZ65CnxmjrN0sw@mail.gmail.com>
References: <CA+nUz_+2LGCR-KS_uQgOtj7g610pxYOGzf7N+L1cohzY08Fg=A@mail.gmail.com>
	<20120313082801.GN67687@mdounin.ru>
	<CAKb9G9hODkWFAkL=qvciQp3P63RokymPqEsSqZ65CnxmjrN0sw@mail.gmail.com>
Message-ID: <CAKb9G9iucFSoF1S6u6qExsU1x0GZBukgfTzxB7gWMXakEswr1Q@mail.gmail.com>

> This is incorrect, newSVpvn creates a new SV and copies header->len
> bytes from header->data into it. It never does strlen() on this data
> and always allocates extra byte itself. So, something like this is
> fine:

I meant "handler" there.


From brad at thehiltons.org  Tue Mar 13 21:59:00 2012
From: brad at thehiltons.org (Brad Hilton)
Date: Tue, 13 Mar 2012 14:59:00 -0700
Subject: Does proxy cache support If-Modified-Since to upstream servers?
Message-ID: <CAESeBWR99_7tocScXy-xGH68D_=zH0RUf+gMN7ncA9e5scmzOw@mail.gmail.com>

Hello!

I am planning to use nginx to cache large files dispersed across several
different regions. I have noticed that when a cached file expires, nginx
always retrieves a fresh copy of the upstream file, regardless of whether
it has changed.

I found a thread from September where someone was working on a patch to
address this:

http://mailman.nginx.org/pipermail/nginx-devel/2011-September/001187.html

But it looks like the thread died out without any code being accepted.

Does anyone know if there has been progress on this feature? What I'm
trying to accomplish is:

1) Early detection of upstream file changes to force a cache refresh
2) Avoid fetching duplicate copies of upstream files due to bandwidth
contraints

It seems that as-is, the nginx proxy cache feature may not allow me to do
this.

Thanks for any tips,
Brad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120313/88cb4d30/attachment.html>

From mattjd at gmail.com  Wed Mar 14 00:34:40 2012
From: mattjd at gmail.com (Matthew Daley)
Date: Wed, 14 Mar 2012 13:34:40 +1300
Subject: [PATCH] perl module: fix SSI parameter termination bug
In-Reply-To: <CAKb9G9hODkWFAkL=qvciQp3P63RokymPqEsSqZ65CnxmjrN0sw@mail.gmail.com>
References: <CA+nUz_+2LGCR-KS_uQgOtj7g610pxYOGzf7N+L1cohzY08Fg=A@mail.gmail.com>
	<20120313082801.GN67687@mdounin.ru>
	<CAKb9G9hODkWFAkL=qvciQp3P63RokymPqEsSqZ65CnxmjrN0sw@mail.gmail.com>
Message-ID: <CA+nUz_+GQgJE=gWvEsJbWpsrpwd0nz2YEENDZAOo8p+a4Nf9sQ@mail.gmail.com>

On Wed, Mar 14, 2012 at 3:16 AM, Alexandr Gomoliako <zzz at zzz.org.ua> wrote:
[...]
>
>>> Correct solution would be to reserve space for NUL character in
>>> ssi module instead. ?Quick and dirty patch which should work:
>
>>> --- a/src/http/modules/ngx_http_ssi_filter_module.c
>>> +++ b/src/http/modules/ngx_http_ssi_filter_module.c
>>> @@ -1204,7 +1204,7 @@ ngx_http_ssi_parse(ngx_http_request_t *r
>>>
>>> ? ? ? ? ? ? ? ? if (ctx->value_buf == NULL) {
>>> ? ? ? ? ? ? ? ? ? ? ctx->param->value.data = ngx_pnalloc(r->pool,
>>> - ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ctx->value_len);
>>> + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ctx->value_len +
>>> 1);
>>> ? ? ? ? ? ? ? ? ? ? if (ctx->param->value.data == NULL) {
>>> ? ? ? ? ? ? ? ? ? ? ? ? return NGX_ERROR;
>>> ? ? ? ? ? ? ? ? ? ? }
>>>
>>
>> I wasn't sure if changing the SSI allocation code was appropriate in
>> this case, but since newSVpvn really does need the NULL termination,
>> this is indeed a better idea.
>
> No, it's not. The only solution is to fix it in ssi module, as
> proposed by Maxim.
>

I think you misunderstood me there; I was indeed agreeing that
applying the fix in the SSI module is a better idea, not my original
patch.

Thanks for the info on newSVpvn.


From goelvivek2011 at gmail.com  Wed Mar 14 01:21:39 2012
From: goelvivek2011 at gmail.com (vivek goel)
Date: Wed, 14 Mar 2012 06:51:39 +0530
Subject: Hashing a header in nginx
In-Reply-To: <CAB+eYNVQb0jQGu-++MtTfxvt8eMa7xX3ycNRSemo-A6XVAGg1w@mail.gmail.com>
References: <CAB+eYNVQb0jQGu-++MtTfxvt8eMa7xX3ycNRSemo-A6XVAGg1w@mail.gmail.com>
Message-ID: <CAB+eYNWc784zj8wEhKe4xUGq24YMzgwbs3SXo9giCGOsJyyKMA@mail.gmail.com>

http://stackoverflow.com/questions/5900447/simplest-example-of-using-google-c-testing-framework-with-cmake
regards
Vivek Goel



On Tue, Mar 13, 2012 at 12:41 PM, vivek goel <goelvivek2011 at gmail.com>wrote:

> I was reading article
> http://wiki.nginx.org/HeadersManagement#headers_in_and_proxy_pass
>
> I am developing a custom nginx module. What is the fastest way to look for
> custom header ?
> Can I add it to headers_in object ?
> Or
> A way to hash it for fast lookup
>
>
> regards
> Vivek Goel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120314/58f9c1bd/attachment.html>

From l at lrowe.co.uk  Wed Mar 14 17:03:56 2012
From: l at lrowe.co.uk (Laurence Rowe)
Date: Wed, 14 Mar 2012 17:03:56 +0000
Subject: HTML parsing support for xslt module
Message-ID: <CAOycyLQHaKkehFm_mO3HEJh9H4Y+dii4kuLKWRVAVtw4b4r3tg@mail.gmail.com>

I'd like to submit the attached patch which implements an
``xslt_html_parser`` directive for consideration. When enabled, the
xslt module uses the libxml2 HTMLParser to parse the response body.
This is useful for people who want to transform HTML using XSLT,
including users of Diazo deploying on Nginx
(http://docs.diazo.org/en/latest/deployment.html#nginx).

The patch is generated from my repository at
https://bitbucket.org/lrowe/nginx-xslt-html-parser, forked from
http://mdounin.ru/hg/nginx-vendor-current/. The xslt_param patch from
http://mailman.nginx.org/pipermail/nginx-devel/2012-March/001926.html
is included in my repository. I'll discuss each of the individual
changesets briefly below:

changeset:   668:bf4d14f51436
user:        Laurence Rowe <laurence at lrowe.co.uk>
date:        Sun Jul 11 23:54:08 2010 +0100
summary:     Skip transform when there is no content (e.g. a proxied redirect)

This was originally reviewed in
http://mailman.nginx.org/pipermail/nginx-devel/2010-July/000390.html:

> Currently the way to disable XSLT processing is MIME type, "text/xml"
> by default. Redirects usually have "text/html" type.

To parse HTML you have to set ``xslt_types text/html;``. This
changeset prevents crashing on responses with an empty body.

changeset:   669:9487b3a0e3ff
user:        Laurence Rowe <laurence at lrowe.co.uk>
date:        Fri Mar 09 21:01:25 2012 +0000
summary:     Use xmlCtxtUseOptions to set options.

This changeset moves most option setting to the xmlCtxtUseOptions
(foundational to next commit.)

changeset:   670:c8349ca87381
user:        Laurence Rowe <laurence at lrowe.co.uk>
date:        Fri Mar 09 21:08:18 2012 +0000
summary:     XML_PARSE_COMPACT to save memory

We can use XML_PARSE_COMPACT as the parsed input document is not
modified (XSLT creates a new result document.)

changeset:   671:7145bd8cc1e2
tag:         tip
user:        Laurence Rowe <laurence at lrowe.co.uk>
date:        Fri Mar 09 21:47:46 2012 +0000
summary:     xslt_html_parser

This changeset adds the ``xslt_html_parser`` directive and uses the
HTMLParser when it is set. HTML parsing is performed with
HTML_PARSE_RECOVER as real-world HTML may not be well formed, error
handling is thus disabled when this option is set.


Laurence
-------------- next part --------------
A non-text attachment was scrubbed...
Name: xslt_html_parser.diff
Type: application/octet-stream
Size: 9209 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120314/1f865912/attachment.obj>

From mdounin at mdounin.ru  Thu Mar 15 01:57:10 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 15 Mar 2012 01:57:10 +0000
Subject: [nginx] svn commit:  r4527 - trunk/misc
Message-ID: <20120315015710.CB5973FA9C3@mail.nginx.com>

Author: mdounin
Date: 2012-03-15 01:57:09 +0000 (Thu, 15 Mar 2012)
New Revision: 4527

Log:
Updated OpenSSL and PCRE used for win32 builds.


Modified:
   trunk/misc/GNUmakefile

Modified: trunk/misc/GNUmakefile
===================================================================
--- trunk/misc/GNUmakefile	2012-03-12 12:29:56 UTC (rev 4526)
+++ trunk/misc/GNUmakefile	2012-03-15 01:57:09 UTC (rev 4527)
@@ -6,9 +6,9 @@
 REPO =		$(shell svn info | sed -n 's/^Repository Root: //p')
 
 OBJS =		objs.msvc8
-OPENSSL =	openssl-1.0.0g
+OPENSSL =	openssl-1.0.0h
 ZLIB =		zlib-1.2.5
-PCRE =		pcre-8.21
+PCRE =		pcre-8.30
 
 
 release:


From mdounin at mdounin.ru  Thu Mar 15 11:21:54 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 15 Mar 2012 11:21:54 +0000
Subject: [nginx] svn commit:  r4528 - trunk/src/http/modules
Message-ID: <20120315112154.9F12C3FA932@mail.nginx.com>

Author: mdounin
Date: 2012-03-15 11:21:54 +0000 (Thu, 15 Mar 2012)
New Revision: 4528

Log:
Uwsgi: merged r->http_version fixes from scgi module.

Fixed incorrect use of r->http_version (r4372).  Removed duplicate function
declaration (r4373).  Removed error if there is no Status header (r4374).


Modified:
   trunk/src/http/modules/ngx_http_uwsgi_module.c

Modified: trunk/src/http/modules/ngx_http_uwsgi_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_uwsgi_module.c	2012-03-15 01:57:09 UTC (rev 4527)
+++ trunk/src/http/modules/ngx_http_uwsgi_module.c	2012-03-15 11:21:54 UTC (rev 4528)
@@ -43,7 +43,6 @@
 static ngx_int_t ngx_http_uwsgi_reinit_request(ngx_http_request_t *r);
 static ngx_int_t ngx_http_uwsgi_process_status_line(ngx_http_request_t *r);
 static ngx_int_t ngx_http_uwsgi_process_header(ngx_http_request_t *r);
-static ngx_int_t ngx_http_uwsgi_process_header(ngx_http_request_t *r);
 static void ngx_http_uwsgi_abort_request(ngx_http_request_t *r);
 static void ngx_http_uwsgi_finalize_request(ngx_http_request_t *r,
     ngx_int_t rc);
@@ -912,10 +911,7 @@
     }
 
     if (rc == NGX_ERROR) {
-        r->http_version = NGX_HTTP_VERSION_9;
-
         u->process_header = ngx_http_uwsgi_process_header;
-
         return ngx_http_uwsgi_process_header(r);
     }
 
@@ -1015,12 +1011,12 @@
             ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                            "http uwsgi header done");
 
-            if (r->http_version > NGX_HTTP_VERSION_9) {
+            u = r->upstream;
+
+            if (u->headers_in.status_n) {
                 return NGX_OK;
             }
 
-            u = r->upstream;
-
             if (u->headers_in.status) {
                 status_line = &u->headers_in.status->value;
 
@@ -1032,20 +1028,15 @@
                     return NGX_HTTP_UPSTREAM_INVALID_HEADER;
                 }
 
-                r->http_version = NGX_HTTP_VERSION_10;
                 u->headers_in.status_n = status;
                 u->headers_in.status_line = *status_line;
 
             } else if (u->headers_in.location) {
-                r->http_version = NGX_HTTP_VERSION_10;
                 u->headers_in.status_n = 302;
                 ngx_str_set(&u->headers_in.status_line,
                             "302 Moved Temporarily");
 
             } else {
-                ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
-                              "upstream sent neither valid HTTP/1.0 header "
-                              "nor \"Status\" header line");
                 u->headers_in.status_n = 200;
                 ngx_str_set(&u->headers_in.status_line, "200 OK");
             }


From mdounin at mdounin.ru  Thu Mar 15 11:23:07 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 15 Mar 2012 11:23:07 +0000
Subject: [nginx] svn commit:  r4529 - trunk/src/http/modules
Message-ID: <20120315112307.BAEB53FA932@mail.nginx.com>

Author: mdounin
Date: 2012-03-15 11:23:07 +0000 (Thu, 15 Mar 2012)
New Revision: 4529

Log:
Fixed ssi and perl interaction.

Embedded perl module assumes there is a space for terminating NUL character,
make sure to provide it in all situations by allocating one extra byte for
value buffer.  Default ssi_value_length is reduced accordingly to
preserve 256 byte allocations.

While here, fixed another one byte value buffer overrun possible in
ssi_quoted_symbol_state.

Reported by Matthew Daley.


Modified:
   trunk/src/http/modules/ngx_http_ssi_filter_module.c

Modified: trunk/src/http/modules/ngx_http_ssi_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_ssi_filter_module.c	2012-03-15 11:21:54 UTC (rev 4528)
+++ trunk/src/http/modules/ngx_http_ssi_filter_module.c	2012-03-15 11:23:07 UTC (rev 4529)
@@ -1204,7 +1204,7 @@
 
                 if (ctx->value_buf == NULL) {
                     ctx->param->value.data = ngx_pnalloc(r->pool,
-                                                         ctx->value_len);
+                                                         ctx->value_len + 1);
                     if (ctx->param->value.data == NULL) {
                         return NGX_ERROR;
                     }
@@ -1375,6 +1375,16 @@
         case ssi_quoted_symbol_state:
             state = ctx->saved_state;
 
+            if (ctx->param->value.len == ctx->value_len) {
+                ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                              "too long \"%V%c...\" value of \"%V\" "
+                              "parameter in \"%V\" SSI command",
+                              &ctx->param->value, ch, &ctx->param->key,
+                              &ctx->command);
+                state = ssi_error_state;
+                break;
+            }
+
             ctx->param->value.data[ctx->param->value.len++] = ch;
 
             break;
@@ -2886,7 +2896,7 @@
                          prev->ignore_recycled_buffers, 0);
 
     ngx_conf_merge_size_value(conf->min_file_chunk, prev->min_file_chunk, 1024);
-    ngx_conf_merge_size_value(conf->value_len, prev->value_len, 256);
+    ngx_conf_merge_size_value(conf->value_len, prev->value_len, 255);
 
     if (ngx_http_merge_types(cf, &conf->types_keys, &conf->types,
                              &prev->types_keys, &prev->types,


From mdounin at mdounin.ru  Thu Mar 15 11:27:13 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 15 Mar 2012 11:27:13 +0000
Subject: [nginx] svn commit:  r4530 - trunk/src/http/modules
Message-ID: <20120315112713.7783A3FAA4C@mail.nginx.com>

Author: mdounin
Date: 2012-03-15 11:27:12 +0000 (Thu, 15 Mar 2012)
New Revision: 4530

Log:
Fixed incorrect ngx_cpystrn() usage in ngx_http_*_process_header().

This resulted in a disclosure of previously freed memory if upstream
server returned specially crafted response, potentially exposing
sensitive information.

Reported by Matthew Daley.


Modified:
   trunk/src/http/modules/ngx_http_fastcgi_module.c
   trunk/src/http/modules/ngx_http_proxy_module.c
   trunk/src/http/modules/ngx_http_scgi_module.c
   trunk/src/http/modules/ngx_http_uwsgi_module.c

Modified: trunk/src/http/modules/ngx_http_fastcgi_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_fastcgi_module.c	2012-03-15 11:23:07 UTC (rev 4529)
+++ trunk/src/http/modules/ngx_http_fastcgi_module.c	2012-03-15 11:27:12 UTC (rev 4530)
@@ -1501,10 +1501,10 @@
                     h->lowcase_key = h->key.data + h->key.len + 1
                                      + h->value.len + 1;
 
-                    ngx_cpystrn(h->key.data, r->header_name_start,
-                                h->key.len + 1);
-                    ngx_cpystrn(h->value.data, r->header_start,
-                                h->value.len + 1);
+                    ngx_memcpy(h->key.data, r->header_name_start, h->key.len);
+                    h->key.data[h->key.len] = '\0';
+                    ngx_memcpy(h->value.data, r->header_start, h->value.len);
+                    h->value.data[h->value.len] = '\0';
                 }
 
                 h->hash = r->header_hash;

Modified: trunk/src/http/modules/ngx_http_proxy_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_proxy_module.c	2012-03-15 11:23:07 UTC (rev 4529)
+++ trunk/src/http/modules/ngx_http_proxy_module.c	2012-03-15 11:27:12 UTC (rev 4530)
@@ -1381,8 +1381,10 @@
             h->value.data = h->key.data + h->key.len + 1;
             h->lowcase_key = h->key.data + h->key.len + 1 + h->value.len + 1;
 
-            ngx_cpystrn(h->key.data, r->header_name_start, h->key.len + 1);
-            ngx_cpystrn(h->value.data, r->header_start, h->value.len + 1);
+            ngx_memcpy(h->key.data, r->header_name_start, h->key.len);
+            h->key.data[h->key.len] = '\0';
+            ngx_memcpy(h->value.data, r->header_start, h->value.len);
+            h->value.data[h->value.len] = '\0';
 
             if (h->key.len == r->lowcase_index) {
                 ngx_memcpy(h->lowcase_key, r->lowcase_header, h->key.len);

Modified: trunk/src/http/modules/ngx_http_scgi_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_scgi_module.c	2012-03-15 11:23:07 UTC (rev 4529)
+++ trunk/src/http/modules/ngx_http_scgi_module.c	2012-03-15 11:27:12 UTC (rev 4530)
@@ -941,8 +941,10 @@
             h->value.data = h->key.data + h->key.len + 1;
             h->lowcase_key = h->key.data + h->key.len + 1 + h->value.len + 1;
 
-            ngx_cpystrn(h->key.data, r->header_name_start, h->key.len + 1);
-            ngx_cpystrn(h->value.data, r->header_start, h->value.len + 1);
+            ngx_memcpy(h->key.data, r->header_name_start, h->key.len);
+            h->key.data[h->key.len] = '\0';
+            ngx_memcpy(h->value.data, r->header_start, h->value.len);
+            h->value.data[h->value.len] = '\0';
 
             if (h->key.len == r->lowcase_index) {
                 ngx_memcpy(h->lowcase_key, r->lowcase_header, h->key.len);

Modified: trunk/src/http/modules/ngx_http_uwsgi_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_uwsgi_module.c	2012-03-15 11:23:07 UTC (rev 4529)
+++ trunk/src/http/modules/ngx_http_uwsgi_module.c	2012-03-15 11:27:12 UTC (rev 4530)
@@ -981,8 +981,10 @@
             h->value.data = h->key.data + h->key.len + 1;
             h->lowcase_key = h->key.data + h->key.len + 1 + h->value.len + 1;
 
-            ngx_cpystrn(h->key.data, r->header_name_start, h->key.len + 1);
-            ngx_cpystrn(h->value.data, r->header_start, h->value.len + 1);
+            ngx_memcpy(h->key.data, r->header_name_start, h->key.len);
+            h->key.data[h->key.len] = '\0';
+            ngx_memcpy(h->value.data, r->header_start, h->value.len);
+            h->value.data[h->value.len] = '\0';
 
             if (h->key.len == r->lowcase_index) {
                 ngx_memcpy(h->lowcase_key, r->lowcase_header, h->key.len);


From mdounin at mdounin.ru  Thu Mar 15 11:27:57 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 15 Mar 2012 11:27:57 +0000
Subject: [nginx] svn commit:  r4531 - trunk/src/http
Message-ID: <20120315112757.F0F3D3FAA4C@mail.nginx.com>

Author: mdounin
Date: 2012-03-15 11:27:57 +0000 (Thu, 15 Mar 2012)
New Revision: 4531

Log:
Headers with null character are now rejected.

Headers with NUL character aren't allowed by HTTP standard and may cause
various security problems.  They are now unconditionally rejected.


Modified:
   trunk/src/http/ngx_http_parse.c

Modified: trunk/src/http/ngx_http_parse.c
===================================================================
--- trunk/src/http/ngx_http_parse.c	2012-03-15 11:27:12 UTC (rev 4530)
+++ trunk/src/http/ngx_http_parse.c	2012-03-15 11:27:57 UTC (rev 4531)
@@ -874,6 +874,10 @@
                     break;
                 }
 
+                if (ch == '\0') {
+                    return NGX_HTTP_PARSE_INVALID_HEADER;
+                }
+
                 r->invalid_header = 1;
 
                 break;
@@ -936,6 +940,10 @@
                 break;
             }
 
+            if (ch == '\0') {
+                return NGX_HTTP_PARSE_INVALID_HEADER;
+            }
+
             r->invalid_header = 1;
 
             break;
@@ -954,6 +962,8 @@
                 r->header_start = p;
                 r->header_end = p;
                 goto done;
+            case '\0':
+                return NGX_HTTP_PARSE_INVALID_HEADER;
             default:
                 r->header_start = p;
                 state = sw_value;
@@ -975,6 +985,8 @@
             case LF:
                 r->header_end = p;
                 goto done;
+            case '\0':
+                return NGX_HTTP_PARSE_INVALID_HEADER;
             }
             break;
 
@@ -988,6 +1000,8 @@
                 break;
             case LF:
                 goto done;
+            case '\0':
+                return NGX_HTTP_PARSE_INVALID_HEADER;
             default:
                 state = sw_value;
                 break;


From mdounin at mdounin.ru  Thu Mar 15 11:32:18 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 15 Mar 2012 11:32:18 +0000
Subject: [nginx] svn commit:  r4532 - trunk/docs/xml/nginx
Message-ID: <20120315113218.8A4A93F9D40@mail.nginx.com>

Author: mdounin
Date: 2012-03-15 11:32:18 +0000 (Thu, 15 Mar 2012)
New Revision: 4532

Log:
nginx-1.1.17-RELEASE


Modified:
   trunk/docs/xml/nginx/changes.xml

Modified: trunk/docs/xml/nginx/changes.xml
===================================================================
--- trunk/docs/xml/nginx/changes.xml	2012-03-15 11:27:57 UTC (rev 4531)
+++ trunk/docs/xml/nginx/changes.xml	2012-03-15 11:32:18 UTC (rev 4532)
@@ -9,6 +9,44 @@
 <title lang="en">nginx changelog</title>
 
 
+<changes ver="1.1.17" date="15.03.2012">
+
+<change type="security">
+<para lang="ru">
+?????????? ????? ????????????? ?????? ????? ???? ?????????? ???????,
+???? ?????? ????????? ?????????? ????????? ?????.<br/>
+??????? Matthew Daley.
+</para>
+<para lang="en">
+content of previously freed memory might be sent to a client
+if backend returned specially crafted response.<br/>
+Thanks to Matthew Daley.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ????????????? ??????????? ????? ?? SSI.<br/>
+??????? Matthew Daley.
+</para>
+<para lang="en">
+in the embedded perl module if used from SSI.<br/>
+Thanks to Matthew Daley.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ?????? ngx_http_uwsgi_module.
+</para>
+<para lang="en">
+in the ngx_http_uwsgi_module.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.1.16" date="29.02.2012">
 
 <change type="change">


From mdounin at mdounin.ru  Thu Mar 15 11:32:40 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 15 Mar 2012 11:32:40 +0000
Subject: [nginx] svn commit:  r4533 - tags
Message-ID: <20120315113240.35C903FB37A@mail.nginx.com>

Author: mdounin
Date: 2012-03-15 11:32:39 +0000 (Thu, 15 Mar 2012)
New Revision: 4533

Log:
release-1.1.17 tag


Added:
   tags/release-1.1.17/


From mdounin at mdounin.ru  Thu Mar 15 11:37:11 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 15 Mar 2012 11:37:11 +0000
Subject: [nginx] svn commit: r4534 - in branches/stable-1.0/src: core
	http/modules/perl
Message-ID: <20120315113712.8A29B3FA802@mail.nginx.com>

Author: mdounin
Date: 2012-03-15 11:37:11 +0000 (Thu, 15 Mar 2012)
New Revision: 4534

Log:
Version bump.


Modified:
   branches/stable-1.0/src/core/nginx.h
   branches/stable-1.0/src/http/modules/perl/nginx.pm

Modified: branches/stable-1.0/src/core/nginx.h
===================================================================
--- branches/stable-1.0/src/core/nginx.h	2012-03-15 11:32:39 UTC (rev 4533)
+++ branches/stable-1.0/src/core/nginx.h	2012-03-15 11:37:11 UTC (rev 4534)
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1000013
-#define NGINX_VERSION      "1.0.13"
+#define nginx_version      1000014
+#define NGINX_VERSION      "1.0.14"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #define NGINX_VAR          "NGINX"

Modified: branches/stable-1.0/src/http/modules/perl/nginx.pm
===================================================================
--- branches/stable-1.0/src/http/modules/perl/nginx.pm	2012-03-15 11:32:39 UTC (rev 4533)
+++ branches/stable-1.0/src/http/modules/perl/nginx.pm	2012-03-15 11:37:11 UTC (rev 4534)
@@ -50,7 +50,7 @@
     HTTP_INSUFFICIENT_STORAGE
 );
 
-our $VERSION = '1.0.13';
+our $VERSION = '1.0.14';
 
 require XSLoader;
 XSLoader::load('nginx', $VERSION);


From mdounin at mdounin.ru  Thu Mar 15 11:41:43 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 15 Mar 2012 11:41:43 +0000
Subject: [nginx] svn commit: r4535 - in branches/stable-1.0: . src/http
	src/http/modules
Message-ID: <20120315114143.94B393FAA33@mail.nginx.com>

Author: mdounin
Date: 2012-03-15 11:41:43 +0000 (Thu, 15 Mar 2012)
New Revision: 4535

Log:
Merge of r4530, r4531: null character fixes.

*) Fixed incorrect ngx_cpystrn() usage in ngx_http_*_process_header().

   This resulted in a disclosure of previously freed memory if upstream
   server returned specially crafted response, potentially exposing
   sensitive information.

   Reported by Matthew Daley.

*) Headers with null character are now rejected.

   Headers with NUL character aren't allowed by HTTP standard and may cause
   various security problems.  They are now unconditionally rejected.


Modified:
   branches/stable-1.0/
   branches/stable-1.0/src/http/modules/ngx_http_fastcgi_module.c
   branches/stable-1.0/src/http/modules/ngx_http_proxy_module.c
   branches/stable-1.0/src/http/modules/ngx_http_scgi_module.c
   branches/stable-1.0/src/http/modules/ngx_http_uwsgi_module.c
   branches/stable-1.0/src/http/ngx_http_parse.c


Property changes on: branches/stable-1.0
___________________________________________________________________
Modified: svn:mergeinfo
   - /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4491-4493,4497-4500
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4491-4493,4497-4500,4530-4531

Modified: branches/stable-1.0/src/http/modules/ngx_http_fastcgi_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_fastcgi_module.c	2012-03-15 11:37:11 UTC (rev 4534)
+++ branches/stable-1.0/src/http/modules/ngx_http_fastcgi_module.c	2012-03-15 11:41:43 UTC (rev 4535)
@@ -1446,10 +1446,10 @@
                     h->lowcase_key = h->key.data + h->key.len + 1
                                      + h->value.len + 1;
 
-                    ngx_cpystrn(h->key.data, r->header_name_start,
-                                h->key.len + 1);
-                    ngx_cpystrn(h->value.data, r->header_start,
-                                h->value.len + 1);
+                    ngx_memcpy(h->key.data, r->header_name_start, h->key.len);
+                    h->key.data[h->key.len] = '\0';
+                    ngx_memcpy(h->value.data, r->header_start, h->value.len);
+                    h->value.data[h->value.len] = '\0';
                 }
 
                 h->hash = r->header_hash;

Modified: branches/stable-1.0/src/http/modules/ngx_http_proxy_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_proxy_module.c	2012-03-15 11:37:11 UTC (rev 4534)
+++ branches/stable-1.0/src/http/modules/ngx_http_proxy_module.c	2012-03-15 11:41:43 UTC (rev 4535)
@@ -1278,8 +1278,10 @@
             h->value.data = h->key.data + h->key.len + 1;
             h->lowcase_key = h->key.data + h->key.len + 1 + h->value.len + 1;
 
-            ngx_cpystrn(h->key.data, r->header_name_start, h->key.len + 1);
-            ngx_cpystrn(h->value.data, r->header_start, h->value.len + 1);
+            ngx_memcpy(h->key.data, r->header_name_start, h->key.len);
+            h->key.data[h->key.len] = '\0';
+            ngx_memcpy(h->value.data, r->header_start, h->value.len);
+            h->value.data[h->value.len] = '\0';
 
             if (h->key.len == r->lowcase_index) {
                 ngx_memcpy(h->lowcase_key, r->lowcase_header, h->key.len);

Modified: branches/stable-1.0/src/http/modules/ngx_http_scgi_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_scgi_module.c	2012-03-15 11:37:11 UTC (rev 4534)
+++ branches/stable-1.0/src/http/modules/ngx_http_scgi_module.c	2012-03-15 11:41:43 UTC (rev 4535)
@@ -894,8 +894,10 @@
             h->value.data = h->key.data + h->key.len + 1;
             h->lowcase_key = h->key.data + h->key.len + 1 + h->value.len + 1;
 
-            ngx_cpystrn(h->key.data, r->header_name_start, h->key.len + 1);
-            ngx_cpystrn(h->value.data, r->header_start, h->value.len + 1);
+            ngx_memcpy(h->key.data, r->header_name_start, h->key.len);
+            h->key.data[h->key.len] = '\0';
+            ngx_memcpy(h->value.data, r->header_start, h->value.len);
+            h->value.data[h->value.len] = '\0';
 
             if (h->key.len == r->lowcase_index) {
                 ngx_memcpy(h->lowcase_key, r->lowcase_header, h->key.len);

Modified: branches/stable-1.0/src/http/modules/ngx_http_uwsgi_module.c
===================================================================
--- branches/stable-1.0/src/http/modules/ngx_http_uwsgi_module.c	2012-03-15 11:37:11 UTC (rev 4534)
+++ branches/stable-1.0/src/http/modules/ngx_http_uwsgi_module.c	2012-03-15 11:41:43 UTC (rev 4535)
@@ -947,8 +947,10 @@
             h->value.data = h->key.data + h->key.len + 1;
             h->lowcase_key = h->key.data + h->key.len + 1 + h->value.len + 1;
 
-            ngx_cpystrn(h->key.data, r->header_name_start, h->key.len + 1);
-            ngx_cpystrn(h->value.data, r->header_start, h->value.len + 1);
+            ngx_memcpy(h->key.data, r->header_name_start, h->key.len);
+            h->key.data[h->key.len] = '\0';
+            ngx_memcpy(h->value.data, r->header_start, h->value.len);
+            h->value.data[h->value.len] = '\0';
 
             if (h->key.len == r->lowcase_index) {
                 ngx_memcpy(h->lowcase_key, r->lowcase_header, h->key.len);

Modified: branches/stable-1.0/src/http/ngx_http_parse.c
===================================================================
--- branches/stable-1.0/src/http/ngx_http_parse.c	2012-03-15 11:37:11 UTC (rev 4534)
+++ branches/stable-1.0/src/http/ngx_http_parse.c	2012-03-15 11:41:43 UTC (rev 4535)
@@ -814,6 +814,10 @@
                     break;
                 }
 
+                if (ch == '\0') {
+                    return NGX_HTTP_PARSE_INVALID_HEADER;
+                }
+
                 r->invalid_header = 1;
 
                 break;
@@ -876,6 +880,10 @@
                 break;
             }
 
+            if (ch == '\0') {
+                return NGX_HTTP_PARSE_INVALID_HEADER;
+            }
+
             r->invalid_header = 1;
 
             break;
@@ -894,6 +902,8 @@
                 r->header_start = p;
                 r->header_end = p;
                 goto done;
+            case '\0':
+                return NGX_HTTP_PARSE_INVALID_HEADER;
             default:
                 r->header_start = p;
                 state = sw_value;
@@ -915,6 +925,8 @@
             case LF:
                 r->header_end = p;
                 goto done;
+            case '\0':
+                return NGX_HTTP_PARSE_INVALID_HEADER;
             }
             break;
 
@@ -928,6 +940,8 @@
                 break;
             case LF:
                 goto done;
+            case '\0':
+                return NGX_HTTP_PARSE_INVALID_HEADER;
             default:
                 state = sw_value;
                 break;


From mdounin at mdounin.ru  Thu Mar 15 11:46:32 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 15 Mar 2012 11:46:32 +0000
Subject: [nginx] svn commit:  r4536 - branches/stable-1.0/misc
Message-ID: <20120315114633.000733FA893@mail.nginx.com>

Author: mdounin
Date: 2012-03-15 11:46:29 +0000 (Thu, 15 Mar 2012)
New Revision: 4536

Log:
Updated OpenSSL and PCRE used for win32 builds.


Modified:
   branches/stable-1.0/misc/GNUmakefile

Modified: branches/stable-1.0/misc/GNUmakefile
===================================================================
--- branches/stable-1.0/misc/GNUmakefile	2012-03-15 11:41:43 UTC (rev 4535)
+++ branches/stable-1.0/misc/GNUmakefile	2012-03-15 11:46:29 UTC (rev 4536)
@@ -6,9 +6,9 @@
 REPO =		$(shell svn info | sed -n 's/^Repository Root: //p')
 
 OBJS =		objs.msvc8
-OPENSSL =	openssl-0.9.8t
+OPENSSL =	openssl-0.9.8u
 ZLIB =		zlib-1.2.5
-PCRE =		pcre-8.21
+PCRE =		pcre-8.30
 
 
 release:


From mdounin at mdounin.ru  Thu Mar 15 11:50:54 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 15 Mar 2012 11:50:54 +0000
Subject: [nginx] svn commit:  r4537 - branches/stable-1.0/docs/xml/nginx
Message-ID: <20120315115054.ACCE23FADCC@mail.nginx.com>

Author: mdounin
Date: 2012-03-15 11:50:53 +0000 (Thu, 15 Mar 2012)
New Revision: 4537

Log:
nginx-1.0.14-RELEASE


Modified:
   branches/stable-1.0/docs/xml/nginx/changes.xml

Modified: branches/stable-1.0/docs/xml/nginx/changes.xml
===================================================================
--- branches/stable-1.0/docs/xml/nginx/changes.xml	2012-03-15 11:46:29 UTC (rev 4536)
+++ branches/stable-1.0/docs/xml/nginx/changes.xml	2012-03-15 11:50:53 UTC (rev 4537)
@@ -9,6 +9,24 @@
 <title lang="en">nginx changelog</title>
 
 
+<changes ver="1.0.14" date="15.03.2012">
+
+<change type="security">
+<para lang="ru">
+?????????? ????? ????????????? ?????? ????? ???? ?????????? ???????,
+???? ?????? ????????? ?????????? ????????? ?????.<br/>
+??????? Matthew Daley.
+</para>
+<para lang="en">
+content of previously freed memory might be sent to a client
+if backend returned specially crafted response.<br/>
+Thanks to Matthew Daley.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.0.13" date="05.03.2012">
 
 <change type="feature">


From mdounin at mdounin.ru  Thu Mar 15 11:51:18 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 15 Mar 2012 11:51:18 +0000
Subject: [nginx] svn commit:  r4538 - in tags: . release-1.0.14
Message-ID: <20120315115118.5938A3FAA4C@mail.nginx.com>

Author: mdounin
Date: 2012-03-15 11:51:16 +0000 (Thu, 15 Mar 2012)
New Revision: 4538

Log:
release-1.0.14 tag


Added:
   tags/release-1.0.14/


Property changes on: tags/release-1.0.14
___________________________________________________________________
Added: svn:ignore
   + access.log
client_body_temp
fastcgi_temp
proxy_temp
GNUmakefile
Makefile
makefile
nginx
nginx.conf
nginx-*.tar.gz
objs*
tmp

Added: svn:mergeinfo
   + /trunk:3960-3974,3977-3987,3991-3996,3998,4000-4018,4020,4023,4025-4027,4034-4065,4073,4077,4086-4090,4094-4102,4106-4108,4113-4114,4129-4137,4143-4144,4147-4158,4177,4179,4182-4184,4186-4187,4189-4205,4207,4209-4210,4212,4217-4223,4227-4232,4235-4237,4265-4268,4270,4274-4276,4278-4280,4282-4284,4294-4295,4298,4300-4309,4313,4315,4320-4321,4326-4327,4335-4336,4338-4343,4372-4375,4377,4379,4381-4385,4393,4396,4398,4400-4406,4413,4415-4416,4422-4423,4460-4461,4468,4470-4471,4473-4474,4491-4493,4497-4500,4530-4531


From mdounin at mdounin.ru  Thu Mar 15 17:43:55 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 15 Mar 2012 17:43:55 +0000
Subject: [nginx] svn commit:  r4539 - in trunk/src: core http/modules/perl
Message-ID: <20120315174356.1B71B3F9D21@mail.nginx.com>

Author: mdounin
Date: 2012-03-15 17:43:54 +0000 (Thu, 15 Mar 2012)
New Revision: 4539

Log:
Version bump.


Modified:
   trunk/src/core/nginx.h
   trunk/src/http/modules/perl/nginx.pm

Modified: trunk/src/core/nginx.h
===================================================================
--- trunk/src/core/nginx.h	2012-03-15 11:51:16 UTC (rev 4538)
+++ trunk/src/core/nginx.h	2012-03-15 17:43:54 UTC (rev 4539)
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1001017
-#define NGINX_VERSION      "1.1.17"
+#define nginx_version      1001018
+#define NGINX_VERSION      "1.1.18"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #define NGINX_VAR          "NGINX"

Modified: trunk/src/http/modules/perl/nginx.pm
===================================================================
--- trunk/src/http/modules/perl/nginx.pm	2012-03-15 11:51:16 UTC (rev 4538)
+++ trunk/src/http/modules/perl/nginx.pm	2012-03-15 17:43:54 UTC (rev 4539)
@@ -50,7 +50,7 @@
     HTTP_INSUFFICIENT_STORAGE
 );
 
-our $VERSION = '1.1.17';
+our $VERSION = '1.1.18';
 
 require XSLoader;
 XSLoader::load('nginx', $VERSION);


From mdounin at mdounin.ru  Thu Mar 15 18:04:39 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 15 Mar 2012 22:04:39 +0400
Subject: [PATCH] Fix ./configure script for less error-forgiving compilers
In-Reply-To: <20120306084503.GB90547@lo0.su>
References: <6D1F240BFA024B0192CAE2E659741B61@Desktop>
	<20101111143328.GW44164@mdounin.ru>
	<50DFC9CF41D24C87AF89E02E62904937@Desktop>
	<C7CD1B1B4BA64AE489638EC91D23E2BD@Desktop>
	<40F392827F72402D819905F1EF31043E@Desktop>
	<20120306084503.GB90547@lo0.su>
Message-ID: <20120315180438.GB67687@mdounin.ru>

Hello!

On Tue, Mar 06, 2012 at 12:45:03PM +0400, Ruslan Ermilov wrote:

> On Mon, Mar 05, 2012 at 10:37:31PM +0100, Piotr Sikora wrote:
> > --- auto/types/uintptr_t.orig	Sun Jul 29 18:24:53 2007
> > +++ auto/types/uintptr_t	Tue Aug 24 19:38:17 2010
> > @@ -14,7 +14,8 @@
> >  $NGX_INTTYPES_H
> >  
> >  int main() {
> > -    uintptr_t i = 0;
> > +    uintptr_t i;
> > +    i = 0;
> >      return 0;
> >  }
> 
> The way this script is written, -Wall -Werror were not passed
> to the compiler.  I've fixed this, and it made your change
> obviously required.

Another obvious test is to use CFLAGS="-Wall -Werror" ./configure.

> > --- auto/unix.orig	Thu Aug 12 04:33:51 2010
> > +++ auto/unix	Thu Aug 12 04:34:20 2010
> > @@ -82,7 +82,7 @@
> >  ngx_feature="setproctitle()"
> >  ngx_feature_name="NGX_HAVE_SETPROCTITLE"
> >  ngx_feature_run=no
> > -ngx_feature_incs=
> > +ngx_feature_incs="#include <stdlib.h>"
> >  ngx_feature_path=
> >  ngx_feature_libs=$NGX_SETPROCTITLE_LIB
> >  ngx_feature_test="setproctitle(\"test\");"
> 
> setproctitle() is in <unistd.h> on FreeBSD, and the latter
> is always included when available.  This change isn't needed.

As already clarified by Piotr, this is needed for OpenBSD / 
NetBSD.

> I've also fixed configure errors on Linux when configured
> with --with-cc=gcc --with-cc-opt="-Wall -Werror".

I don't really think we support --with-cc-opt="-Wall -Werror", it 
will fail on compilation anyway as we really need 
"-Wno-unused-parameter".

But it doesn't hurt to keep configure clean, please restore 
Piotr's setproctitle() change and commit.

[...]

> @@ -585,7 +585,8 @@
>  ngx_feature="memalign()"
>  ngx_feature_name="NGX_HAVE_MEMALIGN"
>  ngx_feature_run=no
> -ngx_feature_incs="#include <stdlib.h>"
> +ngx_feature_incs="#include <stdlib.h>
> +                  #include <malloc.h>"
>  ngx_feature_path=
>  ngx_feature_libs=
>  ngx_feature_test="void *p; p = memalign(4096, 4096)"

Just a side note: this probably should be run only if 
posix_memalign() isn't found.  The same applies to gcc variadic 
macros test, which is redundant in case of C99 variadic macros 
found.

Maxim Dounin


From ru at nginx.com  Thu Mar 15 19:37:32 2012
From: ru at nginx.com (ru at nginx.com)
Date: Thu, 15 Mar 2012 19:37:32 +0000
Subject: [nginx] svn commit:  r4540 - trunk/src/http/modules
Message-ID: <20120315193732.95D5B3F9D6D@mail.nginx.com>

Author: ru
Date: 2012-03-15 19:37:32 +0000 (Thu, 15 Mar 2012)
New Revision: 4540

Log:
- New variable: $connection_requests.
- While here, fixed format specifier for $connection.


Modified:
   trunk/src/http/modules/ngx_http_log_module.c

Modified: trunk/src/http/modules/ngx_http_log_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_log_module.c	2012-03-15 17:43:54 UTC (rev 4539)
+++ trunk/src/http/modules/ngx_http_log_module.c	2012-03-15 19:37:32 UTC (rev 4540)
@@ -80,6 +80,8 @@
 
 static u_char *ngx_http_log_connection(ngx_http_request_t *r, u_char *buf,
     ngx_http_log_op_t *op);
+static u_char *ngx_http_log_connection_requests(ngx_http_request_t *r,
+    u_char *buf, ngx_http_log_op_t *op);
 static u_char *ngx_http_log_pipe(ngx_http_request_t *r, u_char *buf,
     ngx_http_log_op_t *op);
 static u_char *ngx_http_log_time(ngx_http_request_t *r, u_char *buf,
@@ -193,6 +195,8 @@
 
 static ngx_http_log_var_t  ngx_http_log_vars[] = {
     { ngx_string("connection"), NGX_ATOMIC_T_LEN, ngx_http_log_connection },
+    { ngx_string("connection_requests"), NGX_INT_T_LEN,
+                          ngx_http_log_connection_requests },
     { ngx_string("pipe"), 1, ngx_http_log_pipe },
     { ngx_string("time_local"), sizeof("28/Sep/1970:12:00:00 +0600") - 1,
                           ngx_http_log_time },
@@ -501,11 +505,19 @@
 ngx_http_log_connection(ngx_http_request_t *r, u_char *buf,
     ngx_http_log_op_t *op)
 {
-    return ngx_sprintf(buf, "%ui", r->connection->number);
+    return ngx_sprintf(buf, "%uA", r->connection->number);
 }
 
 
 static u_char *
+ngx_http_log_connection_requests(ngx_http_request_t *r, u_char *buf,
+    ngx_http_log_op_t *op)
+{
+    return ngx_sprintf(buf, "%ui", r->connection->requests);
+}
+
+
+static u_char *
 ngx_http_log_pipe(ngx_http_request_t *r, u_char *buf, ngx_http_log_op_t *op)
 {
     if (r->pipeline) {


From ru at nginx.com  Thu Mar 15 19:41:36 2012
From: ru at nginx.com (ru at nginx.com)
Date: Thu, 15 Mar 2012 19:41:36 +0000
Subject: [nginx] svn commit:  r4541 - trunk/src/http
Message-ID: <20120315194136.231753F9D20@mail.nginx.com>

Author: ru
Date: 2012-03-15 19:41:35 +0000 (Thu, 15 Mar 2012)
New Revision: 4541

Log:
Slight optimization in ngx_http_get_variable_index().


Modified:
   trunk/src/http/ngx_http_variables.c

Modified: trunk/src/http/ngx_http_variables.c
===================================================================
--- trunk/src/http/ngx_http_variables.c	2012-03-15 19:37:32 UTC (rev 4540)
+++ trunk/src/http/ngx_http_variables.c	2012-03-15 19:41:35 UTC (rev 4541)
@@ -384,7 +384,7 @@
     v->flags = 0;
     v->index = cmcf->variables.nelts - 1;
 
-    return cmcf->variables.nelts - 1;
+    return v->index;
 }
 
 


From ru at nginx.com  Thu Mar 15 20:04:50 2012
From: ru at nginx.com (ru at nginx.com)
Date: Thu, 15 Mar 2012 20:04:50 +0000
Subject: [nginx] svn commit:  r4542 - trunk/src/http
Message-ID: <20120315200450.D576E3F9E38@mail.nginx.com>

Author: ru
Date: 2012-03-15 20:04:50 +0000 (Thu, 15 Mar 2012)
New Revision: 4542

Log:
The "error_log" directive specified in the "http", "server", and
"location" sections now understands the special "stderr" parameter.
It was already treated specially when specified in the main section.


Modified:
   trunk/src/http/ngx_http_core_module.c

Modified: trunk/src/http/ngx_http_core_module.c
===================================================================
--- trunk/src/http/ngx_http_core_module.c	2012-03-15 19:41:35 UTC (rev 4541)
+++ trunk/src/http/ngx_http_core_module.c	2012-03-15 20:04:50 UTC (rev 4542)
@@ -4647,7 +4647,7 @@
 {
     ngx_http_core_loc_conf_t *clcf = conf;
 
-    ngx_str_t  *value;
+    ngx_str_t  *value, name;
 
     if (clcf->error_log) {
         return "is duplicate";
@@ -4655,7 +4655,14 @@
 
     value = cf->args->elts;
 
-    clcf->error_log = ngx_log_create(cf->cycle, &value[1]);
+    if (ngx_strcmp(value[1].data, "stderr") == 0) {
+        ngx_str_null(&name);
+
+    } else {
+        name = value[1];
+    }
+
+    clcf->error_log = ngx_log_create(cf->cycle, &name);
     if (clcf->error_log == NULL) {
         return NGX_CONF_ERROR;
     }


From ru at nginx.com  Thu Mar 15 20:09:00 2012
From: ru at nginx.com (ru at nginx.com)
Date: Thu, 15 Mar 2012 20:09:00 +0000
Subject: [nginx] svn commit:  r4543 - trunk/src/http
Message-ID: <20120315200901.8E4ED3F9D47@mail.nginx.com>

Author: ru
Date: 2012-03-15 20:08:58 +0000 (Thu, 15 Mar 2012)
New Revision: 4543

Log:
Local variable "ngx_http_next_filter" renamed to "ngx_http_next_body_filter"
for consistency with other modules.


Modified:
   trunk/src/http/ngx_http_copy_filter_module.c
   trunk/src/http/ngx_http_postpone_filter_module.c

Modified: trunk/src/http/ngx_http_copy_filter_module.c
===================================================================
--- trunk/src/http/ngx_http_copy_filter_module.c	2012-03-15 20:04:50 UTC (rev 4542)
+++ trunk/src/http/ngx_http_copy_filter_module.c	2012-03-15 20:08:58 UTC (rev 4543)
@@ -74,7 +74,7 @@
 };
 
 
-static ngx_http_output_body_filter_pt    ngx_http_next_filter;
+static ngx_http_output_body_filter_pt    ngx_http_next_body_filter;
 
 
 static ngx_int_t
@@ -115,7 +115,8 @@
         ctx->bufs = conf->bufs;
         ctx->tag = (ngx_buf_tag_t) &ngx_http_copy_filter_module;
 
-        ctx->output_filter = (ngx_output_chain_filter_pt) ngx_http_next_filter;
+        ctx->output_filter = (ngx_output_chain_filter_pt)
+                                  ngx_http_next_body_filter;
         ctx->filter_ctx = r;
 
 #if (NGX_HAVE_FILE_AIO)
@@ -292,7 +293,7 @@
 static ngx_int_t
 ngx_http_copy_filter_init(ngx_conf_t *cf)
 {
-    ngx_http_next_filter = ngx_http_top_body_filter;
+    ngx_http_next_body_filter = ngx_http_top_body_filter;
     ngx_http_top_body_filter = ngx_http_copy_filter;
 
     return NGX_OK;

Modified: trunk/src/http/ngx_http_postpone_filter_module.c
===================================================================
--- trunk/src/http/ngx_http_postpone_filter_module.c	2012-03-15 20:04:50 UTC (rev 4542)
+++ trunk/src/http/ngx_http_postpone_filter_module.c	2012-03-15 20:08:58 UTC (rev 4543)
@@ -46,7 +46,7 @@
 };
 
 
-static ngx_http_output_body_filter_pt    ngx_http_next_filter;
+static ngx_http_output_body_filter_pt    ngx_http_next_body_filter;
 
 
 static ngx_int_t
@@ -80,7 +80,7 @@
     if (r->postponed == NULL) {
 
         if (in || c->buffered) {
-            return ngx_http_next_filter(r->main, in);
+            return ngx_http_next_body_filter(r->main, in);
         }
 
         return NGX_OK;
@@ -116,7 +116,7 @@
                            "http postpone filter output \"%V?%V\"",
                            &r->uri, &r->args);
 
-            if (ngx_http_next_filter(r->main, pr->out) == NGX_ERROR) {
+            if (ngx_http_next_body_filter(r->main, pr->out) == NGX_ERROR) {
                 return NGX_ERROR;
             }
         }
@@ -171,7 +171,7 @@
 static ngx_int_t
 ngx_http_postpone_filter_init(ngx_conf_t *cf)
 {
-    ngx_http_next_filter = ngx_http_top_body_filter;
+    ngx_http_next_body_filter = ngx_http_top_body_filter;
     ngx_http_top_body_filter = ngx_http_postpone_filter;
 
     return NGX_OK;


From ru at nginx.com  Thu Mar 15 20:39:39 2012
From: ru at nginx.com (ru at nginx.com)
Date: Thu, 15 Mar 2012 20:39:39 +0000
Subject: [nginx] svn commit:  r4544 - in trunk/auto: . os types
Message-ID: <20120315203939.76F7A3F9DC4@mail.nginx.com>

Author: ru
Date: 2012-03-15 20:39:38 +0000 (Thu, 15 Mar 2012)
New Revision: 4544

Log:
Fixed compilation warnings in configuration C tests.

Based on a patch by Piotr Sikora.


Modified:
   trunk/auto/os/linux
   trunk/auto/types/sizeof
   trunk/auto/types/typedef
   trunk/auto/types/uintptr_t
   trunk/auto/unix

Modified: trunk/auto/os/linux
===================================================================
--- trunk/auto/os/linux	2012-03-15 20:08:58 UTC (rev 4543)
+++ trunk/auto/os/linux	2012-03-15 20:39:38 UTC (rev 4544)
@@ -52,7 +52,7 @@
 ngx_feature_incs="#include <sys/epoll.h>"
 ngx_feature_path=
 ngx_feature_libs=
-ngx_feature_test="int efd = 0, fd = 1, n;
+ngx_feature_test="int efd = 0;
                   struct epoll_event ee;
                   ee.events = EPOLLIN|EPOLLOUT|EPOLLET;
                   ee.data.ptr = NULL;
@@ -142,7 +142,7 @@
 ngx_feature_path=
 ngx_feature_libs=-lcrypt
 ngx_feature_test="struct crypt_data  cd;
-                  crypt_r(NULL, NULL, &cd);"
+                  crypt_r(\"key\", \"salt\", &cd);"
 . auto/feature
 
 

Modified: trunk/auto/types/sizeof
===================================================================
--- trunk/auto/types/sizeof	2012-03-15 20:08:58 UTC (rev 4543)
+++ trunk/auto/types/sizeof	2012-03-15 20:39:38 UTC (rev 4544)
@@ -20,12 +20,13 @@
 #include <sys/time.h>
 $NGX_INCLUDE_UNISTD_H
 #include <signal.h>
+#include <stdio.h>
 #include <sys/resource.h>
 $NGX_INCLUDE_INTTYPES_H
 $NGX_INCLUDE_AUTO_CONFIG_H
 
 int main() {
-    printf("%d", sizeof($ngx_type));
+    printf("%zu", sizeof($ngx_type));
     return 0;
 }
 

Modified: trunk/auto/types/typedef
===================================================================
--- trunk/auto/types/typedef	2012-03-15 20:08:58 UTC (rev 4543)
+++ trunk/auto/types/typedef	2012-03-15 20:39:38 UTC (rev 4544)
@@ -28,7 +28,8 @@
 $NGX_INCLUDE_INTTYPES_H
 
 int main() {
-    $ngx_try i = 0;
+    $ngx_try i;
+    i = 0;
     return 0;
 }
 

Modified: trunk/auto/types/uintptr_t
===================================================================
--- trunk/auto/types/uintptr_t	2012-03-15 20:08:58 UTC (rev 4543)
+++ trunk/auto/types/uintptr_t	2012-03-15 20:39:38 UTC (rev 4544)
@@ -4,8 +4,8 @@
 
 
 echo $ngx_n "checking for uintptr_t ...$ngx_c"
-echo >> $NGX_ERR
-echo "checking for uintptr_t" >> $NGX_ERR
+echo >> $NGX_AUTOCONF_ERR
+echo "checking for uintptr_t" >> $NGX_AUTOCONF_ERR
 
 found=no
 
@@ -15,14 +15,18 @@
 $NGX_INTTYPES_H
 
 int main() {
-    uintptr_t i = 0;
+    uintptr_t i;
+    i = 0;
     return 0;
 }
 
 END
 
-eval "$CC -o $NGX_AUTOTEST $NGX_AUTOTEST.c >> $NGX_ERR 2>&1"
+ngx_test="$CC $CC_TEST_FLAGS $CC_AUX_FLAGS \
+          -o $NGX_AUTOTEST $NGX_AUTOTEST.c $NGX_LD_OPT"
 
+eval "$ngx_test >> $NGX_AUTOCONF_ERR 2>&1"
+
 if [ -x $NGX_AUTOTEST ]; then
     echo " uintptr_t found"
     found=yes

Modified: trunk/auto/unix
===================================================================
--- trunk/auto/unix	2012-03-15 20:08:58 UTC (rev 4543)
+++ trunk/auto/unix	2012-03-15 20:39:38 UTC (rev 4544)
@@ -237,7 +237,7 @@
 ngx_feature_path=
 ngx_feature_libs=
 ngx_feature_test="struct statfs  fs;
-                  statfs(NULL, &fs);"
+                  statfs(\".\", &fs);"
 . auto/feature
 
 
@@ -249,7 +249,7 @@
 ngx_feature_path=
 ngx_feature_libs=
 ngx_feature_test="struct statvfs  fs;
-                  statvfs(NULL, &fs);"
+                  statvfs(\".\", &fs);"
 . auto/feature
 
 
@@ -481,7 +481,7 @@
 ngx_feature="setproctitle()"
 ngx_feature_name="NGX_HAVE_SETPROCTITLE"
 ngx_feature_run=no
-ngx_feature_incs=
+ngx_feature_incs="#include <stdlib.h>"
 ngx_feature_path=
 ngx_feature_libs=$NGX_SETPROCTITLE_LIB
 ngx_feature_test="setproctitle(\"test\");"
@@ -585,7 +585,8 @@
 ngx_feature="memalign()"
 ngx_feature_name="NGX_HAVE_MEMALIGN"
 ngx_feature_run=no
-ngx_feature_incs="#include <stdlib.h>"
+ngx_feature_incs="#include <stdlib.h>
+                  #include <malloc.h>"
 ngx_feature_path=
 ngx_feature_libs=
 ngx_feature_test="void *p; p = memalign(4096, 4096)"


From ru at nginx.com  Fri Mar 16 07:33:56 2012
From: ru at nginx.com (ru at nginx.com)
Date: Fri, 16 Mar 2012 07:33:56 +0000
Subject: [nginx] svn commit:  r4545 - trunk/auto/types
Message-ID: <20120316073356.83F073F9DB3@mail.nginx.com>

Author: ru
Date: 2012-03-16 07:33:55 +0000 (Fri, 16 Mar 2012)
New Revision: 4545

Log:
Some older OSes (notably FreeBSD 4.x) did not have %zu
format specifier, so revert to using %d.


Modified:
   trunk/auto/types/sizeof

Modified: trunk/auto/types/sizeof
===================================================================
--- trunk/auto/types/sizeof	2012-03-15 20:39:38 UTC (rev 4544)
+++ trunk/auto/types/sizeof	2012-03-16 07:33:55 UTC (rev 4545)
@@ -26,7 +26,7 @@
 $NGX_INCLUDE_AUTO_CONFIG_H
 
 int main() {
-    printf("%zu", sizeof($ngx_type));
+    printf("%d", (int) sizeof($ngx_type));
     return 0;
 }
 


From piotr.sikora at frickle.com  Fri Mar 16 08:05:45 2012
From: piotr.sikora at frickle.com (Piotr Sikora)
Date: Fri, 16 Mar 2012 09:05:45 +0100
Subject: [nginx] svn commit:  r4545 - trunk/auto/types
In-Reply-To: <20120316073356.83F073F9DB3@mail.nginx.com>
References: <20120316073356.83F073F9DB3@mail.nginx.com>
Message-ID: <CCC1A9E0BE5E44C9A09B331802D16459@Desktop>

Hi,

> Some older OSes (notably FreeBSD 4.x) did not have %zu
> format specifier, so revert to using %d.

I don't mind this change, but is there really a point in supporting 
operating systems well past their End-of-Life date?

Best regards,
Piotr Sikora < piotr.sikora at frickle.com >


From ru at nginx.com  Fri Mar 16 08:34:44 2012
From: ru at nginx.com (Ruslan Ermilov)
Date: Fri, 16 Mar 2012 12:34:44 +0400
Subject: [nginx] svn commit:  r4545 - trunk/auto/types
In-Reply-To: <CCC1A9E0BE5E44C9A09B331802D16459@Desktop>
References: <20120316073356.83F073F9DB3@mail.nginx.com>
	<CCC1A9E0BE5E44C9A09B331802D16459@Desktop>
Message-ID: <20120316083444.GB71575@lo0.su>

On Fri, Mar 16, 2012 at 09:05:45AM +0100, Piotr Sikora wrote:
> Hi,
> 
> > Some older OSes (notably FreeBSD 4.x) did not have %zu
> > format specifier, so revert to using %d.
> 
> I don't mind this change, but is there really a point in supporting 
> operating systems well past their End-of-Life date?

Probably not, but it doesn't really matter.

We want the nginx autoconfiguration to depend on the minimum
features provided by the host, and it's possible to face the
system (FreeBSD 4.X is one such example) that doesn't support
%zu printf() format specifier.  As it costs us nearly nothing
to support them here, why not?

nginx still has some FreeBSD 4.X conditional code.  Whether it
should be kept or removed is another unrelated question.


From nginx.devlist.alias at fremnet.net  Fri Mar 16 13:33:53 2012
From: nginx.devlist.alias at fremnet.net (nginx.devlist.alias at fremnet.net)
Date: Fri, 16 Mar 2012 23:33:53 +1000
Subject: Attempting to write a plugin
In-Reply-To: <mailman.0.1331903434.38417.nginx-devel@nginx.org>
References: <mailman.0.1331903434.38417.nginx-devel@nginx.org>
Message-ID: <4F634141.7050507@fremnet.net>

Hi

I'm attempting to write my first plugin, it's going swimmingly except 
it's not.

I'm trying to add to the SSI module the same way the perl module does 
and have spent a lot of time flicking between the two (c isn't one of my 
strong languages).

It appears to work, the very last log line does get called, then it 
segfaults

2012/03/16 20:14:55 [emerg] 10663#0: *1 working with the bufferzzzzzz 
while sending response to client, client: 118.xx.xx.66, server: 
flatus.xx.net, request: "GET /index.html HTTP/1.1", host: 
"flatus.xx.net:8081"

2012/03/16 20:14:55 [emerg] 10663#0: *1 working with the buffer failing 
btw YqEgHCzpROFbVQPah9mA_w==,1331892955 while sending response to 
client, client: 118.xx.xx.66, server: flatus.xx.net, request: "GET 
/index.html HTTP/1.1", host: "flatus.xx.net:8081"

I've attached the plugin, such as it is...

You get the log message from line 130
ngx_log_error(NGX_LOG_EMERG, c->log, 0, "working with the buffer failing 
btw %V", &foo);

but then it promptly crashes....

Any ideas? I thought it was going along so well :(

Thanks

Shannon.
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ngx_http_secure_link_ssi_module.c
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120316/1b1734c2/attachment.c>

From goelvivek2011 at gmail.com  Fri Mar 16 14:26:48 2012
From: goelvivek2011 at gmail.com (vivek goel)
Date: Fri, 16 Mar 2012 19:56:48 +0530
Subject: How to parse post data in nginx module ?
In-Reply-To: <CAB+eYNWQ8HmuaxuOXTojDe1mbVH8OnVFBs=xNriYB2DraTmtPg@mail.gmail.com>
References: <CAB+eYNWQ8HmuaxuOXTojDe1mbVH8OnVFBs=xNriYB2DraTmtPg@mail.gmail.com>
Message-ID: <CAB+eYNXRXiUGkAJfRcoOc8zad4aXL0XVkvHMood5yaDtmTsQMQ@mail.gmail.com>

I was tried to create hello world module to read post request but I am
getting ctx as null value on line no 112
    ctx = ngx_http_get_module_ctx(r, ngx_http_helloworld_module);

I am attaching the source code

regards
Vivek Goel



On Sun, Feb 26, 2012 at 11:15 PM, vivek goel <goelvivek2011 at gmail.com>wrote:

> For reading get request I can use function ngx_http_form_input_arg but I
> couldn't find any function by using which I can read post data in nginx ?
> Is there any function in devel module or nginx api I can use to read post
> data ?
>
> regards
> Vivek Goel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120316/db59ffaf/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ngx_http_helloworld.c
Type: text/x-csrc
Size: 3023 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120316/db59ffaf/attachment.bin>

From goelvivek2011 at gmail.com  Fri Mar 16 14:28:53 2012
From: goelvivek2011 at gmail.com (vivek goel)
Date: Fri, 16 Mar 2012 19:58:53 +0530
Subject: How to parse post data in nginx module ?
In-Reply-To: <CAB+eYNXRXiUGkAJfRcoOc8zad4aXL0XVkvHMood5yaDtmTsQMQ@mail.gmail.com>
References: <CAB+eYNWQ8HmuaxuOXTojDe1mbVH8OnVFBs=xNriYB2DraTmtPg@mail.gmail.com>
	<CAB+eYNXRXiUGkAJfRcoOc8zad4aXL0XVkvHMood5yaDtmTsQMQ@mail.gmail.com>
Message-ID: <CAB+eYNVLu6SHyM8uZjV1LLzhF0aKkLq6PE1XXsDy-MobR6mVyA@mail.gmail.com>

/*
 * Copyright (C) Igor Sysoev
 * Copyright (C) Nginx, Inc.
 */

#include <ngx_config.h>
#include <ngx_core.h>
#include <ngx_http.h>



static char *ngx_http_helloworld(ngx_conf_t *cf, ngx_command_t *cmd,
        void *conf);
static void ngx_http_form_input_post_read(ngx_http_request_t *r);

typedef struct {
    ngx_flag_t done : 1;
    ngx_flag_t waiting_more_body : 1;
} ngx_http_helloworld_ctx_t;

static ngx_command_t ngx_http_helloworld_commands[] = {

    { ngx_string("helloworld"),
        NGX_HTTP_MAIN_CONF | NGX_HTTP_SRV_CONF | NGX_HTTP_LOC_CONF |
NGX_CONF_NOARGS,
        ngx_http_helloworld,
        NGX_HTTP_LOC_CONF_OFFSET,
        0,
        NULL},

    ngx_null_command
};




static u_char ngx_helloworld[] = "ABCD";


static ngx_http_module_t ngx_http_helloworld_module_ctx = {
    NULL, /* preconfiguration */
    NULL, /* postconfiguration */

    NULL, /* create main configuration */
    NULL, /* init main configuration */

    NULL, /* create server configuration */
    NULL, /* merge server configuration */

    NULL, /* create location configuration */
    NULL /* merge location configuration */
};


ngx_module_t ngx_http_helloworld_module = {
    NGX_MODULE_V1,
    &ngx_http_helloworld_module_ctx, /* module context */
    ngx_http_helloworld_commands, /* module directives */
    NGX_HTTP_MODULE, /* module type */
    NULL, /* init master */
    NULL, /* init module */
    NULL, /* init process */
    NULL, /* init thread */
    NULL, /* exit thread */
    NULL, /* exit process */
    NULL, /* exit master */
    NGX_MODULE_V1_PADDING
};


static ngx_str_t ngx_http_gif_type = ngx_string("text/html");

static ngx_int_t
ngx_http_helloworld_handler(ngx_http_request_t *r) {
    ngx_http_complex_value_t cv;
    char* response;



    if (!(r->method & (NGX_HTTP_GET | NGX_HTTP_POST))) {
        return NGX_HTTP_NOT_ALLOWED;
    }

    ngx_memzero(&cv, sizeof (ngx_http_complex_value_t));
    if (r->method == NGX_HTTP_POST) {
        ngx_int_t rc = ngx_http_read_client_request_body(r,
ngx_http_form_input_post_read);
    }

    //do some processing here
    return ngx_http_send_response(r, NGX_HTTP_OK, &ngx_http_gif_type, &cv);
}

static char *
ngx_http_helloworld(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) {
    ngx_http_core_loc_conf_t *clcf;

    clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);
    clcf->handler = ngx_http_helloworld_handler;

    return NGX_CONF_OK;
}

static void ngx_http_form_input_post_read(ngx_http_request_t *r) {
    ngx_http_helloworld_input_ctx_t *ctx;

    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
            "http form_input post read request body");

    r->read_event_handler = ngx_http_request_empty_handler;

    ctx = ngx_http_get_module_ctx(r, ngx_http_helloworld_module);
    ctx->done = 1;

#if defined(nginx_version) && nginx_version >= 8011

    r->main->count--;
#endif




    if (ctx->waiting_more_body) {
        ctx->waiting_more_body = 0;

        ngx_http_core_run_phases(r);
    }
}


regards
Vivek Goel



On Fri, Mar 16, 2012 at 7:56 PM, vivek goel <goelvivek2011 at gmail.com> wrote:

> I was tried to create hello world module to read post request but I am
> getting ctx as null value on line no 112
>     ctx = ngx_http_get_module_ctx(r, ngx_http_helloworld_module);
>
> I am attaching the source code
>
> regards
> Vivek Goel
>
>
>
> On Sun, Feb 26, 2012 at 11:15 PM, vivek goel <goelvivek2011 at gmail.com>wrote:
>
>> For reading get request I can use function ngx_http_form_input_arg but I
>> couldn't find any function by using which I can read post data in nginx ?
>> Is there any function in devel module or nginx api I can use to read post
>> data ?
>>
>> regards
>> Vivek Goel
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120316/41511e68/attachment.html>

From mikegagnon at gmail.com  Fri Mar 16 15:10:56 2012
From: mikegagnon at gmail.com (Mike Gagnon)
Date: Fri, 16 Mar 2012 08:10:56 -0700
Subject: Attempting to write a plugin
In-Reply-To: <4F634141.7050507@fremnet.net>
References: <mailman.0.1331903434.38417.nginx-devel@nginx.org>
	<4F634141.7050507@fremnet.net>
Message-ID: <CAG23spP-urh0n24bEWNfsBQuufVwzi5HLtcgYDs9VbJJT8Qe-w@mail.gmail.com>

I'm new to nginx myself, so I can't offer much nginx specific advice.

Are you familiar with the GDB debugger? It's very helpful for debugging
segfaults. The nginx core module also has some useful directives that help
with debugging, such as daemon and debug_points.

I only mention these basic tips because you said C isn't one of your strong
languages.

Cheers,
Mike Gagnon


On Fri, Mar 16, 2012 at 6:33 AM, <nginx.devlist.alias at fremnet.net> wrote:

> Hi
>
> I'm attempting to write my first plugin, it's going swimmingly except it's
> not.
>
> I'm trying to add to the SSI module the same way the perl module does and
> have spent a lot of time flicking between the two (c isn't one of my strong
> languages).
>
> It appears to work, the very last log line does get called, then it
> segfaults
>
> 2012/03/16 20:14:55 [emerg] 10663#0: *1 working with the bufferzzzzzz
> while sending response to client, client: 118.xx.xx.66, server:
> flatus.xx.net, request: "GET /index.html HTTP/1.1", host: "
> flatus.xx.net:8081"
>
> 2012/03/16 20:14:55 [emerg] 10663#0: *1 working with the buffer failing
> btw YqEgHCzpROFbVQPah9mA_w==,**1331892955 while sending response to
> client, client: 118.xx.xx.66, server: flatus.xx.net, request: "GET
> /index.html HTTP/1.1", host: "flatus.xx.net:8081"
>
> I've attached the plugin, such as it is...
>
> You get the log message from line 130
> ngx_log_error(NGX_LOG_EMERG, c->log, 0, "working with the buffer failing
> btw %V", &foo);
>
> but then it promptly crashes....
>
> Any ideas? I thought it was going along so well :(
>
> Thanks
>
> Shannon.
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120316/6ef7792e/attachment-0001.html>

From ne at vbart.ru  Fri Mar 16 15:16:08 2012
From: ne at vbart.ru (Valentin V. Bartenev)
Date: Fri, 16 Mar 2012 19:16:08 +0400
Subject: How to parse post data in nginx module ?
In-Reply-To: <CAB+eYNXRXiUGkAJfRcoOc8zad4aXL0XVkvHMood5yaDtmTsQMQ@mail.gmail.com>
References: <CAB+eYNWQ8HmuaxuOXTojDe1mbVH8OnVFBs=xNriYB2DraTmtPg@mail.gmail.com>
	<CAB+eYNXRXiUGkAJfRcoOc8zad4aXL0XVkvHMood5yaDtmTsQMQ@mail.gmail.com>
Message-ID: <201203161916.08530.ne@vbart.ru>

On Friday 16 March 2012 18:26:48 vivek goel wrote:
> I was tried to create hello world module to read post request but I am
> getting ctx as null value on line no 112
>     ctx = ngx_http_get_module_ctx(r, ngx_http_helloworld_module);
> 
> I am attaching the source code

To be able to get ctx, you must set it first by ngx_http_set_ctx() function.

 wbr, Valentin V. Bartenev


From ru at nginx.com  Fri Mar 16 19:15:33 2012
From: ru at nginx.com (ru at nginx.com)
Date: Fri, 16 Mar 2012 19:15:33 +0000
Subject: [nginx] svn commit:  r4546 - in trunk: auto src/http
Message-ID: <20120316191533.DC0773FA409@mail.nginx.com>

Author: ru
Date: 2012-03-16 19:15:33 +0000 (Fri, 16 Mar 2012)
New Revision: 4546

Log:
Implemented $tcpinfo_rtt, $tcpinfo_rttvar, $tcpinfo_snd_cwnd, and
$tcpinfo_rcv_space variables.  Supported on Linux and FreeBSD.


Modified:
   trunk/auto/unix
   trunk/src/http/ngx_http_variables.c

Modified: trunk/auto/unix
===================================================================
--- trunk/auto/unix	2012-03-16 07:33:55 UTC (rev 4545)
+++ trunk/auto/unix	2012-03-16 19:15:33 UTC (rev 4546)
@@ -343,6 +343,19 @@
 . auto/feature
 
 
+ngx_feature="TCP_INFO"
+ngx_feature_name="NGX_HAVE_TCP_INFO"
+ngx_feature_run=no
+ngx_feature_incs="#include <sys/socket.h>
+                  #include <netinet/in.h>
+                  #include <netinet/tcp.h>"
+ngx_feature_path=
+ngx_feature_libs=
+ngx_feature_test="socklen_t optlen = sizeof(struct tcp_info);
+                  getsockopt(0, IPPROTO_TCP, TCP_INFO, NULL, &optlen)"
+. auto/feature
+
+
 ngx_feature="accept4()"
 ngx_feature_name="NGX_HAVE_ACCEPT4"
 ngx_feature_run=no

Modified: trunk/src/http/ngx_http_variables.c
===================================================================
--- trunk/src/http/ngx_http_variables.c	2012-03-16 07:33:55 UTC (rev 4545)
+++ trunk/src/http/ngx_http_variables.c	2012-03-16 19:15:33 UTC (rev 4546)
@@ -34,6 +34,10 @@
     ngx_http_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_http_variable_argument(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
+#if (NGX_HAVE_TCP_INFO)
+static ngx_int_t ngx_http_variable_tcpinfo(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data);
+#endif
 
 static ngx_int_t ngx_http_variable_host(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
@@ -259,6 +263,20 @@
     { ngx_string("pid"), NULL, ngx_http_variable_pid,
       0, 0, 0 },
 
+#if (NGX_HAVE_TCP_INFO)
+    { ngx_string("tcpinfo_rtt"), NULL, ngx_http_variable_tcpinfo,
+      0, NGX_HTTP_VAR_NOCACHEABLE, 0 },
+
+    { ngx_string("tcpinfo_rttvar"), NULL, ngx_http_variable_tcpinfo,
+      1, NGX_HTTP_VAR_NOCACHEABLE, 0 },
+
+    { ngx_string("tcpinfo_snd_cwnd"), NULL, ngx_http_variable_tcpinfo,
+      2, NGX_HTTP_VAR_NOCACHEABLE, 0 },
+
+    { ngx_string("tcpinfo_rcv_space"), NULL, ngx_http_variable_tcpinfo,
+      3, NGX_HTTP_VAR_NOCACHEABLE, 0 },
+#endif
+
     { ngx_null_string, NULL, NULL, 0, 0, 0 }
 };
 
@@ -884,7 +902,62 @@
 }
 
 
+#if (NGX_HAVE_TCP_INFO)
+
 static ngx_int_t
+ngx_http_variable_tcpinfo(ngx_http_request_t *r, ngx_http_variable_value_t *v,
+    uintptr_t data)
+{
+    struct tcp_info  ti;
+    socklen_t        len;
+    uint32_t         value;
+
+    len = sizeof(struct tcp_info);
+    if (getsockopt(r->connection->fd, IPPROTO_TCP, TCP_INFO, &ti, &len) == -1) {
+        v->not_found = 1;
+        return NGX_OK;
+    }
+
+    v->data = ngx_pnalloc(r->pool, NGX_INT32_LEN);
+    if (v->data == NULL) {
+        return NGX_ERROR;
+    }
+
+    switch (data) {
+    case 0:
+        value = ti.tcpi_rtt;
+        break;
+
+    case 1:
+        value = ti.tcpi_rttvar;
+        break;
+
+    case 2:
+        value = ti.tcpi_snd_cwnd;
+        break;
+
+    case 3:
+        value = ti.tcpi_rcv_space;
+        break;
+
+    /* suppress warning */
+    default:
+        value = 0;
+        break;
+    }
+
+    v->len = ngx_sprintf(v->data, "%uD", value) - v->data;
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+
+    return NGX_OK;
+}
+
+#endif
+
+
+static ngx_int_t
 ngx_http_variable_host(ngx_http_request_t *r, ngx_http_variable_value_t *v,
     uintptr_t data)
 {


From nginx.devlist.alias at fremnet.net  Sat Mar 17 00:11:22 2012
From: nginx.devlist.alias at fremnet.net (nginx.devlist.alias at fremnet.net)
Date: Sat, 17 Mar 2012 10:11:22 +1000
Subject: Attempting to write a plugin
Message-ID: <4F63D6AA.7060707@fremnet.net>

Thanks for the reply Mike

I've been using strace to watch the progress of execution but that didn't help, I'm not sure how much gdb will help me (it's one of those tools that needs a manual of it's own).

I was kinda hoping someone who knew more about the specifics of nginx dev would take one glance and call me an idiot for not knowing X or Y specific thing and pointing the error out to me :D

I'll have a poke at it with GDB but I don't hold my breath :D

> I'm new to nginx myself, so I can't offer much nginx specific advice.
>
> Are you familiar with the GDB debugger? It's very helpful for debugging
> segfaults. The nginx core module also has some useful directives that help
> with debugging, such as daemon and debug_points.
>
> I only mention these basic tips because you said C isn't one of your strong
> languages.
>
> Cheers,
> Mike Gagnon


From zzz at zzz.org.ua  Sat Mar 17 00:18:59 2012
From: zzz at zzz.org.ua (Alexandr Gomoliako)
Date: Sat, 17 Mar 2012 02:18:59 +0200
Subject: Attempting to write a plugin
In-Reply-To: <4F63D6AA.7060707@fremnet.net>
References: <4F63D6AA.7060707@fremnet.net>
Message-ID: <CAKb9G9j1RQQUnnoVzBnq36pUvw24YtAc2Zg6DaRKdAgAXc8f2w@mail.gmail.com>

On Sat, Mar 17, 2012 at 2:11 AM,  <nginx.devlist.alias at fremnet.net> wrote:
> Thanks for the reply Mike
>
> I've been using strace to watch the progress of execution but that didn't
> help, I'm not sure how much gdb will help me (it's one of those tools that
> needs a manual of it's own).

Try this one instead:
    https://github.com/zzzcpan/ngx_trace

> I was kinda hoping someone who knew more about the specifics of nginx dev
> would take one glance and call me an idiot for not knowing X or Y specific
> thing and pointing the error out to me :D
>
> I'll have a poke at it with GDB but I don't hold my breath :D


From nginx.devlist.alias at fremnet.net  Sat Mar 17 00:54:00 2012
From: nginx.devlist.alias at fremnet.net (nginx.devlist.alias at fremnet.net)
Date: Sat, 17 Mar 2012 10:54:00 +1000
Subject: Attempting to write a plugin
In-Reply-To: <CAKb9G9j1RQQUnnoVzBnq36pUvw24YtAc2Zg6DaRKdAgAXc8f2w@mail.gmail.com>
References: <4F63D6AA.7060707@fremnet.net>
	<CAKb9G9j1RQQUnnoVzBnq36pUvw24YtAc2Zg6DaRKdAgAXc8f2w@mail.gmail.com>
Message-ID: <4F63E0A8.5050004@fremnet.net>

Thanks Alexandr

I'm afraid all that did was showed me that the crash was happening in 
the gzip filter - but it wasn't crashing before I added my module so I'm 
still lost...

+000.000002s ngx_http_secure_link_ssi 
src/http/modules/ngx_http_secure_link_ssi_module.c:54
+000.000003s ngx_http_ssi_body_filter 
src/http/modules/ngx_http_ssi_filter_module.c:371
+000.000002s ngx_http_ssi_body_filter 
src/http/modules/ngx_http_ssi_filter_module.c:371
+000.000002s ngx_http_ssi_output 
src/http/modules/ngx_http_ssi_filter_module.c:897
+000.000002s    ngx_http_postpone_filter 
src/http/ngx_http_postpone_filter_module.c:53
+000.000002s       ngx_http_gzip_body_filter 
src/http/modules/ngx_http_gzip_filter_module.c:315
+000.000017s          ngx_http_gzip_body_filter 
src/http/modules/ngx_http_gzip_filter_module.c:315
+000.000001s          ngx_http_gzip_body_filter 
src/http/modules/ngx_http_gzip_filter_module.c:315
+000.000001s          ngx_http_gzip_body_filter 
src/http/modules/ngx_http_gzip_filter_module.c:315
+000.000001s          ngx_http_gzip_body_filter 
src/http/modules/ngx_http_gzip_filter_module.c:315
+000.000002s          ngx_http_gzip_body_filter 
src/http/modules/ngx_http_gzip_filter_module.c:315
+000.000001s          ngx_http_gzip_body_filter 
src/http/modules/ngx_http_gzip_filter_module.c:315
+000.000001s          ngx_http_gzip_body_filter 
src/http/modules/ngx_http_gzip_filter_module.c:315
+000.000001s          ngx_http_gzip_body_filter 
src/http/modules/ngx_http_gzip_filter_module.c:315


On 17/03/2012 10:18 AM, Alexandr Gomoliako wrote:
> On Sat, Mar 17, 2012 at 2:11 AM,<nginx.devlist.alias at fremnet.net>  wrote:
>> Thanks for the reply Mike
>>
>> I've been using strace to watch the progress of execution but that didn't
>> help, I'm not sure how much gdb will help me (it's one of those tools that
>> needs a manual of it's own).
> Try this one instead:
>      https://github.com/zzzcpan/ngx_trace
>
>> I was kinda hoping someone who knew more about the specifics of nginx dev
>> would take one glance and call me an idiot for not knowing X or Y specific
>> thing and pointing the error out to me :D
>>
>> I'll have a poke at it with GDB but I don't hold my breath :D
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel


From zzz at zzz.org.ua  Sat Mar 17 01:29:12 2012
From: zzz at zzz.org.ua (Alexandr Gomoliako)
Date: Sat, 17 Mar 2012 03:29:12 +0200
Subject: Attempting to write a plugin
In-Reply-To: <4F63E0A8.5050004@fremnet.net>
References: <4F63D6AA.7060707@fremnet.net>
	<CAKb9G9j1RQQUnnoVzBnq36pUvw24YtAc2Zg6DaRKdAgAXc8f2w@mail.gmail.com>
	<4F63E0A8.5050004@fremnet.net>
Message-ID: <CAKb9G9gi96=X_BjMxQ2OXXTLN+X13NEbOuKMd7gKhQSbrw0KQA@mail.gmail.com>

> I'm afraid all that did was showed me that the crash was happening in the
> gzip filter - but it wasn't crashing before I added my module so I'm still
> lost...

I guess you have no choice but to play with gdb. It's not that hard actually:

    gdb --args ./objs/nginx -p mynginx
    > r
    ...
    > bt


From goelvivek2011 at gmail.com  Sat Mar 17 04:50:13 2012
From: goelvivek2011 at gmail.com (vivek goel)
Date: Sat, 17 Mar 2012 10:20:13 +0530
Subject: How to parse post data in nginx module ?
In-Reply-To: <CAB+eYNXRXiUGkAJfRcoOc8zad4aXL0XVkvHMood5yaDtmTsQMQ@mail.gmail.com>
References: <CAB+eYNWQ8HmuaxuOXTojDe1mbVH8OnVFBs=xNriYB2DraTmtPg@mail.gmail.com>
	<CAB+eYNXRXiUGkAJfRcoOc8zad4aXL0XVkvHMood5yaDtmTsQMQ@mail.gmail.com>
Message-ID: <CAB+eYNVatDr82KE0eUen6GS0Jx61xbEwG4NaOU-Etoxd6LBCRw@mail.gmail.com>

Thanks by using ngx_http_set_ctx now problem is fixed.
But now I am getting strange error with post request  many times I am
getting request timed out or connection reset by pear.
What I might be doing wrong  while reading post request ?


regards
Vivek Goel



On Fri, Mar 16, 2012 at 7:56 PM, vivek goel <goelvivek2011 at gmail.com> wrote:

> I was tried to create hello world module to read post request but I am
> getting ctx as null value on line no 112
>     ctx = ngx_http_get_module_ctx(r, ngx_http_helloworld_module);
>
> I am attaching the source code
>
> regards
> Vivek Goel
>
>
>
> On Sun, Feb 26, 2012 at 11:15 PM, vivek goel <goelvivek2011 at gmail.com>wrote:
>
>> For reading get request I can use function ngx_http_form_input_arg but I
>> couldn't find any function by using which I can read post data in nginx ?
>> Is there any function in devel module or nginx api I can use to read post
>> data ?
>>
>> regards
>> Vivek Goel
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120317/f414106a/attachment.html>

From nginx.devlist.alias at fremnet.net  Sun Mar 18 11:26:09 2012
From: nginx.devlist.alias at fremnet.net (nginx.devlist.alias at fremnet.net)
Date: Sun, 18 Mar 2012 21:26:09 +1000
Subject: Attempting to write a plugin
In-Reply-To: <CAKb9G9gi96=X_BjMxQ2OXXTLN+X13NEbOuKMd7gKhQSbrw0KQA@mail.gmail.com>
References: <4F63D6AA.7060707@fremnet.net>
	<CAKb9G9j1RQQUnnoVzBnq36pUvw24YtAc2Zg6DaRKdAgAXc8f2w@mail.gmail.com>
	<4F63E0A8.5050004@fremnet.net>
	<CAKb9G9gi96=X_BjMxQ2OXXTLN+X13NEbOuKMd7gKhQSbrw0KQA@mail.gmail.com>
Message-ID: <4F65C651.5090600@fremnet.net>

So in the end, I gave up on C and made a perl module... not what I 
wanted but it works - and a whole lot less code

package SecureLink;

use strict;
use warnings;
use nginx;
use Digest::MD5 qw/md5/;
use MIME::Base64;

sub secure {
         my ($r, $secret, $path, $ttl) = @_;
         $ttl //= 1800;
         my $timeout = time() + $ttl;
         my $secure = encode_base64(md5("$timeout.$path.$secret"));
         $secure =~ tr!+/!-_!;
         chomp($secure);
         $r->print("$secure,$timeout");
         return OK;
}


On 17/03/2012 11:29 AM, Alexandr Gomoliako wrote:
>> I'm afraid all that did was showed me that the crash was happening in the
>> gzip filter - but it wasn't crashing before I added my module so I'm still
>> lost...
> I guess you have no choice but to play with gdb. It's not that hard actually:
>
>      gdb --args ./objs/nginx -p mynginx
>      >  r
>      ...
>      >  bt
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel


From pasik at iki.fi  Mon Mar 19 08:33:04 2012
From: pasik at iki.fi (Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?=)
Date: Mon, 19 Mar 2012 10:33:04 +0200
Subject: Manipulating the body of a PUT/POST
In-Reply-To: <20120309142308.GI67687@mdounin.ru>
References: <4F2B9FEE.3060601@one.com> <20120203100450.GH67687@mdounin.ru>
	<4F58BBBC.9000903@one.com> <20120308201058.GZ12984@reaktio.net>
	<20120309142308.GI67687@mdounin.ru>
Message-ID: <20120319083304.GI12984@reaktio.net>

On Fri, Mar 09, 2012 at 06:23:08PM +0400, Maxim Dounin wrote:
> Hello!
> 
> On Thu, Mar 08, 2012 at 10:10:59PM +0200, Pasi K?rkk?inen wrote:
> 
> > On Thu, Mar 08, 2012 at 03:01:32PM +0100, Adam Hasselbalch Hansen wrote:
> > > On 2012-02-03 11:04, Maxim Dounin wrote:
> > >
> > >> I'm working on prototyping input body filtering, it's expected to
> > >> appear somewhere in 1.2.x.  It will allow manipulation of request
> > >> body, as well as other tasks like content inspection and so on.
> > >
> > > Any ETA on that, btw?
> > >
> > 
> > also does this prototyping work allow to bypass nginx local saving of put/post requests to a temp file?
> > I'm working on disabling nginx local caching of uploads..
> 
> It's considered as a "nice to have somewhere in the future", but 
> not as the goal of this particular work.
> 

Ok. If you have any ideas about how to properly implement cache-bypass for put/post,
let me know.. I'm planning to work on this in the near future, and I'd appreciate any comments/pointers.

-- Pasi


From goelvivek2011 at gmail.com  Mon Mar 19 12:33:46 2012
From: goelvivek2011 at gmail.com (vivek goel)
Date: Mon, 19 Mar 2012 18:03:46 +0530
Subject: Random timeout while using ngx_http_read_client_request_body
Message-ID: <CAB+eYNUBW-wjouXpdatVqqqjj_oJA65JesbORT-YH7Ntccn4iQ@mail.gmail.com>

I am writing my own module to generate contents.
I am getting random time-outs with it.
What can be the issue ?

Following is the source code for my module


/*
 * Copyright (C) Igor Sysoev
 * Copyright (C) Nginx, Inc.
 */

#include <ngx_config.h>
#include <ngx_core.h>
#include <ngx_http.h>

typedef struct {
    ngx_flag_t done : 1;
    ngx_flag_t waiting_more_body : 1;
} ngx_http_helloworld_ctx_t;


static char *ngx_http_helloworld(ngx_conf_t *cf, ngx_command_t *cmd,
        void *conf);
static void ngx_http_form_input_post_read(ngx_http_request_t *r);

static ngx_command_t ngx_http_helloworld_commands[] = {

    { ngx_string("helloworld"),
        NGX_HTTP_MAIN_CONF | NGX_HTTP_SRV_CONF | NGX_HTTP_LOC_CONF |
NGX_CONF_NOARGS,
        ngx_http_helloworld,
        0,
        0,
        NULL},

    ngx_null_command
};




//static u_char ngx_helloworld[] = "ABCD";


static ngx_http_module_t ngx_http_helloworld_module_ctx = {
    NULL, /* preconfiguration */
    NULL, /* postconfiguration */

    NULL, /* create main configuration */
    NULL, /* init main configuration */

    NULL, /* create server configuration */
    NULL, /* merge server configuration */

    NULL, /* create location configuration */
    NULL /* merge location configuration */
};


ngx_module_t ngx_http_helloworld_module = {
    NGX_MODULE_V1,
    &ngx_http_helloworld_module_ctx, /* module context */
    ngx_http_helloworld_commands, /* module directives */
    NGX_HTTP_MODULE, /* module type */
    NULL, /* init master */
    NULL, /* init module */
    NULL, /* init process */
    NULL, /* init thread */
    NULL, /* exit thread */
    NULL, /* exit process */
    NULL, /* exit master */
    NGX_MODULE_V1_PADDING
};


static ngx_str_t ngx_http_gif_type = ngx_string("text/html");

static ngx_int_t
ngx_http_helloworld_handler(ngx_http_request_t *r) {
    ngx_http_complex_value_t cv;
    ngx_http_helloworld_ctx_t *ctx;



    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
            "http form_input rewrite phase handler");

    ctx = ngx_http_get_module_ctx(r, ngx_http_helloworld_module);

    if (ctx != NULL) {
        if (ctx->done) {
            ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                    "http form_input rewrite phase handler done");

            return NGX_DECLINED;
        }

        return NGX_DONE;
    }

    ctx = ngx_pcalloc(r->pool, sizeof (ngx_http_helloworld_ctx_t));
    if (ctx == NULL) {
        return NGX_ERROR;
    }

    /* set by ngx_pcalloc:
     *      ctx->done = 0;
     *      ctx->waiting_more_body = 0;
     */

    ngx_http_set_ctx(r, ctx, ngx_http_helloworld_module);





    if (!(r->method & (NGX_HTTP_GET | NGX_HTTP_POST))) {
        return NGX_HTTP_NOT_ALLOWED;
    }

    ngx_memzero(&cv, sizeof (ngx_http_complex_value_t));
    if (r->method == NGX_HTTP_POST) {
        ngx_int_t rc = ngx_http_read_client_request_body(r,
ngx_http_form_input_post_read);
        if (rc) {
        }
    }

    /*
          ngx_str_t b;
          ngx_str_set(&b,"/b");
          return ngx_http_internal_redirect(r, &b,NULL);
     */
    return ngx_http_send_response(r, NGX_HTTP_OK, &ngx_http_gif_type, &cv);
}

static char *
ngx_http_helloworld(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) {
    ngx_http_core_loc_conf_t *clcf;

    clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);
    clcf->handler = ngx_http_helloworld_handler;

    return NGX_CONF_OK;
}

static void ngx_http_form_input_post_read(ngx_http_request_t *r) {
    ngx_http_helloworld_ctx_t *ctx;

    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
            "http form_input post read request body");

    r->read_event_handler = ngx_http_request_empty_handler;

    ctx = ngx_http_get_module_ctx(r, ngx_http_helloworld_module);
    if (ctx == NULL) {
        return;
    }
    ctx->done = 1;

#if defined(nginx_version) && nginx_version >= 8011

    r->main->count--;
#endif




    if (ctx->waiting_more_body) {
        ctx->waiting_more_body = 0;

        ngx_http_core_run_phases(r);
    }
}


regards
Vivek Goel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120319/f20ef104/attachment.html>

From mdounin at mdounin.ru  Mon Mar 19 12:48:01 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 19 Mar 2012 16:48:01 +0400
Subject: Random timeout while using ngx_http_read_client_request_body
In-Reply-To: <CAB+eYNUBW-wjouXpdatVqqqjj_oJA65JesbORT-YH7Ntccn4iQ@mail.gmail.com>
References: <CAB+eYNUBW-wjouXpdatVqqqjj_oJA65JesbORT-YH7Ntccn4iQ@mail.gmail.com>
Message-ID: <20120319124800.GU67687@mdounin.ru>

Hello!

On Mon, Mar 19, 2012 at 06:03:46PM +0530, vivek goel wrote:

> I am writing my own module to generate contents.
> I am getting random time-outs with it.
> What can be the issue ?

You incorrectly handle ngx_http_read_client_request_body().  See 
nginx sources for correct examples.

Maxim Dounin


From goelvivek2011 at gmail.com  Mon Mar 19 14:13:19 2012
From: goelvivek2011 at gmail.com (vivek goel)
Date: Mon, 19 Mar 2012 19:43:19 +0530
Subject: Random timeout while using ngx_http_read_client_request_body
In-Reply-To: <CAB+eYNUBW-wjouXpdatVqqqjj_oJA65JesbORT-YH7Ntccn4iQ@mail.gmail.com>
References: <CAB+eYNUBW-wjouXpdatVqqqjj_oJA65JesbORT-YH7Ntccn4iQ@mail.gmail.com>
Message-ID: <CAB+eYNV8DhR+j18--Z1h-VPoCoVFXpjxcEzL05rk8WQeoW8Dtw@mail.gmail.com>

Hi* *Maxim
Can you please tell the file name I should be looking for correct example
of using  ngx_http_read_client_request_body  for content handler module ?


regards
Vivek Goel



On Mon, Mar 19, 2012 at 6:03 PM, vivek goel <goelvivek2011 at gmail.com> wrote:

> I am writing my own module to generate contents.
> I am getting random time-outs with it.
> What can be the issue ?
>
> Following is the source code for my module
>
>
> /*
>  * Copyright (C) Igor Sysoev
>  * Copyright (C) Nginx, Inc.
>  */
>
> #include <ngx_config.h>
> #include <ngx_core.h>
> #include <ngx_http.h>
>
> typedef struct {
>     ngx_flag_t done : 1;
>     ngx_flag_t waiting_more_body : 1;
> } ngx_http_helloworld_ctx_t;
>
>
> static char *ngx_http_helloworld(ngx_conf_t *cf, ngx_command_t *cmd,
>         void *conf);
> static void ngx_http_form_input_post_read(ngx_http_request_t *r);
>
> static ngx_command_t ngx_http_helloworld_commands[] = {
>
>     { ngx_string("helloworld"),
>         NGX_HTTP_MAIN_CONF | NGX_HTTP_SRV_CONF | NGX_HTTP_LOC_CONF |
> NGX_CONF_NOARGS,
>         ngx_http_helloworld,
>         0,
>         0,
>         NULL},
>
>     ngx_null_command
> };
>
>
>
>
> //static u_char ngx_helloworld[] = "ABCD";
>
>
> static ngx_http_module_t ngx_http_helloworld_module_ctx = {
>     NULL, /* preconfiguration */
>     NULL, /* postconfiguration */
>
>     NULL, /* create main configuration */
>     NULL, /* init main configuration */
>
>     NULL, /* create server configuration */
>     NULL, /* merge server configuration */
>
>     NULL, /* create location configuration */
>     NULL /* merge location configuration */
> };
>
>
> ngx_module_t ngx_http_helloworld_module = {
>     NGX_MODULE_V1,
>     &ngx_http_helloworld_module_ctx, /* module context */
>     ngx_http_helloworld_commands, /* module directives */
>     NGX_HTTP_MODULE, /* module type */
>     NULL, /* init master */
>     NULL, /* init module */
>     NULL, /* init process */
>     NULL, /* init thread */
>     NULL, /* exit thread */
>     NULL, /* exit process */
>     NULL, /* exit master */
>     NGX_MODULE_V1_PADDING
> };
>
>
> static ngx_str_t ngx_http_gif_type = ngx_string("text/html");
>
> static ngx_int_t
> ngx_http_helloworld_handler(ngx_http_request_t *r) {
>     ngx_http_complex_value_t cv;
>     ngx_http_helloworld_ctx_t *ctx;
>
>
>
>     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
>             "http form_input rewrite phase handler");
>
>     ctx = ngx_http_get_module_ctx(r, ngx_http_helloworld_module);
>
>     if (ctx != NULL) {
>         if (ctx->done) {
>             ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
>                     "http form_input rewrite phase handler done");
>
>             return NGX_DECLINED;
>         }
>
>         return NGX_DONE;
>     }
>
>     ctx = ngx_pcalloc(r->pool, sizeof (ngx_http_helloworld_ctx_t));
>     if (ctx == NULL) {
>         return NGX_ERROR;
>     }
>
>     /* set by ngx_pcalloc:
>      *      ctx->done = 0;
>      *      ctx->waiting_more_body = 0;
>      */
>
>     ngx_http_set_ctx(r, ctx, ngx_http_helloworld_module);
>
>
>
>
>
>     if (!(r->method & (NGX_HTTP_GET | NGX_HTTP_POST))) {
>         return NGX_HTTP_NOT_ALLOWED;
>     }
>
>     ngx_memzero(&cv, sizeof (ngx_http_complex_value_t));
>     if (r->method == NGX_HTTP_POST) {
>         ngx_int_t rc = ngx_http_read_client_request_body(r,
> ngx_http_form_input_post_read);
>         if (rc) {
>         }
>     }
>
>     /*
>           ngx_str_t b;
>           ngx_str_set(&b,"/b");
>           return ngx_http_internal_redirect(r, &b,NULL);
>      */
>     return ngx_http_send_response(r, NGX_HTTP_OK, &ngx_http_gif_type, &cv);
> }
>
> static char *
> ngx_http_helloworld(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) {
>     ngx_http_core_loc_conf_t *clcf;
>
>     clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);
>     clcf->handler = ngx_http_helloworld_handler;
>
>     return NGX_CONF_OK;
> }
>
> static void ngx_http_form_input_post_read(ngx_http_request_t *r) {
>     ngx_http_helloworld_ctx_t *ctx;
>
>     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
>             "http form_input post read request body");
>
>     r->read_event_handler = ngx_http_request_empty_handler;
>
>     ctx = ngx_http_get_module_ctx(r, ngx_http_helloworld_module);
>     if (ctx == NULL) {
>         return;
>     }
>     ctx->done = 1;
>
> #if defined(nginx_version) && nginx_version >= 8011
>
>     r->main->count--;
> #endif
>
>
>
>
>     if (ctx->waiting_more_body) {
>         ctx->waiting_more_body = 0;
>
>         ngx_http_core_run_phases(r);
>     }
> }
>
>
> regards
> Vivek Goel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120319/7096a02b/attachment.html>

From maxim at nginx.com  Mon Mar 19 14:57:30 2012
From: maxim at nginx.com (maxim at nginx.com)
Date: Mon, 19 Mar 2012 14:57:30 +0000
Subject: [nginx] svn commit:  r4547 - trunk/src/http
Message-ID: <20120319145731.009103FA295@mail.nginx.com>

Author: maxim
Date: 2012-03-19 14:57:29 +0000 (Mon, 19 Mar 2012)
New Revision: 4547

Log:
For the sake of case/switch code readability, 'fall through'
comments added.


Modified:
   trunk/src/http/ngx_http_parse.c
   trunk/src/http/ngx_http_upstream.c

Modified: trunk/src/http/ngx_http_parse.c
===================================================================
--- trunk/src/http/ngx_http_parse.c	2012-03-16 19:15:33 UTC (rev 4546)
+++ trunk/src/http/ngx_http_parse.c	2012-03-19 14:57:29 UTC (rev 4547)
@@ -1153,6 +1153,7 @@
                 break;
             case '+':
                 r->plus_in_uri = 1;
+                /* fall through */
             default:
                 *u++ = ch;
                 break;

Modified: trunk/src/http/ngx_http_upstream.c
===================================================================
--- trunk/src/http/ngx_http_upstream.c	2012-03-16 19:15:33 UTC (rev 4546)
+++ trunk/src/http/ngx_http_upstream.c	2012-03-19 14:57:29 UTC (rev 4547)
@@ -3312,6 +3312,8 @@
         switch (n) {
         case 0:
             u->cacheable = 0;
+            /* fall through */
+
         case NGX_ERROR:
             return NGX_OK;
 
@@ -4460,6 +4462,8 @@
     case NGX_DECLINED:
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "invalid address \"%V\"", &value[1]);
+        /* fall through */
+
     default:
         return NGX_CONF_ERROR;
     }


From mdounin at mdounin.ru  Mon Mar 19 17:03:27 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 19 Mar 2012 21:03:27 +0400
Subject: HTML parsing support for xslt module
In-Reply-To: <CAOycyLQHaKkehFm_mO3HEJh9H4Y+dii4kuLKWRVAVtw4b4r3tg@mail.gmail.com>
References: <CAOycyLQHaKkehFm_mO3HEJh9H4Y+dii4kuLKWRVAVtw4b4r3tg@mail.gmail.com>
Message-ID: <20120319170327.GX67687@mdounin.ru>

Hello!

On Wed, Mar 14, 2012 at 05:03:56PM +0000, Laurence Rowe wrote:

> I'd like to submit the attached patch which implements an
> ``xslt_html_parser`` directive for consideration. When enabled, the
> xslt module uses the libxml2 HTMLParser to parse the response body.
> This is useful for people who want to transform HTML using XSLT,
> including users of Diazo deploying on Nginx
> (http://docs.diazo.org/en/latest/deployment.html#nginx).
> 
> The patch is generated from my repository at
> https://bitbucket.org/lrowe/nginx-xslt-html-parser, forked from
> http://mdounin.ru/hg/nginx-vendor-current/. The xslt_param patch from
> http://mailman.nginx.org/pipermail/nginx-devel/2012-March/001926.html
> is included in my repository. I'll discuss each of the individual
> changesets briefly below:
> 
> changeset:   668:bf4d14f51436
> user:        Laurence Rowe <laurence at lrowe.co.uk>
> date:        Sun Jul 11 23:54:08 2010 +0100
> summary:     Skip transform when there is no content (e.g. a proxied redirect)
> 
> This was originally reviewed in
> http://mailman.nginx.org/pipermail/nginx-devel/2010-July/000390.html:
> 
> > Currently the way to disable XSLT processing is MIME type, "text/xml"
> > by default. Redirects usually have "text/html" type.
> 
> To parse HTML you have to set ``xslt_types text/html;``. This
> changeset prevents crashing on responses with an empty body.

I don't think that skipping transformation based on 
content_length_n is a correct behaviour.

1) There are more than one way to represent empty response, and 
content_length_n == 0 is just one of them.

2) And the empty response is just another case of malformed input, 
I see no reason to handle this specially.

> changeset:   669:9487b3a0e3ff
> user:        Laurence Rowe <laurence at lrowe.co.uk>
> date:        Fri Mar 09 21:01:25 2012 +0000
> summary:     Use xmlCtxtUseOptions to set options.
> 
> This changeset moves most option setting to the xmlCtxtUseOptions
> (foundational to next commit.)

This doesn't produce identical ctxt->loadsubset, and hence need 
more details.  I tend to think the current code is wrong and 
your's is ok, though it would be good to see confirmation.

(Additionally, there is a style issue in this patch.)

> changeset:   670:c8349ca87381
> user:        Laurence Rowe <laurence at lrowe.co.uk>
> date:        Fri Mar 09 21:08:18 2012 +0000
> summary:     XML_PARSE_COMPACT to save memory
> 
> We can use XML_PARSE_COMPACT as the parsed input document is not
> modified (XSLT creates a new result document.)

Is it supported in libxml2 versions used by various OSes now?  It 
appeared relatively recently, in libxml2 2.6.21, and if some OSes 
still use older versions by default - this may need configure 
test.

Another question to consider - is it actually safe to use?  Note 
that nginx does modify input document to provide DTDs.

It would be also cool to see some stats on real world memory usage 
with and without this option set.

> changeset:   671:7145bd8cc1e2
> tag:         tip
> user:        Laurence Rowe <laurence at lrowe.co.uk>
> date:        Fri Mar 09 21:47:46 2012 +0000
> summary:     xslt_html_parser
> 
> This changeset adds the ``xslt_html_parser`` directive and uses the
> HTMLParser when it is set. HTML parsing is performed with
> HTML_PARSE_RECOVER as real-world HTML may not be well formed, error
> handling is thus disabled when this option is set.

I in general like the feature, though I tend to think this patch 
needs more polishing.  In no particular order:

1) Hardcoded "utf-8" charset detection scares me.

2) The are no error handling for html code, and while it's 
probably ok to ignore parsing problems - it's certainly not ok to 
ignore fatal problems like memory allocation ones.  

3) DTD and entities handling needs clarification.

Maxim Dounin

p.s. You may want to use patchbomb mercurial extension (hg email) 
to submit patches.


From snowjn at aol.com  Mon Mar 19 17:20:24 2012
From: snowjn at aol.com (John Snow)
Date: Mon, 19 Mar 2012 13:20:24 -0400
Subject: Support for IP blocking in the Mail protocols.
Message-ID: <4F676AD8.4080809@aol.com>

Has anyone written a patch that could block access to mail from certain 
IP addresses?    I'd like to do a DNSBL check on each connection, before 
even sending the protocol banner. 

Thanks,

john.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120319/6b001a09/attachment.html>

From bmoran at onehub.com  Tue Mar 20 04:31:59 2012
From: bmoran at onehub.com (Brian Moran)
Date: Mon, 19 Mar 2012 21:31:59 -0700
Subject: Handling a subrequest response - determining upstream vs. static file
	and size
Message-ID: <CAEQ+dFOHh3HWE5bxd-0FCXKzKXwDzs8QjQyURqi0dEhK7fGTxQ@mail.gmail.com>

I am modifying an existing working filter module. I am issuing a
subrequest, which is to a URI, or to a static local file (via a location in
the config); I'd like to determine the size of the response that I'm
getting back from the subrequest.

In the subrequest's header filter, I'm doing this to find out the response
size:
if (r->upstream) {
r->upstream->headers_in.content_length_n

-- 
e: bmoran at onehub.com
p: +1 206 390 4376

Onehub, Inc.
www.onehub.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120319/aa381b5b/attachment.html>

From bmoran at onehub.com  Tue Mar 20 04:34:56 2012
From: bmoran at onehub.com (Brian Moran)
Date: Mon, 19 Mar 2012 21:34:56 -0700
Subject: Handling a subrequest response - determining upstream vs. static
	file and size
In-Reply-To: <CAEQ+dFOHh3HWE5bxd-0FCXKzKXwDzs8QjQyURqi0dEhK7fGTxQ@mail.gmail.com>
References: <CAEQ+dFOHh3HWE5bxd-0FCXKzKXwDzs8QjQyURqi0dEhK7fGTxQ@mail.gmail.com>
Message-ID: <CAEQ+dFN_sWmKduY2rJRUj2RQd5SFi76YvsXVWFyVyrFDeeih6A@mail.gmail.com>

Sorry about that, inadvertently hit send...

I am modifying an existing working filter module. I am issuing a
subrequest, which is to a URI, or to a static local file (via a location in
the config); I'd like to determine the size of the response that I'm
getting back from the subrequest.

In the subrequest's header filter, I'm doing this to find out the response
size:
if (r->upstream) {
   response_size = r->upstream->headers_in.content_length_n;
 ...
}

For the subrequests that result in the static file being streamed from
disk, r->upstream is NULL.
What's the right nginx way to determine in my filter what the size of the
'body' part of the subrequest response is going to be?

Thanks!


On Mon, Mar 19, 2012 at 9:31 PM, Brian Moran <bmoran at onehub.com> wrote:

> --
> e: bmoran at onehub.com
> p: +1 206 390 4376
>
> Onehub, Inc.
> www.onehub.com
>
>


-- 
e: bmoran at onehub.com
p: +1 206 390 4376

Onehub, Inc.
www.onehub.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120319/8c0fece7/attachment.html>

From zealot33 at gmail.com  Tue Mar 20 08:04:16 2012
From: zealot33 at gmail.com (=?EUC-KR?B?w9a/tbyu?=)
Date: Tue, 20 Mar 2012 17:04:16 +0900
Subject: Handling a subrequest response - determining upstream vs. static
	file and size
In-Reply-To: <CAEQ+dFN_sWmKduY2rJRUj2RQd5SFi76YvsXVWFyVyrFDeeih6A@mail.gmail.com>
References: <CAEQ+dFOHh3HWE5bxd-0FCXKzKXwDzs8QjQyURqi0dEhK7fGTxQ@mail.gmail.com>
	<CAEQ+dFN_sWmKduY2rJRUj2RQd5SFi76YvsXVWFyVyrFDeeih6A@mail.gmail.com>
Message-ID: <CAK-vuDE5i4q261gwnq+ePmBijLYEWRL49RADo0FogTHnxXjQCw@mail.gmail.com>

Hi.

You can refer echo-nginx-module written by agentzh.
There's a *ngx_http_echo_subrequest.c *file in
https://github.com/agentzh/echo-nginx-module

There's lots of sub-request handling code in this file.
I'm on the way of reading this module.
Since the subrequest is itself a request.
There's no difference with getting response body size of a normal request.

272 if (body_str != NULL && body_str->len) {
273        rb = ngx_pcalloc(r->pool, sizeof(ngx_http_request_body_t));
274
275        if (rb == NULL) {
276            return NGX_HTTP_INTERNAL_SERVER_ERROR;
277        }
278
279        parsed_sr->content_length_n = body_str->len;
280
281        b = ngx_calloc_buf(r->pool);
282        if (b == NULL) {
283            return NGX_HTTP_INTERNAL_SERVER_ERROR;
284        }


To extract a subrequest from a request, I guess there's a function named
ngx_http_subrequest in ngx_http_core_module.c

Hope this can help you!

2012/3/20 Brian Moran <bmoran at onehub.com>

> Sorry about that, inadvertently hit send...
>
> I am modifying an existing working filter module. I am issuing a
> subrequest, which is to a URI, or to a static local file (via a location in
> the config); I'd like to determine the size of the response that I'm
> getting back from the subrequest.
>
> In the subrequest's header filter, I'm doing this to find out the response
> size:
> if (r->upstream) {
>    response_size = r->upstream->headers_in.content_length_n;
>  ...
> }
>
> For the subrequests that result in the static file being streamed from
> disk, r->upstream is NULL.
> What's the right nginx way to determine in my filter what the size of the
> 'body' part of the subrequest response is going to be?
>
> Thanks!
>
>
> On Mon, Mar 19, 2012 at 9:31 PM, Brian Moran <bmoran at onehub.com> wrote:
>
>> --
>> e: bmoran at onehub.com
>> p: +1 206 390 4376
>>
>> Onehub, Inc.
>> www.onehub.com
>>
>>
>
>
> --
> e: bmoran at onehub.com
> p: +1 206 390 4376
>
> Onehub, Inc.
> www.onehub.com
>
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120320/edb16992/attachment-0001.html>

From mdounin at mdounin.ru  Tue Mar 20 08:21:38 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 20 Mar 2012 12:21:38 +0400
Subject: Handling a subrequest response - determining upstream vs. static
	file and size
In-Reply-To: <CAEQ+dFN_sWmKduY2rJRUj2RQd5SFi76YvsXVWFyVyrFDeeih6A@mail.gmail.com>
References: <CAEQ+dFOHh3HWE5bxd-0FCXKzKXwDzs8QjQyURqi0dEhK7fGTxQ@mail.gmail.com>
	<CAEQ+dFN_sWmKduY2rJRUj2RQd5SFi76YvsXVWFyVyrFDeeih6A@mail.gmail.com>
Message-ID: <20120320082138.GC67687@mdounin.ru>

Hello!

On Mon, Mar 19, 2012 at 09:34:56PM -0700, Brian Moran wrote:

> Sorry about that, inadvertently hit send...
> 
> I am modifying an existing working filter module. I am issuing a
> subrequest, which is to a URI, or to a static local file (via a location in
> the config); I'd like to determine the size of the response that I'm
> getting back from the subrequest.
> 
> In the subrequest's header filter, I'm doing this to find out the response
> size:
> if (r->upstream) {
>    response_size = r->upstream->headers_in.content_length_n;
>  ...
> }
> 
> For the subrequests that result in the static file being streamed from
> disk, r->upstream is NULL.
> What's the right nginx way to determine in my filter what the size of the
> 'body' part of the subrequest response is going to be?

In both cases correct way to estimate response size (if known) is 
to check r->headers_out->content_length_n.  Note well: it may be 
set to -1, which means response size isn't known.

Maxim Dounin


From adelino at ainou.net  Tue Mar 20 11:34:57 2012
From: adelino at ainou.net (Adelino Monteiro)
Date: Tue, 20 Mar 2012 11:34:57 +0000
Subject: Access to memcache variable in module
Message-ID: <CAPU-3ORctaP2Xeus08U0GxrYsbdiX+aMcQ63U8kjpJto6c_jHg@mail.gmail.com>

Hello,

This maybe a stupidly simple question but I haven't found an answer yet.
I have a working nginx module where I process one of the http variabels
using ngx_http_arg function ngxinx provides.
What I'm doing know is to have this same http variable being searching in
memcache and if found I want to pass that value to my module.

What would be the correct way to achieve this?

Thanks for all the help.

AM
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120320/f8374c09/attachment.html>

From goelvivek2011 at gmail.com  Tue Mar 20 11:53:47 2012
From: goelvivek2011 at gmail.com (vivek goel)
Date: Tue, 20 Mar 2012 17:23:47 +0530
Subject: Nginx http module putting config in NGX_MAIN_CONF
Message-ID: <CAB+eYNVg92pzr7zn4Cmk12Qpn5Z0trar0OuCgynM7pZv+7A6ng@mail.gmail.com>

hi,
I want to put variable inside nginx NGX_MAIN_CONF

I want written code like this

 { ngx_string("mydata"),
   NGX_MAIN_CONF|NGX_DIRECT_CONF|NGX_CONF_TAKE1,
   ngx_conf_set_str_slot,
   0,
   offsetof(ngx_http_my_main_conf_t, my_data),
   NULL },


But when I put in in configuration file
worker_processes  2;
mydata "a";

I am getting following error
nginx: [emerg] unknown directive "mydata"

How to fix this ?


regards
Vivek Goel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120320/b1e7796b/attachment.html>

From mdounin at mdounin.ru  Tue Mar 20 12:39:17 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 20 Mar 2012 16:39:17 +0400
Subject: Access to memcache variable in module
In-Reply-To: <CAPU-3ORctaP2Xeus08U0GxrYsbdiX+aMcQ63U8kjpJto6c_jHg@mail.gmail.com>
References: <CAPU-3ORctaP2Xeus08U0GxrYsbdiX+aMcQ63U8kjpJto6c_jHg@mail.gmail.com>
Message-ID: <20120320123917.GK67687@mdounin.ru>

Hello!

On Tue, Mar 20, 2012 at 11:34:57AM +0000, Adelino Monteiro wrote:

> Hello,
> 
> This maybe a stupidly simple question but I haven't found an answer yet.
> I have a working nginx module where I process one of the http variabels
> using ngx_http_arg function ngxinx provides.
> What I'm doing know is to have this same http variable being searching in
> memcache and if found I want to pass that value to my module.
> 
> What would be the correct way to achieve this?

The ngx_http_subrequest() function with 
NGX_HTTP_SUBREQUEST_IN_MEMORY flag should do the trick.  Take a 
look at ngx_http_ssi_filter_module.c for an example.

Maxim Dounin


From mdounin at mdounin.ru  Tue Mar 20 12:52:23 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 20 Mar 2012 16:52:23 +0400
Subject: Nginx http module putting config in NGX_MAIN_CONF
In-Reply-To: <CAB+eYNVg92pzr7zn4Cmk12Qpn5Z0trar0OuCgynM7pZv+7A6ng@mail.gmail.com>
References: <CAB+eYNVg92pzr7zn4Cmk12Qpn5Z0trar0OuCgynM7pZv+7A6ng@mail.gmail.com>
Message-ID: <20120320125222.GL67687@mdounin.ru>

Hello!

On Tue, Mar 20, 2012 at 05:23:47PM +0530, vivek goel wrote:

> hi,
> I want to put variable inside nginx NGX_MAIN_CONF
> 
> I want written code like this
> 
>  { ngx_string("mydata"),
>    NGX_MAIN_CONF|NGX_DIRECT_CONF|NGX_CONF_TAKE1,
>    ngx_conf_set_str_slot,
>    0,
>    offsetof(ngx_http_my_main_conf_t, my_data),
>    NULL },
> 
> 
> But when I put in in configuration file
> worker_processes  2;
> mydata "a";
> 
> I am getting following error
> nginx: [emerg] unknown directive "mydata"
> 
> How to fix this ?

You can't define global directives from a http module.  Either use 
NGX_HTTP_MAIN_CONF instead (and the directive will be available in 
http{} section), or define a core module (with module type set 
to NGX_CORE_MODULE).

Maxim Dounin


From l at lrowe.co.uk  Tue Mar 20 16:05:42 2012
From: l at lrowe.co.uk (Laurence Rowe)
Date: Tue, 20 Mar 2012 16:05:42 +0000
Subject: HTML parsing support for xslt module
In-Reply-To: <20120319170327.GX67687@mdounin.ru>
References: <CAOycyLQHaKkehFm_mO3HEJh9H4Y+dii4kuLKWRVAVtw4b4r3tg@mail.gmail.com>
	<20120319170327.GX67687@mdounin.ru>
Message-ID: <CAOycyLTrxE9DBM5ZWY6KPMVEHJeN3ppwXwm8LpugNcPCn5ozGA@mail.gmail.com>

Hi Maxim,

Thanks for your review. I've made a few comments / requests for
clarification below and will pull them together into a new patch set
over the week or so.

On 19 March 2012 17:03, Maxim Dounin <mdounin at mdounin.ru> wrote:
> Hello!
>
> On Wed, Mar 14, 2012 at 05:03:56PM +0000, Laurence Rowe wrote:
>
>> I'd like to submit the attached patch which implements an
>> ``xslt_html_parser`` directive for consideration. When enabled, the
>> xslt module uses the libxml2 HTMLParser to parse the response body.
>> This is useful for people who want to transform HTML using XSLT,
>> including users of Diazo deploying on Nginx
>> (http://docs.diazo.org/en/latest/deployment.html#nginx).
>>
>> The patch is generated from my repository at
>> https://bitbucket.org/lrowe/nginx-xslt-html-parser, forked from
>> http://mdounin.ru/hg/nginx-vendor-current/. The xslt_param patch from
>> http://mailman.nginx.org/pipermail/nginx-devel/2012-March/001926.html
>> is included in my repository. I'll discuss each of the individual
>> changesets briefly below:
>>
>> changeset: ? 668:bf4d14f51436
>> user: ? ? ? ?Laurence Rowe <laurence at lrowe.co.uk>
>> date: ? ? ? ?Sun Jul 11 23:54:08 2010 +0100
>> summary: ? ? Skip transform when there is no content (e.g. a proxied redirect)
>>
>> This was originally reviewed in
>> http://mailman.nginx.org/pipermail/nginx-devel/2010-July/000390.html:
>>
>> > Currently the way to disable XSLT processing is MIME type, "text/xml"
>> > by default. Redirects usually have "text/html" type.
>>
>> To parse HTML you have to set ``xslt_types text/html;``. This
>> changeset prevents crashing on responses with an empty body.
>
> I don't think that skipping transformation based on
> content_length_n is a correct behaviour.
>
> 1) There are more than one way to represent empty response, and
> content_length_n == 0 is just one of them.
>
> 2) And the empty response is just another case of malformed input,
> I see no reason to handle this specially.

Is there a good example of a filter module doing correct error
handling to use here, perhaps ngx_http_sub_filter_module.c?

>> changeset: ? 669:9487b3a0e3ff
>> user: ? ? ? ?Laurence Rowe <laurence at lrowe.co.uk>
>> date: ? ? ? ?Fri Mar 09 21:01:25 2012 +0000
>> summary: ? ? Use xmlCtxtUseOptions to set options.
>>
>> This changeset moves most option setting to the xmlCtxtUseOptions
>> (foundational to next commit.)
>
> This doesn't produce identical ctxt->loadsubset, and hence need
> more details. ?I tend to think the current code is wrong and
> your's is ok, though it would be good to see confirmation.

In the current version of libxml2, loadsubset is a bit field. The
previous value of 1 does not match any of the loadsubset bit field
options (XML_DETECT_IDS=2, XML_COMPLETE_ATTRS=4, XML_SKIP_IDS=8) but
from reading the source, XML_DETECT_IDS is not tested for directly in
any relevant files, with SAX2.c using the test (ctxt->loadsubset !=
0). The loadsubset value is set to XML_DETECT_IDS by the
xmlCtxtUseOptions call with XML_PARSE_DTDLOAD.

I guess it must have been a change sometime in the distant past to
make it a bit field, but it was definitely a bit field as far back as
LIBXML2_2_6_21 from 2005.

> (Additionally, there is a style issue in this patch.)

Is there a style guide for nginx code? I couldn't find an example of
how a long bitfield should be split across lines.

>> changeset: ? 670:c8349ca87381
>> user: ? ? ? ?Laurence Rowe <laurence at lrowe.co.uk>
>> date: ? ? ? ?Fri Mar 09 21:08:18 2012 +0000
>> summary: ? ? XML_PARSE_COMPACT to save memory
>>
>> We can use XML_PARSE_COMPACT as the parsed input document is not
>> modified (XSLT creates a new result document.)
>
> Is it supported in libxml2 versions used by various OSes now? ?It
> appeared relatively recently, in libxml2 2.6.21, and if some OSes
> still use older versions by default - this may need configure
> test.
>
> Another question to consider - is it actually safe to use? ?Note
> that nginx does modify input document to provide DTDs.
>
> It would be also cool to see some stats on real world memory usage
> with and without this option set.

The best benchmark I could find was here:
http://comments.gmane.org/gmane.comp.python.lxml.devel/1507.
Essentially it is helpful for documents with lots of short text nodes.

I hadn't noticed the document modification to provide DTDs. That makes
it unsafe to use and it should be dropped.

>> changeset: ? 671:7145bd8cc1e2
>> tag: ? ? ? ? tip
>> user: ? ? ? ?Laurence Rowe <laurence at lrowe.co.uk>
>> date: ? ? ? ?Fri Mar 09 21:47:46 2012 +0000
>> summary: ? ? xslt_html_parser
>>
>> This changeset adds the ``xslt_html_parser`` directive and uses the
>> HTMLParser when it is set. HTML parsing is performed with
>> HTML_PARSE_RECOVER as real-world HTML may not be well formed, error
>> handling is thus disabled when this option is set.
>
> I in general like the feature, though I tend to think this patch
> needs more polishing. ?In no particular order:
>
> 1) Hardcoded "utf-8" charset detection scares me.

Looks like xmlParseCharEncoding is the way to go here.
http://apache.webthing.com/mod_proxy_html30/mod_proxy_html-2.5.2.c

> 2) The are no error handling for html code, and while it's
> probably ok to ignore parsing problems - it's certainly not ok to
> ignore fatal problems like memory allocation ones.

I'll add back the fatal error handler and add an off switch to it.

> 3) DTD and entities handling needs clarification.

Anything other to the XML_PARSE_COMPACT concern noted above? The DTD
is ignored by the HTMLParser, so I do not set
ctxt->sax->externalSubset when parsing html.

> Maxim Dounin
>
> p.s. You may want to use patchbomb mercurial extension (hg email)
> to submit patches.

I'll experiment with this for the next version of the patch.

Laurence


From abioy.sun at gmail.com  Wed Mar 21 02:53:45 2012
From: abioy.sun at gmail.com (Abioy Sun)
Date: Wed, 21 Mar 2012 10:53:45 +0800
Subject: In memory subrequest hang if backend return 500
Message-ID: <CAM6b-v=N4Bm3AS077GQO0+5uAS+Tyxq9dBiCXrsMFQT8d_aeFQ@mail.gmail.com>

Hi All,

In the file ngx_http_upstream.c I found some code like this:

1618     if (u->headers_in.status_n > NGX_HTTP_SPECIAL_RESPONSE) {
1619
1620         if (r->subrequest_in_memory) {
1621             u->buffer.last = u->buffer.pos;
1622         }

that means, if the status backend return if greater than 300, the first
part of the body would be abandoned.  Will this cause the issue that the
main request won't be woken again and keep sleeping until timeout? Or did I
misunderstand anything?

Yours,
Abioy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120321/248d69af/attachment.html>

From ru at nginx.com  Wed Mar 21 06:19:13 2012
From: ru at nginx.com (ru at nginx.com)
Date: Wed, 21 Mar 2012 06:19:13 +0000
Subject: [nginx] svn commit:  r4548 - trunk/src/http/modules
Message-ID: <20120321061913.77B9B3F9D09@mail.nginx.com>

Author: ru
Date: 2012-03-21 06:19:11 +0000 (Wed, 21 Mar 2012)
New Revision: 4548
URL: http://trac.nginx.org/nginx/changeset/4548/nginx

Log:
Minor ngx_http_headers_filter_module.c code cleanup.

- Removed "hash" element from ngx_http_header_val_t which was always 1.
- Replaced NGX_HTTP_EXPIRES_* with ngx_http_expires_t enum type.
- Added prototype for ngx_http_add_header()
- Simplified ngx_http_set_last_modified().


Modified:
   trunk/src/http/modules/ngx_http_headers_filter_module.c

Modified: trunk/src/http/modules/ngx_http_headers_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_headers_filter_module.c	2012-03-19 14:57:29 UTC (rev 4547)
+++ trunk/src/http/modules/ngx_http_headers_filter_module.c	2012-03-21 06:19:11 UTC (rev 4548)
@@ -25,23 +25,25 @@
 
 struct ngx_http_header_val_s {
     ngx_http_complex_value_t   value;
-    ngx_uint_t                 hash;
     ngx_str_t                  key;
     ngx_http_set_header_pt     handler;
     ngx_uint_t                 offset;
 };
 
 
-#define NGX_HTTP_EXPIRES_OFF       0
-#define NGX_HTTP_EXPIRES_EPOCH     1
-#define NGX_HTTP_EXPIRES_MAX       2
-#define NGX_HTTP_EXPIRES_ACCESS    3
-#define NGX_HTTP_EXPIRES_MODIFIED  4
-#define NGX_HTTP_EXPIRES_DAILY     5
+typedef enum {
+    NGX_HTTP_EXPIRES_OFF,
+    NGX_HTTP_EXPIRES_EPOCH,
+    NGX_HTTP_EXPIRES_MAX,
+    NGX_HTTP_EXPIRES_ACCESS,
+    NGX_HTTP_EXPIRES_MODIFIED,
+    NGX_HTTP_EXPIRES_DAILY,
+    NGX_HTTP_EXPIRES_UNSET
+} ngx_http_expires_t;
 
 
 typedef struct {
-    ngx_uint_t               expires;
+    ngx_http_expires_t       expires;
     time_t                   expires_time;
     ngx_array_t             *headers;
 } ngx_http_headers_conf_t;
@@ -51,6 +53,8 @@
     ngx_http_headers_conf_t *conf);
 static ngx_int_t ngx_http_add_cache_control(ngx_http_request_t *r,
     ngx_http_header_val_t *hv, ngx_str_t *value);
+static ngx_int_t ngx_http_add_header(ngx_http_request_t *r,
+    ngx_http_header_val_t *hv, ngx_str_t *value);
 static ngx_int_t ngx_http_set_last_modified(ngx_http_request_t *r,
     ngx_http_header_val_t *hv, ngx_str_t *value);
 
@@ -313,7 +317,7 @@
             return NGX_ERROR;
         }
 
-        h->hash = hv->hash;
+        h->hash = 1;
         h->key = hv->key;
         h->value = *value;
     }
@@ -366,16 +370,11 @@
 {
     ngx_table_elt_t  *h, **old;
 
-    if (hv->offset) {
-        old = (ngx_table_elt_t **) ((char *) &r->headers_out + hv->offset);
+    old = (ngx_table_elt_t **) ((char *) &r->headers_out + hv->offset);
 
-    } else {
-        old = NULL;
-    }
-
     r->headers_out.last_modified_time = -1;
 
-    if (old == NULL || *old == NULL) {
+    if (*old == NULL) {
 
         if (value->len == 0) {
             return NGX_OK;
@@ -395,7 +394,7 @@
         }
     }
 
-    h->hash = hv->hash;
+    h->hash = 1;
     h->key = hv->key;
     h->value = *value;
 
@@ -420,7 +419,7 @@
      *     conf->expires_time = 0;
      */
 
-    conf->expires = NGX_CONF_UNSET_UINT;
+    conf->expires = NGX_HTTP_EXPIRES_UNSET;
 
     return conf;
 }
@@ -432,11 +431,11 @@
     ngx_http_headers_conf_t *prev = parent;
     ngx_http_headers_conf_t *conf = child;
 
-    if (conf->expires == NGX_CONF_UNSET_UINT) {
+    if (conf->expires == NGX_HTTP_EXPIRES_UNSET) {
         conf->expires = prev->expires;
         conf->expires_time = prev->expires_time;
 
-        if (conf->expires == NGX_CONF_UNSET_UINT) {
+        if (conf->expires == NGX_HTTP_EXPIRES_UNSET) {
             conf->expires = NGX_HTTP_EXPIRES_OFF;
         }
     }
@@ -467,7 +466,7 @@
     ngx_uint_t   minus, n;
     ngx_str_t   *value;
 
-    if (hcf->expires != NGX_CONF_UNSET_UINT) {
+    if (hcf->expires != NGX_HTTP_EXPIRES_UNSET) {
         return "is duplicate";
     }
 
@@ -576,7 +575,6 @@
         return NGX_CONF_ERROR;
     }
 
-    hv->hash = 1;
     hv->key = value[1];
     hv->handler = ngx_http_add_header;
     hv->offset = 0;


From ru at nginx.com  Wed Mar 21 07:35:43 2012
From: ru at nginx.com (ru at nginx.com)
Date: Wed, 21 Mar 2012 07:35:43 +0000
Subject: [nginx] svn commit:  r4549 - trunk/src/http/modules
Message-ID: <20120321073543.C46893F9E1C@mail.nginx.com>

Author: ru
Date: 2012-03-21 07:35:43 +0000 (Wed, 21 Mar 2012)
New Revision: 4549
URL: http://trac.nginx.org/nginx/changeset/4549/nginx

Log:
If we inserted "Last-Modified" in r->headers_out.headers, don't
forget to set the r->headers_out.last_modified pointer to it.


Modified:
   trunk/src/http/modules/ngx_http_headers_filter_module.c

Modified: trunk/src/http/modules/ngx_http_headers_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_headers_filter_module.c	2012-03-21 06:19:11 UTC (rev 4548)
+++ trunk/src/http/modules/ngx_http_headers_filter_module.c	2012-03-21 07:35:43 UTC (rev 4549)
@@ -385,6 +385,8 @@
             return NGX_ERROR;
         }
 
+        *old = h;
+
     } else {
         h = *old;
 


From mdounin at mdounin.ru  Wed Mar 21 08:53:56 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 21 Mar 2012 12:53:56 +0400
Subject: In memory subrequest hang if backend return 500
In-Reply-To: <CAM6b-v=N4Bm3AS077GQO0+5uAS+Tyxq9dBiCXrsMFQT8d_aeFQ@mail.gmail.com>
References: <CAM6b-v=N4Bm3AS077GQO0+5uAS+Tyxq9dBiCXrsMFQT8d_aeFQ@mail.gmail.com>
Message-ID: <20120321085356.GA88788@mdounin.ru>

Hello!

On Wed, Mar 21, 2012 at 10:53:45AM +0800, Abioy Sun wrote:

> Hi All,
> 
> In the file ngx_http_upstream.c I found some code like this:
> 
> 1618     if (u->headers_in.status_n > NGX_HTTP_SPECIAL_RESPONSE) {
> 1619
> 1620         if (r->subrequest_in_memory) {
> 1621             u->buffer.last = u->buffer.pos;
> 1622         }
> 
> that means, if the status backend return if greater than 300, the first
> part of the body would be abandoned.  Will this cause the issue that the
> main request won't be woken again and keep sleeping until timeout? Or did I
> misunderstand anything?

Do you use keepalive connections to backends?

As far as I see, this may indeed cause problems with keepalive 
connections, though should be ok in non-keepalive case.

Maxim Dounin


From zealot33 at gmail.com  Wed Mar 21 09:13:37 2012
From: zealot33 at gmail.com (=?EUC-KR?B?w9a/tbyu?=)
Date: Wed, 21 Mar 2012 18:13:37 +0900
Subject: In memory subrequest hang if backend return 500
In-Reply-To: <20120321085356.GA88788@mdounin.ru>
References: <CAM6b-v=N4Bm3AS077GQO0+5uAS+Tyxq9dBiCXrsMFQT8d_aeFQ@mail.gmail.com>
	<20120321085356.GA88788@mdounin.ru>
Message-ID: <CAK-vuDFG=M--h3XfoA8rvg5+mSCHvP5TFw6AVuovEkf9OzOnZA@mail.gmail.com>

Hi, Maxim.

Can you please explain how this part could make problems for keepalive
connections of backends?

Thank you!

2012/3/21 Maxim Dounin <mdounin at mdounin.ru>

> Hello!
>
> On Wed, Mar 21, 2012 at 10:53:45AM +0800, Abioy Sun wrote:
>
> > Hi All,
> >
> > In the file ngx_http_upstream.c I found some code like this:
> >
> > 1618     if (u->headers_in.status_n > NGX_HTTP_SPECIAL_RESPONSE) {
> > 1619
> > 1620         if (r->subrequest_in_memory) {
> > 1621             u->buffer.last = u->buffer.pos;
> > 1622         }
> >
> > that means, if the status backend return if greater than 300, the first
> > part of the body would be abandoned.  Will this cause the issue that the
> > main request won't be woken again and keep sleeping until timeout? Or
> did I
> > misunderstand anything?
>
> Do you use keepalive connections to backends?
>
> As far as I see, this may indeed cause problems with keepalive
> connections, though should be ok in non-keepalive case.
>
> Maxim Dounin
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120321/f4dff23e/attachment-0001.html>

From mdounin at mdounin.ru  Wed Mar 21 10:45:54 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 21 Mar 2012 14:45:54 +0400
Subject: In memory subrequest hang if backend return 500
In-Reply-To: <CAK-vuDFG=M--h3XfoA8rvg5+mSCHvP5TFw6AVuovEkf9OzOnZA@mail.gmail.com>
References: <CAM6b-v=N4Bm3AS077GQO0+5uAS+Tyxq9dBiCXrsMFQT8d_aeFQ@mail.gmail.com>
	<20120321085356.GA88788@mdounin.ru>
	<CAK-vuDFG=M--h3XfoA8rvg5+mSCHvP5TFw6AVuovEkf9OzOnZA@mail.gmail.com>
Message-ID: <20120321104554.GF88788@mdounin.ru>

Hello!

On Wed, Mar 21, 2012 at 06:13:37PM +0900, ??? wrote:

> Hi, Maxim.
> 
> Can you please explain how this part could make problems for keepalive
> connections of backends?

Consider response with status 500, keepalive used and 
Content-Length set: this code will drop part of the response, and 
it won't be counted by u->length, thus causing nginx to wait for 
data it already got (and discarded).

Maxim Dounin

> 
> Thank you!
> 
> 2012/3/21 Maxim Dounin <mdounin at mdounin.ru>
> 
> > Hello!
> >
> > On Wed, Mar 21, 2012 at 10:53:45AM +0800, Abioy Sun wrote:
> >
> > > Hi All,
> > >
> > > In the file ngx_http_upstream.c I found some code like this:
> > >
> > > 1618     if (u->headers_in.status_n > NGX_HTTP_SPECIAL_RESPONSE) {
> > > 1619
> > > 1620         if (r->subrequest_in_memory) {
> > > 1621             u->buffer.last = u->buffer.pos;
> > > 1622         }
> > >
> > > that means, if the status backend return if greater than 300, the first
> > > part of the body would be abandoned.  Will this cause the issue that the
> > > main request won't be woken again and keep sleeping until timeout? Or
> > did I
> > > misunderstand anything?
> >
> > Do you use keepalive connections to backends?
> >
> > As far as I see, this may indeed cause problems with keepalive
> > connections, though should be ok in non-keepalive case.
> >
> > Maxim Dounin
> >
> > _______________________________________________
> > nginx-devel mailing list
> > nginx-devel at nginx.org
> > http://mailman.nginx.org/mailman/listinfo/nginx-devel
> >

> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel


From mdounin at mdounin.ru  Wed Mar 21 12:55:24 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 21 Mar 2012 16:55:24 +0400
Subject: HTML parsing support for xslt module
In-Reply-To: <CAOycyLTrxE9DBM5ZWY6KPMVEHJeN3ppwXwm8LpugNcPCn5ozGA@mail.gmail.com>
References: <CAOycyLQHaKkehFm_mO3HEJh9H4Y+dii4kuLKWRVAVtw4b4r3tg@mail.gmail.com>
	<20120319170327.GX67687@mdounin.ru>
	<CAOycyLTrxE9DBM5ZWY6KPMVEHJeN3ppwXwm8LpugNcPCn5ozGA@mail.gmail.com>
Message-ID: <20120321125524.GN88788@mdounin.ru>

Hello!

On Tue, Mar 20, 2012 at 04:05:42PM +0000, Laurence Rowe wrote:

> Hi Maxim,
> 
> Thanks for your review. I've made a few comments / requests for
> clarification below and will pull them together into a new patch set
> over the week or so.
> 
> On 19 March 2012 17:03, Maxim Dounin <mdounin at mdounin.ru> wrote:
> > Hello!
> >
> > On Wed, Mar 14, 2012 at 05:03:56PM +0000, Laurence Rowe wrote:
> >
> >> I'd like to submit the attached patch which implements an
> >> ``xslt_html_parser`` directive for consideration. When enabled, the
> >> xslt module uses the libxml2 HTMLParser to parse the response body.
> >> This is useful for people who want to transform HTML using XSLT,
> >> including users of Diazo deploying on Nginx
> >> (http://docs.diazo.org/en/latest/deployment.html#nginx).
> >>
> >> The patch is generated from my repository at
> >> https://bitbucket.org/lrowe/nginx-xslt-html-parser, forked from
> >> http://mdounin.ru/hg/nginx-vendor-current/. The xslt_param patch from
> >> http://mailman.nginx.org/pipermail/nginx-devel/2012-March/001926.html
> >> is included in my repository. I'll discuss each of the individual
> >> changesets briefly below:
> >>
> >> changeset: ? 668:bf4d14f51436
> >> user: ? ? ? ?Laurence Rowe <laurence at lrowe.co.uk>
> >> date: ? ? ? ?Sun Jul 11 23:54:08 2010 +0100
> >> summary: ? ? Skip transform when there is no content (e.g. a proxied redirect)
> >>
> >> This was originally reviewed in
> >> http://mailman.nginx.org/pipermail/nginx-devel/2010-July/000390.html:
> >>
> >> > Currently the way to disable XSLT processing is MIME type, "text/xml"
> >> > by default. Redirects usually have "text/html" type.
> >>
> >> To parse HTML you have to set ``xslt_types text/html;``. This
> >> changeset prevents crashing on responses with an empty body.
> >
> > I don't think that skipping transformation based on
> > content_length_n is a correct behaviour.
> >
> > 1) There are more than one way to represent empty response, and
> > content_length_n == 0 is just one of them.
> >
> > 2) And the empty response is just another case of malformed input,
> > I see no reason to handle this specially.
> 
> Is there a good example of a filter module doing correct error
> handling to use here, perhaps ngx_http_sub_filter_module.c?

The ngx_http_image_filter_module.c is closest one to xslt (both 
transform full response and may return error instead).  Though I 
think error handling in xslt as of now is good enough.

> >> changeset: ? 669:9487b3a0e3ff
> >> user: ? ? ? ?Laurence Rowe <laurence at lrowe.co.uk>
> >> date: ? ? ? ?Fri Mar 09 21:01:25 2012 +0000
> >> summary: ? ? Use xmlCtxtUseOptions to set options.
> >>
> >> This changeset moves most option setting to the xmlCtxtUseOptions
> >> (foundational to next commit.)
> >
> > This doesn't produce identical ctxt->loadsubset, and hence need
> > more details. ?I tend to think the current code is wrong and
> > your's is ok, though it would be good to see confirmation.
> 
> In the current version of libxml2, loadsubset is a bit field. The
> previous value of 1 does not match any of the loadsubset bit field
> options (XML_DETECT_IDS=2, XML_COMPLETE_ATTRS=4, XML_SKIP_IDS=8) but
> from reading the source, XML_DETECT_IDS is not tested for directly in
> any relevant files, with SAX2.c using the test (ctxt->loadsubset !=
> 0). The loadsubset value is set to XML_DETECT_IDS by the
> xmlCtxtUseOptions call with XML_PARSE_DTDLOAD.
> 
> I guess it must have been a change sometime in the distant past to
> make it a bit field, but it was definitely a bit field as far back as
> LIBXML2_2_6_21 from 2005.

Ok, my investigations match.

> > (Additionally, there is a style issue in this patch.)
> 
> Is there a style guide for nginx code?

Not really.  We probably have to write one, at least with some basics.

> I couldn't find an example of
> how a long bitfield should be split across lines.

See e.g. in ngx_http_core_module.c:

    { ngx_string("limit_rate"),
      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF
                        |NGX_CONF_TAKE1,
      ngx_conf_set_size_slot,
      NGX_HTTP_LOC_CONF_OFFSET,
      offsetof(ngx_http_core_loc_conf_t, limit_rate),
      NULL },

In extreme cases short form is used, like this (from 
ngx_http_upstream.c):

    uscf = ngx_http_upstream_add(cf, &u, NGX_HTTP_UPSTREAM_CREATE
                                         |NGX_HTTP_UPSTREAM_WEIGHT
                                         |NGX_HTTP_UPSTREAM_MAX_FAILS
                                         |NGX_HTTP_UPSTREAM_FAIL_TIMEOUT
                                         |NGX_HTTP_UPSTREAM_DOWN
                                         |NGX_HTTP_UPSTREAM_BACKUP);

Though usually it's better to avoid the split.

> >> changeset: ? 670:c8349ca87381
> >> user: ? ? ? ?Laurence Rowe <laurence at lrowe.co.uk>
> >> date: ? ? ? ?Fri Mar 09 21:08:18 2012 +0000
> >> summary: ? ? XML_PARSE_COMPACT to save memory
> >>
> >> We can use XML_PARSE_COMPACT as the parsed input document is not
> >> modified (XSLT creates a new result document.)
> >
> > Is it supported in libxml2 versions used by various OSes now? ?It
> > appeared relatively recently, in libxml2 2.6.21, and if some OSes
> > still use older versions by default - this may need configure
> > test.
> >
> > Another question to consider - is it actually safe to use? ?Note
> > that nginx does modify input document to provide DTDs.
> >
> > It would be also cool to see some stats on real world memory usage
> > with and without this option set.
> 
> The best benchmark I could find was here:
> http://comments.gmane.org/gmane.comp.python.lxml.devel/1507.
> Essentially it is helpful for documents with lots of short text nodes.

It looks like performance benchmark, and I doubt the speedup will 
be compareable with nginx given the fact it uses nginx pool 
allocator which is expected to be faster than normal malloc().

> I hadn't noticed the document modification to provide DTDs. That makes
> it unsafe to use and it should be dropped.

Ok.

> >> changeset: ? 671:7145bd8cc1e2
> >> tag: ? ? ? ? tip
> >> user: ? ? ? ?Laurence Rowe <laurence at lrowe.co.uk>
> >> date: ? ? ? ?Fri Mar 09 21:47:46 2012 +0000
> >> summary: ? ? xslt_html_parser
> >>
> >> This changeset adds the ``xslt_html_parser`` directive and uses the
> >> HTMLParser when it is set. HTML parsing is performed with
> >> HTML_PARSE_RECOVER as real-world HTML may not be well formed, error
> >> handling is thus disabled when this option is set.
> >
> > I in general like the feature, though I tend to think this patch
> > needs more polishing. ?In no particular order:
> >
> > 1) Hardcoded "utf-8" charset detection scares me.
> 
> Looks like xmlParseCharEncoding is the way to go here.
> http://apache.webthing.com/mod_proxy_html30/mod_proxy_html-2.5.2.c

Yes, probably.

> > 2) The are no error handling for html code, and while it's
> > probably ok to ignore parsing problems - it's certainly not ok to
> > ignore fatal problems like memory allocation ones.
> 
> I'll add back the fatal error handler and add an off switch to it.

Not really understand.  Why do you think it should be possible to 
switch off handling of memory allocation errors?

> > 3) DTD and entities handling needs clarification.
> 
> Anything other to the XML_PARSE_COMPACT concern noted above? The DTD
> is ignored by the HTMLParser, so I do not set
> ctxt->sax->externalSubset when parsing html.

My concerns were mostly about "what will happen if DTD referenced in 
a document".  It's ok as long as it's not used anyway.

Maxim Dounin


From abioy.sun at gmail.com  Wed Mar 21 13:03:52 2012
From: abioy.sun at gmail.com (Abioy Sun)
Date: Wed, 21 Mar 2012 21:03:52 +0800
Subject: In memory subrequest hang if backend return 500
In-Reply-To: <20120321085356.GA88788@mdounin.ru>
References: <CAM6b-v=N4Bm3AS077GQO0+5uAS+Tyxq9dBiCXrsMFQT8d_aeFQ@mail.gmail.com>
	<20120321085356.GA88788@mdounin.ru>
Message-ID: <CAM6b-vnH9wUdqZo29JfK3KVcNt7WstpveBjbBo2nFu=NaYSLzg@mail.gmail.com>

Hi,

2012/3/21 Maxim Dounin <mdounin at mdounin.ru>
>
> Do you use keepalive connections to backends?
>
Yes

> As far as I see, this may indeed cause problems with keepalive
> connections, though should be ok in non-keepalive case.
that might be the key problem,  thx.

>
> Maxim Dounin
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel


From agentzh at gmail.com  Wed Mar 21 13:42:02 2012
From: agentzh at gmail.com (agentzh)
Date: Wed, 21 Mar 2012 21:42:02 +0800
Subject: [PATCH] ngx_http_upstream state machine fixes
Message-ID: <CAB4Tn6MzNVGXFW8s3ybtOgdGjaHrDvSQtrJFxijGcMz8QdCw=A@mail.gmail.com>

Hello!

I've just found that ngx_http_upstream seems to assume that the
upstream request is already fully sent (i.e., fully flushed into the
socket send buffers) after the first read event on the upstream
connection, and in this case, ngx_http_upstream_send_request_handler
will just silently refuse to re-send the requests on subsequent write
events even if the request has not been fully sent out yet, hence
leading to a hang.

This was observed when doing pipelined requests atop ngx_http_upstream
where the remote server can start sending back response data *before*
all the request data is flushed out (the latter can be trivially
emulated by our mockeagain tool: https://github.com/agentzh/mockeagain
).

I've attached a patch that (hopefully) fixes this issue by introducing
an extra 1-bit "request_all_sent" flag to ngx_http_upstream_t, which
can be set to 1 after ngx_output_chain returns NGX_OK in
ngx_http_upstream_send_request. I know there is already a
"request_sent" flag which indicates whether there is the first attempt
of sending out the request) :)

Feedback will be highly appreciated :)

Thanks!
-agentzh

diff -ur nginx-1.0.11/src/http/ngx_http_upstream.c
nginx-1.0.11-patched/src/http/ngx_http_upstream.c
--- nginx-1.0.11/src/http/ngx_http_upstream.c	2011-12-14
02:34:34.000000000 +0800
+++ nginx-1.0.11-patched/src/http/ngx_http_upstream.c	2012-03-21
21:20:17.333111806 +0800
@@ -1385,6 +1385,8 @@

     /* rc == NGX_OK */

+    u->request_all_sent = 1;
+
     if (c->tcp_nopush == NGX_TCP_NOPUSH_SET) {
         if (ngx_tcp_push(c->fd) == NGX_ERROR) {
             ngx_log_error(NGX_LOG_CRIT, c->log, ngx_socket_errno,
@@ -1451,7 +1453,7 @@

 #endif

-    if (u->header_sent) {
+    if (u->request_all_sent) {
         u->write_event_handler = ngx_http_upstream_dummy_handler;

         (void) ngx_handle_write_event(c->write, 0);
Only in nginx-1.0.11-patched/src/http: ngx_http_upstream.c~
diff -ur nginx-1.0.11/src/http/ngx_http_upstream.h
nginx-1.0.11-patched/src/http/ngx_http_upstream.h
--- nginx-1.0.11/src/http/ngx_http_upstream.h	2011-11-01
22:18:10.000000000 +0800
+++ nginx-1.0.11-patched/src/http/ngx_http_upstream.h	2012-03-21
21:18:21.041237173 +0800
@@ -313,6 +313,7 @@
     unsigned                         buffering:1;

     unsigned                         request_sent:1;
+    unsigned                         request_all_sent:1;
     unsigned                         header_sent:1;
 };
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nginx-1.0.11-upstream_pipelining.patch
Type: application/octet-stream
Size: 1310 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120321/90a92047/attachment.obj>

From l at lrowe.co.uk  Wed Mar 21 13:55:09 2012
From: l at lrowe.co.uk (Laurence Rowe)
Date: Wed, 21 Mar 2012 13:55:09 +0000
Subject: HTML parsing support for xslt module
In-Reply-To: <20120321125524.GN88788@mdounin.ru>
References: <CAOycyLQHaKkehFm_mO3HEJh9H4Y+dii4kuLKWRVAVtw4b4r3tg@mail.gmail.com>
	<20120319170327.GX67687@mdounin.ru>
	<CAOycyLTrxE9DBM5ZWY6KPMVEHJeN3ppwXwm8LpugNcPCn5ozGA@mail.gmail.com>
	<20120321125524.GN88788@mdounin.ru>
Message-ID: <CAOycyLSudqtjiVroj7T9hEHGa0ahcnt8pQq9mT3aUuTMDwNUDQ@mail.gmail.com>

Snipping for brevity:

On 21 March 2012 12:55, Maxim Dounin <mdounin at mdounin.ru> wrote:
>> > 2) The are no error handling for html code, and while it's
>> > probably ok to ignore parsing problems - it's certainly not ok to
>> > ignore fatal problems like memory allocation ones.
>>
>> I'll add back the fatal error handler and add an off switch to it.
>
> Not really understand. ?Why do you think it should be possible to
> switch off handling of memory allocation errors?

I wasn't clear here. Currently errors are detected with the return
value from xmlParseChunk. With htmlParseChunk we cannot use the return
value as we want to ignore any recoverable parsing errors, so we need
another way to detect when a fatal error has occurred. I'll add a flag
(or 'off switch') to ngx_http_xslt_filter_ctx_t, set it in
ngx_http_xslt_sax_error when an error and test that instead.

Laurence


From ru at nginx.com  Wed Mar 21 13:58:52 2012
From: ru at nginx.com (ru at nginx.com)
Date: Wed, 21 Mar 2012 13:58:52 +0000
Subject: [nginx] svn commit: r4550 - in trunk: auto auto/os src/core
	src/os/unix
Message-ID: <20120321135852.714D43F9D0A@mail.nginx.com>

Author: ru
Date: 2012-03-21 13:58:51 +0000 (Wed, 21 Mar 2012)
New Revision: 4550
URL: http://trac.nginx.org/nginx/changeset/4550/nginx

Log:
worker_cpu_affinity: cleaned up Linux implementation, added FreeBSD support.


Added:
   trunk/src/os/unix/ngx_setaffinity.c
   trunk/src/os/unix/ngx_setaffinity.h
Modified:
   trunk/auto/os/freebsd
   trunk/auto/os/linux
   trunk/auto/sources
   trunk/src/core/nginx.c
   trunk/src/core/ngx_cycle.h
   trunk/src/os/unix/ngx_process.h
   trunk/src/os/unix/ngx_process_cycle.c

Modified: trunk/auto/os/freebsd
===================================================================
--- trunk/auto/os/freebsd	2012-03-21 07:35:43 UTC (rev 4549)
+++ trunk/auto/os/freebsd	2012-03-21 13:58:51 UTC (rev 4550)
@@ -134,3 +134,11 @@
         exit 1
     fi
 fi
+
+
+# cpuset_setaffinity()
+
+if [ $version -ge 701000 ]; then
+    echo " + cpuset_setaffinity() found"
+    have=NGX_HAVE_CPUSET_SETAFFINITY . auto/have
+fi

Modified: trunk/auto/os/linux
===================================================================
--- trunk/auto/os/linux	2012-03-21 07:35:43 UTC (rev 4549)
+++ trunk/auto/os/linux	2012-03-21 13:58:51 UTC (rev 4550)
@@ -128,8 +128,9 @@
 ngx_feature_incs="#include <sched.h>"
 ngx_feature_path=
 ngx_feature_libs=
-ngx_feature_test="long mask = 0;
-                  sched_setaffinity(0, 32, (cpu_set_t *) &mask)"
+ngx_feature_test="cpu_set_t mask;
+                  CPU_ZERO(&mask);
+                  sched_setaffinity(0, sizeof(cpu_set_t), &mask)"
 . auto/feature
 
 

Modified: trunk/auto/sources
===================================================================
--- trunk/auto/sources	2012-03-21 07:35:43 UTC (rev 4549)
+++ trunk/auto/sources	2012-03-21 13:58:51 UTC (rev 4550)
@@ -145,6 +145,7 @@
             src/os/unix/ngx_channel.h \
             src/os/unix/ngx_shmem.h \
             src/os/unix/ngx_process.h \
+            src/os/unix/ngx_setaffinity.h \
             src/os/unix/ngx_setproctitle.h \
             src/os/unix/ngx_atomic.h \
             src/os/unix/ngx_gcc_atomic_x86.h \
@@ -179,6 +180,7 @@
             src/os/unix/ngx_shmem.c \
             src/os/unix/ngx_process.c \
             src/os/unix/ngx_daemon.c \
+            src/os/unix/ngx_setaffinity.c \
             src/os/unix/ngx_setproctitle.c \
             src/os/unix/ngx_posix_init.c \
             src/os/unix/ngx_user.c \

Modified: trunk/src/core/nginx.c
===================================================================
--- trunk/src/core/nginx.c	2012-03-21 07:35:43 UTC (rev 4549)
+++ trunk/src/core/nginx.c	2012-03-21 13:58:51 UTC (rev 4550)
@@ -983,15 +983,15 @@
     ngx_conf_init_value(ccf->worker_processes, 1);
     ngx_conf_init_value(ccf->debug_points, 0);
 
-#if (NGX_HAVE_SCHED_SETAFFINITY)
+#if (NGX_HAVE_CPU_AFFINITY)
 
     if (ccf->cpu_affinity_n
         && ccf->cpu_affinity_n != 1
         && ccf->cpu_affinity_n != (ngx_uint_t) ccf->worker_processes)
     {
         ngx_log_error(NGX_LOG_WARN, cycle->log, 0,
-                      "number of the \"worker_processes\" is not equal to "
-                      "the number of the \"worker_cpu_affinity\" mask, "
+                      "the number of \"worker_processes\" is not equal to "
+                      "the number of \"worker_cpu_affinity\" masks, "
                       "using last mask for remaining worker processes");
     }
 
@@ -1242,11 +1242,11 @@
 static char *
 ngx_set_cpu_affinity(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
 {
-#if (NGX_HAVE_SCHED_SETAFFINITY)
+#if (NGX_HAVE_CPU_AFFINITY)
     ngx_core_conf_t  *ccf = conf;
 
     u_char            ch;
-    u_long           *mask;
+    uint64_t         *mask;
     ngx_str_t        *value;
     ngx_uint_t        i, n;
 
@@ -1254,7 +1254,7 @@
         return "is duplicate";
     }
 
-    mask = ngx_palloc(cf->pool, (cf->args->nelts - 1) * sizeof(long));
+    mask = ngx_palloc(cf->pool, (cf->args->nelts - 1) * sizeof(uint64_t));
     if (mask == NULL) {
         return NGX_CONF_ERROR;
     }
@@ -1266,9 +1266,9 @@
 
     for (n = 1; n < cf->args->nelts; n++) {
 
-        if (value[n].len > 32) {
+        if (value[n].len > 64) {
             ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
-                         "\"worker_cpu_affinity\" supports up to 32 CPU only");
+                         "\"worker_cpu_affinity\" supports up to 64 CPUs only");
             return NGX_CONF_ERROR;
         }
 
@@ -1311,7 +1311,7 @@
 }
 
 
-u_long
+uint64_t
 ngx_get_cpu_affinity(ngx_uint_t n)
 {
     ngx_core_conf_t  *ccf;

Modified: trunk/src/core/ngx_cycle.h
===================================================================
--- trunk/src/core/ngx_cycle.h	2012-03-21 07:35:43 UTC (rev 4549)
+++ trunk/src/core/ngx_cycle.h	2012-03-21 13:58:51 UTC (rev 4550)
@@ -86,7 +86,7 @@
      int                      priority;
 
      ngx_uint_t               cpu_affinity_n;
-     u_long                  *cpu_affinity;
+     uint64_t                *cpu_affinity;
 
      char                    *username;
      ngx_uid_t                user;
@@ -124,7 +124,7 @@
 void ngx_reopen_files(ngx_cycle_t *cycle, ngx_uid_t user);
 char **ngx_set_environment(ngx_cycle_t *cycle, ngx_uint_t *last);
 ngx_pid_t ngx_exec_new_binary(ngx_cycle_t *cycle, char *const *argv);
-u_long ngx_get_cpu_affinity(ngx_uint_t n);
+uint64_t ngx_get_cpu_affinity(ngx_uint_t n);
 ngx_shm_zone_t *ngx_shared_memory_add(ngx_conf_t *cf, ngx_str_t *name,
     size_t size, void *tag);
 

Modified: trunk/src/os/unix/ngx_process.h
===================================================================
--- trunk/src/os/unix/ngx_process.h	2012-03-21 07:35:43 UTC (rev 4549)
+++ trunk/src/os/unix/ngx_process.h	2012-03-21 13:58:51 UTC (rev 4550)
@@ -9,6 +9,7 @@
 #define _NGX_PROCESS_H_INCLUDED_
 
 
+#include <ngx_setaffinity.h>
 #include <ngx_setproctitle.h>
 
 

Modified: trunk/src/os/unix/ngx_process_cycle.c
===================================================================
--- trunk/src/os/unix/ngx_process_cycle.c	2012-03-21 07:35:43 UTC (rev 4549)
+++ trunk/src/os/unix/ngx_process_cycle.c	2012-03-21 13:58:51 UTC (rev 4550)
@@ -62,7 +62,7 @@
 #endif
 
 
-u_long         cpu_affinity;
+uint64_t       cpu_affinity;
 static u_char  master_process[] = "master process";
 
 
@@ -913,23 +913,10 @@
         }
     }
 
-#if (NGX_HAVE_SCHED_SETAFFINITY)
-
     if (cpu_affinity) {
-        ngx_log_error(NGX_LOG_NOTICE, cycle->log, 0,
-                      "sched_setaffinity(0x%08Xl)", cpu_affinity);
-
-        if (sched_setaffinity(0, sizeof(cpu_affinity),
-                              (cpu_set_t *) &cpu_affinity)
-            == -1)
-        {
-            ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno,
-                          "sched_setaffinity(0x%08Xl) failed", cpu_affinity);
-        }
+        ngx_setaffinity(cpu_affinity, cycle->log);
     }
 
-#endif
-
 #if (NGX_HAVE_PR_SET_DUMPABLE)
 
     /* allow coredump after setuid() in Linux 2.4.x */

Added: trunk/src/os/unix/ngx_setaffinity.c
===================================================================
--- trunk/src/os/unix/ngx_setaffinity.c	                        (rev 0)
+++ trunk/src/os/unix/ngx_setaffinity.c	2012-03-21 13:58:51 UTC (rev 4550)
@@ -0,0 +1,69 @@
+
+/*
+ * Copyright (C) Nginx, Inc.
+ */
+
+
+#include <ngx_config.h>
+#include <ngx_core.h>
+
+
+#if (NGX_HAVE_CPUSET_SETAFFINITY)
+
+#include <sys/cpuset.h>
+
+void
+ngx_setaffinity(uint64_t cpu_affinity, ngx_log_t *log)
+{
+    cpuset_t    mask;
+    ngx_uint_t  i;
+
+    ngx_log_error(NGX_LOG_NOTICE, log, 0,
+                  "cpuset_setaffinity(0x%08Xl)", cpu_affinity);
+
+    CPU_ZERO(&mask);
+    i = 0;
+    do {
+        if (cpu_affinity & 1) {
+            CPU_SET(i, &mask);
+        }
+        i++;
+        cpu_affinity >>= 1;
+    } while (cpu_affinity);
+
+    if (cpuset_setaffinity(CPU_LEVEL_WHICH, CPU_WHICH_PID, -1,
+                           sizeof(cpuset_t), &mask) == -1)
+    {
+        ngx_log_error(NGX_LOG_ALERT, log, ngx_errno,
+                      "cpuset_setaffinity() failed");
+    }
+}
+
+#elif (NGX_HAVE_SCHED_SETAFFINITY)
+
+void
+ngx_setaffinity(uint64_t cpu_affinity, ngx_log_t *log)
+{
+    cpu_set_t   mask;
+    ngx_uint_t  i;
+
+    ngx_log_error(NGX_LOG_NOTICE, log, 0,
+                  "sched_setaffinity(0x%08Xl)", cpu_affinity);
+
+    CPU_ZERO(&mask);
+    i = 0;
+    do {
+        if (cpu_affinity & 1) {
+            CPU_SET(i, &mask);
+        }
+        i++;
+        cpu_affinity >>= 1;
+    } while (cpu_affinity);
+
+    if (sched_setaffinity(0, sizeof(cpu_set_t), &mask) == -1) {
+        ngx_log_error(NGX_LOG_ALERT, log, ngx_errno,
+                      "sched_setaffinity() failed");
+    }
+}
+
+#endif

Added: trunk/src/os/unix/ngx_setaffinity.h
===================================================================
--- trunk/src/os/unix/ngx_setaffinity.h	                        (rev 0)
+++ trunk/src/os/unix/ngx_setaffinity.h	2012-03-21 13:58:51 UTC (rev 4550)
@@ -0,0 +1,23 @@
+
+/*
+ * Copyright (C) Nginx, Inc.
+ */
+
+#ifndef _NGX_SETAFFINITY_H_INCLUDED_
+#define _NGX_SETAFFINITY_H_INCLUDED_
+
+
+#if (NGX_HAVE_SCHED_SETAFFINITY || NGX_HAVE_CPUSET_SETAFFINITY)
+
+#define NGX_HAVE_CPU_AFFINITY 1
+
+void ngx_setaffinity(uint64_t cpu_affinity, ngx_log_t *log);
+
+#else
+
+#define ngx_setaffinity(cpu_affinity, log)
+
+#endif
+
+
+#endif /* _NGX_SETAFFINITY_H_INCLUDED_ */


From mdounin at mdounin.ru  Wed Mar 21 14:31:43 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 21 Mar 2012 18:31:43 +0400
Subject: HTML parsing support for xslt module
In-Reply-To: <CAOycyLSudqtjiVroj7T9hEHGa0ahcnt8pQq9mT3aUuTMDwNUDQ@mail.gmail.com>
References: <CAOycyLQHaKkehFm_mO3HEJh9H4Y+dii4kuLKWRVAVtw4b4r3tg@mail.gmail.com>
	<20120319170327.GX67687@mdounin.ru>
	<CAOycyLTrxE9DBM5ZWY6KPMVEHJeN3ppwXwm8LpugNcPCn5ozGA@mail.gmail.com>
	<20120321125524.GN88788@mdounin.ru>
	<CAOycyLSudqtjiVroj7T9hEHGa0ahcnt8pQq9mT3aUuTMDwNUDQ@mail.gmail.com>
Message-ID: <20120321143143.GP88788@mdounin.ru>

Hello!

On Wed, Mar 21, 2012 at 01:55:09PM +0000, Laurence Rowe wrote:

> Snipping for brevity:
> 
> On 21 March 2012 12:55, Maxim Dounin <mdounin at mdounin.ru> wrote:
> >> > 2) The are no error handling for html code, and while it's
> >> > probably ok to ignore parsing problems - it's certainly not ok to
> >> > ignore fatal problems like memory allocation ones.
> >>
> >> I'll add back the fatal error handler and add an off switch to it.
> >
> > Not really understand. ?Why do you think it should be possible to
> > switch off handling of memory allocation errors?
> 
> I wasn't clear here. Currently errors are detected with the return
> value from xmlParseChunk. With htmlParseChunk we cannot use the return
> value as we want to ignore any recoverable parsing errors, so we need
> another way to detect when a fatal error has occurred. I'll add a flag
> (or 'off switch') to ngx_http_xslt_filter_ctx_t, set it in
> ngx_http_xslt_sax_error when an error and test that instead.

Ok, this looks fine.  Thanks for clarification.

Maxim Dounin


From ru at nginx.com  Wed Mar 21 15:35:06 2012
From: ru at nginx.com (ru at nginx.com)
Date: Wed, 21 Mar 2012 15:35:06 +0000
Subject: [nginx] svn commit:  r4551 - trunk/auto
Message-ID: <20120321153506.3B2E43FA63D@mail.nginx.com>

Author: ru
Date: 2012-03-21 15:35:05 +0000 (Wed, 21 Mar 2012)
New Revision: 4551
URL: http://trac.nginx.org/nginx/changeset/4551/nginx

Log:
The addition of $tcpinfo_* variables has broken the build on Linux
systems with glibc versions prior to 2.7.  Fixed this by checking
the existence of "struct tcp_info" members during configuration.


Modified:
   trunk/auto/unix

Modified: trunk/auto/unix
===================================================================
--- trunk/auto/unix	2012-03-21 13:58:51 UTC (rev 4550)
+++ trunk/auto/unix	2012-03-21 15:35:05 UTC (rev 4551)
@@ -352,6 +352,11 @@
 ngx_feature_path=
 ngx_feature_libs=
 ngx_feature_test="socklen_t optlen = sizeof(struct tcp_info);
+                  struct tcp_info ti;
+                  ti.tcpi_rtt = 0;
+                  ti.tcpi_rttvar = 0;
+                  ti.tcpi_snd_cwnd = 0;
+                  ti.tcpi_rcv_space = 0;
                   getsockopt(0, IPPROTO_TCP, TCP_INFO, NULL, &optlen)"
 . auto/feature
 


From mdounin at mdounin.ru  Wed Mar 21 15:38:51 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 21 Mar 2012 19:38:51 +0400
Subject: [PATCH] ngx_http_upstream state machine fixes
In-Reply-To: <CAB4Tn6MzNVGXFW8s3ybtOgdGjaHrDvSQtrJFxijGcMz8QdCw=A@mail.gmail.com>
References: <CAB4Tn6MzNVGXFW8s3ybtOgdGjaHrDvSQtrJFxijGcMz8QdCw=A@mail.gmail.com>
Message-ID: <20120321153851.GS88788@mdounin.ru>

Hello!

On Wed, Mar 21, 2012 at 09:42:02PM +0800, agentzh wrote:

> Hello!
> 
> I've just found that ngx_http_upstream seems to assume that the
> upstream request is already fully sent (i.e., fully flushed into the
> socket send buffers) after the first read event on the upstream
> connection, and in this case, ngx_http_upstream_send_request_handler
> will just silently refuse to re-send the requests on subsequent write
> events even if the request has not been fully sent out yet, hence
> leading to a hang.
> 
> This was observed when doing pipelined requests atop ngx_http_upstream
> where the remote server can start sending back response data *before*
> all the request data is flushed out (the latter can be trivially
> emulated by our mockeagain tool: https://github.com/agentzh/mockeagain
> ).
> 
> I've attached a patch that (hopefully) fixes this issue by introducing
> an extra 1-bit "request_all_sent" flag to ngx_http_upstream_t, which
> can be set to 1 after ngx_output_chain returns NGX_OK in
> ngx_http_upstream_send_request. I know there is already a
> "request_sent" flag which indicates whether there is the first attempt
> of sending out the request) :)
> 
> Feedback will be highly appreciated :)

I've looked into this a while ago (the same problem appears e.g. 
if fastcgi app returns response header before reading body, and 
then tries to read body).

The major question seems to be "what should be done if we've not 
yet sent full request, but already got response"?  Right now line 
is drawn at "if we've got response header, we stop sending 
request".  And if we move the line - we should clearly understand 
where the new line is and why.

As for the patch from coding point of view - I belive adding 
another flag isn't really needed, it should be enough to reset 
r->write_event_handler just after we've done writing.

Maxim Dounin


From l at lrowe.co.uk  Wed Mar 21 21:28:30 2012
From: l at lrowe.co.uk (Laurence Rowe)
Date: Wed, 21 Mar 2012 21:28:30 +0000
Subject: [PATCH 0 of 4] XSLT HTML parsing
Message-ID: <patchbomb.1332365310@Laurence-Rowes-Mac-mini.local>

Hi,

These patches add HTML parsing support to the XSLT filter module. They have
been updated to incorporate earlier feedback from the list and irc and
correspond with the current head of:
https://bitbucket.org/lrowe/nginx-xslt-html-parser

They apply on top of the xslt_param changes from:
http://mailman.nginx.org/pipermail/nginx-devel/2012-March/001926.html

Note that the current ordering of filter modules means that the charset is not
normally set on headers_out before the XSLT filter runs. As you can control
the output charset encoding from XSL with "<xsl:output encoding=.../>" it
probably makes sense to move the xslt filter to execute after the charset
filter. However, this is left outside the scope of this patch set.

Laurence


From l at lrowe.co.uk  Wed Mar 21 21:28:31 2012
From: l at lrowe.co.uk (Laurence Rowe)
Date: Wed, 21 Mar 2012 21:28:31 +0000
Subject: [PATCH 1 of 4] Set parser options with xmlCtxtUseOptions
In-Reply-To: <patchbomb.1332365310@Laurence-Rowes-Mac-mini.local>
References: <patchbomb.1332365310@Laurence-Rowes-Mac-mini.local>
Message-ID: <28a1437a8b4492530a24.1332365311@Laurence-Rowes-Mac-mini.local>

# HG changeset patch
# User Laurence Rowe <laurence at lrowe.co.uk>
# Date 1331326885 0
# Node ID 28a1437a8b4492530a24d51b85aca7923627c330
# Parent  aed89f1bb357e289086f9a768e2e9958c4076b7c
Set parser options with xmlCtxtUseOptions

In the current version of libxml2, loadsubset is a bit field. The
previous value of 1 does not match any of the loadsubset bit field
options (XML_DETECT_IDS=2, XML_COMPLETE_ATTRS=4, XML_SKIP_IDS=8) but
from reading the source, XML_DETECT_IDS is not tested for directly in
any relevant files, with SAX2.c using the test (ctxt->loadsubset !=
0). The loadsubset value is set to XML_DETECT_IDS by the
xmlCtxtUseOptions call with XML_PARSE_DTDLOAD.

diff --git a/src/http/modules/ngx_http_xslt_filter_module.c b/src/http/modules/ngx_http_xslt_filter_module.c
--- a/src/http/modules/ngx_http_xslt_filter_module.c
+++ b/src/http/modules/ngx_http_xslt_filter_module.c
@@ -362,15 +362,14 @@
                           "xmlCreatePushParserCtxt() failed");
             return NGX_ERROR;
         }
+        xmlCtxtUseOptions(ctxt, XML_PARSE_NOENT|XML_PARSE_DTDLOAD
+                                               |XML_PARSE_NOWARNING);
 
         ctxt->sax->externalSubset = ngx_http_xslt_sax_external_subset;
         ctxt->sax->setDocumentLocator = NULL;
-        ctxt->sax->warning = NULL;
         ctxt->sax->error = ngx_http_xslt_sax_error;
         ctxt->sax->fatalError = ngx_http_xslt_sax_error;
         ctxt->sax->_private = ctx;
-        ctxt->replaceEntities = 1;
-        ctxt->loadsubset = 1;
 
         ctx->ctxt = ctxt;
         ctx->request = r;


From l at lrowe.co.uk  Wed Mar 21 21:28:32 2012
From: l at lrowe.co.uk (Laurence Rowe)
Date: Wed, 21 Mar 2012 21:28:32 +0000
Subject: [PATCH 2 of 4] Set done flag on module context to stop further chunk
	parsing
In-Reply-To: <patchbomb.1332365310@Laurence-Rowes-Mac-mini.local>
References: <patchbomb.1332365310@Laurence-Rowes-Mac-mini.local>
Message-ID: <151124d060d3f725c02b.1332365312@Laurence-Rowes-Mac-mini.local>

# HG changeset patch
# User Laurence Rowe <laurence at lrowe.co.uk>
# Date 1332363776 0
# Node ID 151124d060d3f725c02b656d39c10575ff009cdb
# Parent  28a1437a8b4492530a24d51b85aca7923627c330
Set done flag on module context to stop further chunk parsing.

As HTML parsing will recover from non-fatal errors, the return value of
(x|ht)mlParseChunk cannot be used.

diff --git a/src/http/modules/ngx_http_xslt_filter_module.c b/src/http/modules/ngx_http_xslt_filter_module.c
--- a/src/http/modules/ngx_http_xslt_filter_module.c
+++ b/src/http/modules/ngx_http_xslt_filter_module.c
@@ -378,7 +378,7 @@
     err = xmlParseChunk(ctx->ctxt, (char *) b->pos, (int) (b->last - b->pos),
                         (b->last_buf) || (b->last_in_chain));
 
-    if (err == 0) {
+    if (ctx->done == 0) {
         b->pos = b->last;
         return NGX_OK;
     }
@@ -463,6 +463,8 @@
 
     ngx_log_error(NGX_LOG_ERR, ctx->request->connection->log, 0,
                   "libxml2 error: \"%*s\"", n + 1, buf);
+
+    ctx->done = 1; /* stop further chunk parsing */
 }
 
 


From l at lrowe.co.uk  Wed Mar 21 21:28:34 2012
From: l at lrowe.co.uk (Laurence Rowe)
Date: Wed, 21 Mar 2012 21:28:34 +0000
Subject: [PATCH 4 of 4] Handle empty response body
In-Reply-To: <patchbomb.1332365310@Laurence-Rowes-Mac-mini.local>
References: <patchbomb.1332365310@Laurence-Rowes-Mac-mini.local>
Message-ID: <1a21700079c90be39eaf.1332365314@Laurence-Rowes-Mac-mini.local>

# HG changeset patch
# User Laurence Rowe <laurence at lrowe.co.uk>
# Date 1332363819 0
# Node ID 1a21700079c90be39eaf5f6c5d7aef65be43ff95
# Parent  65fd4892a78371e863d43e31d4430cdb7333a35d
Handle empty response body

Responses, and especially proxied responses, may have empty bodies. These
should pass through the filter without causing an internal server error or
crashing nginx.

Example config:

    location = /testme {
        default_type text/xml;
        return 204;
        xslt_stylesheet example.xsl;
    }

diff --git a/src/http/modules/ngx_http_xslt_filter_module.c b/src/http/modules/ngx_http_xslt_filter_module.c
--- a/src/http/modules/ngx_http_xslt_filter_module.c
+++ b/src/http/modules/ngx_http_xslt_filter_module.c
@@ -284,10 +284,17 @@
 
         if (cl->buf->last_buf || cl->buf->last_in_chain) {
 
+            if (ctx->ctxt == NULL) {
+                /* empty body */
+                return ngx_http_next_header_filter(r);
+            }
+
             ctx->doc = ctx->ctxt->myDoc;
 
 #if (NGX_HTTP_XSLT_REUSE_DTD)
-            ctx->doc->extSubset = NULL;
+            if (ctx->doc) {
+                ctx->doc->extSubset = NULL;
+            }
 #endif
 
             wellFormed = ctx->ctxt->wellFormed;
@@ -377,6 +384,10 @@
 
     if (ctx->ctxt == NULL) {
 
+        if (b->last == b->pos) {
+            return NGX_OK;
+        }
+
         if (ctx->html_parser) {
             if (r->headers_out.charset.len) {
                 enc = xmlParseCharEncoding(


From l at lrowe.co.uk  Wed Mar 21 21:28:33 2012
From: l at lrowe.co.uk (Laurence Rowe)
Date: Wed, 21 Mar 2012 21:28:33 +0000
Subject: [PATCH 3 of 4] xslt_html_parser directive
In-Reply-To: <patchbomb.1332365310@Laurence-Rowes-Mac-mini.local>
References: <patchbomb.1332365310@Laurence-Rowes-Mac-mini.local>
Message-ID: <65fd4892a78371e863d4.1332365313@Laurence-Rowes-Mac-mini.local>

# HG changeset patch
# User Laurence Rowe <laurence at lrowe.co.uk>
# Date 1331329666 0
# Node ID 65fd4892a78371e863d43e31d4430cdb7333a35d
# Parent  151124d060d3f725c02b656d39c10575ff009cdb
xslt_html_parser directive

When ```xslt_html_parser on;`` the HTMLParser is used. Parsing is performed
with HTML_PARSE_RECOVER as real-world HTML may not be well formed, so only
fatal error handling is enabled when this option is set.

diff --git a/src/http/modules/ngx_http_xslt_filter_module.c b/src/http/modules/ngx_http_xslt_filter_module.c
--- a/src/http/modules/ngx_http_xslt_filter_module.c
+++ b/src/http/modules/ngx_http_xslt_filter_module.c
@@ -10,6 +10,7 @@
 #include <ngx_http.h>
 
 #include <libxml/parser.h>
+#include <libxml/HTMLparser.h>
 #include <libxml/tree.h>
 #include <libxslt/xslt.h>
 #include <libxslt/xsltInternals.h>
@@ -58,6 +59,7 @@
     ngx_hash_t                 types;
     ngx_array_t               *types_keys;
     ngx_array_t               *params;       /* ngx_http_xslt_param_t */
+    ngx_flag_t                 html_parser;
 } ngx_http_xslt_filter_loc_conf_t;
 
 
@@ -67,6 +69,7 @@
     xsltTransformContextPtr    transform;
     ngx_http_request_t        *request;
     ngx_array_t                params;
+    ngx_flag_t                 html_parser;
 
     ngx_uint_t                 done;         /* unsigned  done:1; */
 } ngx_http_xslt_filter_ctx_t;
@@ -150,6 +153,13 @@
       offsetof(ngx_http_xslt_filter_loc_conf_t, types_keys),
       &ngx_http_xslt_default_types[0] },
 
+    { ngx_string("xslt_html_parser"),
+      NGX_HTTP_LOC_CONF|NGX_CONF_FLAG,
+      ngx_conf_set_flag_slot,
+      NGX_HTTP_LOC_CONF_OFFSET,
+      offsetof(ngx_http_xslt_filter_loc_conf_t, html_parser),
+      NULL },
+
       ngx_null_command
 };
 
@@ -225,6 +235,8 @@
 
     r->main_filter_need_in_memory = 1;
 
+    ctx->html_parser = conf->html_parser;
+
     return NGX_OK;
 }
 
@@ -261,7 +273,11 @@
                 xmlFreeDoc(ctx->ctxt->myDoc);
             }
 
-            xmlFreeParserCtxt(ctx->ctxt);
+            if (ctx->html_parser) {
+                htmlFreeParserCtxt(ctx->ctxt);
+            } else {
+                xmlFreeParserCtxt(ctx->ctxt);
+            }
 
             return ngx_http_xslt_send(r, ctx, NULL);
         }
@@ -276,9 +292,13 @@
 
             wellFormed = ctx->ctxt->wellFormed;
 
-            xmlFreeParserCtxt(ctx->ctxt);
+            if (ctx->html_parser) {
+                htmlFreeParserCtxt(ctx->ctxt);
+            } else {
+                xmlFreeParserCtxt(ctx->ctxt);
+            }
 
-            if (wellFormed) {
+            if (wellFormed || ctx->html_parser) {
                 return ngx_http_xslt_send(r, ctx,
                                        ngx_http_xslt_apply_stylesheet(r, ctx));
             }
@@ -352,22 +372,48 @@
     ngx_buf_t *b)
 {
     int               err;
-    xmlParserCtxtPtr  ctxt;
+    xmlParserCtxtPtr  ctxt = NULL;
+    xmlCharEncoding   enc = XML_CHAR_ENCODING_NONE;
 
     if (ctx->ctxt == NULL) {
 
-        ctxt = xmlCreatePushParserCtxt(NULL, NULL, NULL, 0, NULL);
-        if (ctxt == NULL) {
-            ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
-                          "xmlCreatePushParserCtxt() failed");
-            return NGX_ERROR;
+        if (ctx->html_parser) {
+            if (r->headers_out.charset.len) {
+                enc = xmlParseCharEncoding(
+                                   (const char *) r->headers_out.charset.data);
+                if (enc == XML_CHAR_ENCODING_ERROR) {
+                    ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                                  "xmlParseCharEncoding() failed charset: %s",
+                                  r->headers_out.charset.data);
+                    return NGX_ERROR;
+                }
+            }
+
+            ctxt = htmlCreatePushParserCtxt(NULL, NULL, NULL, 0, NULL, enc);
+            if (ctxt == NULL) {
+                ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                              "htmlCreatePushParserCtxt() failed");
+                return NGX_ERROR;
+            }
+
+            htmlCtxtUseOptions(ctxt, HTML_PARSE_RECOVER|HTML_PARSE_NOERROR
+                                                       |HTML_PARSE_NOWARNING);
+
+        } else {
+            ctxt = xmlCreatePushParserCtxt(NULL, NULL, NULL, 0, NULL);
+            if (ctxt == NULL) {
+                ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                              "xmlCreatePushParserCtxt() failed");
+                return NGX_ERROR;
+            }
+
+            xmlCtxtUseOptions(ctxt, XML_PARSE_NOENT|XML_PARSE_DTDLOAD
+                                                   |XML_PARSE_NOWARNING);
+            ctxt->sax->externalSubset = ngx_http_xslt_sax_external_subset;
+            ctxt->sax->error = ngx_http_xslt_sax_error;
         }
-        xmlCtxtUseOptions(ctxt, XML_PARSE_NOENT|XML_PARSE_DTDLOAD
-                                               |XML_PARSE_NOWARNING);
 
-        ctxt->sax->externalSubset = ngx_http_xslt_sax_external_subset;
         ctxt->sax->setDocumentLocator = NULL;
-        ctxt->sax->error = ngx_http_xslt_sax_error;
         ctxt->sax->fatalError = ngx_http_xslt_sax_error;
         ctxt->sax->_private = ctx;
 
@@ -375,8 +421,16 @@
         ctx->request = r;
     }
 
-    err = xmlParseChunk(ctx->ctxt, (char *) b->pos, (int) (b->last - b->pos),
-                        (b->last_buf) || (b->last_in_chain));
+    if (ctx->html_parser) {
+        err = htmlParseChunk(ctx->ctxt, (char *) b->pos,
+                             (int) (b->last - b->pos),
+                             (b->last_buf) || (b->last_in_chain));
+
+    } else {
+        err = xmlParseChunk(ctx->ctxt, (char *) b->pos,
+                            (int) (b->last - b->pos),
+                            (b->last_buf) || (b->last_in_chain));
+    }
 
     if (ctx->done == 0) {
         b->pos = b->last;
@@ -1059,6 +1113,8 @@
      *     conf->params = NULL;
      */
 
+    conf->html_parser = NGX_CONF_UNSET;
+
     return conf;
 }
 
@@ -1081,6 +1137,8 @@
         conf->params = prev->params;
     }
 
+    ngx_conf_merge_value(conf->html_parser, prev->html_parser, 0);
+
     if (ngx_http_merge_types(cf, &conf->types_keys, &conf->types,
                              &prev->types_keys, &prev->types,
                              ngx_http_xslt_default_types)


From mdounin at mdounin.ru  Thu Mar 22 10:41:29 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 22 Mar 2012 10:41:29 +0000
Subject: [nginx] svn commit:  r4552 - trunk/src/http
Message-ID: <20120322104130.153653F9D85@mail.nginx.com>

Author: mdounin
Date: 2012-03-22 10:41:29 +0000 (Thu, 22 Mar 2012)
New Revision: 4552
URL: http://trac.nginx.org/nginx/changeset/4552/nginx

Log:
Removed safari from keepalive_disable default.

The bug in question is likely already fixed (though unfortunately we have
no information available as Apple's bugtracker isn't open), and the
workaround seems to be too pessimistic for modern versions of Safari
as well as other webkit-based browsers pretending to be Safari.


Modified:
   trunk/src/http/ngx_http_core_module.c

Modified: trunk/src/http/ngx_http_core_module.c
===================================================================
--- trunk/src/http/ngx_http_core_module.c	2012-03-21 15:35:05 UTC (rev 4551)
+++ trunk/src/http/ngx_http_core_module.c	2012-03-22 10:41:29 UTC (rev 4552)
@@ -3568,8 +3568,7 @@
     ngx_conf_merge_bitmask_value(conf->keepalive_disable,
                               prev->keepalive_disable,
                               (NGX_CONF_BITMASK_SET
-                               |NGX_HTTP_KEEPALIVE_DISABLE_MSIE6
-                               |NGX_HTTP_KEEPALIVE_DISABLE_SAFARI));
+                               |NGX_HTTP_KEEPALIVE_DISABLE_MSIE6));
     ngx_conf_merge_uint_value(conf->satisfy, prev->satisfy,
                               NGX_HTTP_SATISFY_ALL);
     ngx_conf_merge_uint_value(conf->if_modified_since, prev->if_modified_since,


From mdounin at mdounin.ru  Thu Mar 22 10:42:27 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 22 Mar 2012 10:42:27 +0000
Subject: [nginx] svn commit:  r4553 - trunk/src/http
Message-ID: <20120322104227.CAC743F9D85@mail.nginx.com>

Author: mdounin
Date: 2012-03-22 10:42:27 +0000 (Thu, 22 Mar 2012)
New Revision: 4553
URL: http://trac.nginx.org/nginx/changeset/4553/nginx

Log:
Restricted keepalive_disable safari to OS X only.

The problem doesn't affect non-Apple systems for sure, and many pretend
to be Safari now.

Prodded by Piotr Sikora.


Modified:
   trunk/src/http/ngx_http_request.c

Modified: trunk/src/http/ngx_http_request.c
===================================================================
--- trunk/src/http/ngx_http_request.c	2012-03-22 10:41:29 UTC (rev 4552)
+++ trunk/src/http/ngx_http_request.c	2012-03-22 10:42:27 UTC (rev 4553)
@@ -1493,7 +1493,9 @@
         } else if (ngx_strstrn(user_agent, "Chrome/", 7 - 1)) {
             r->headers_in.chrome = 1;
 
-        } else if (ngx_strstrn(user_agent, "Safari/", 7 - 1)) {
+        } else if (ngx_strstrn(user_agent, "Safari/", 7 - 1)
+                   && ngx_strstrn(user_agent, "Mac OS X", 8 - 1))
+        {
             r->headers_in.safari = 1;
 
         } else if (ngx_strstrn(user_agent, "Konqueror", 9 - 1)) {


From mdounin at mdounin.ru  Thu Mar 22 10:43:34 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 22 Mar 2012 10:43:34 +0000
Subject: [nginx] svn commit:  r4554 - trunk/src/http/modules
Message-ID: <20120322104334.186523F9D85@mail.nginx.com>

Author: mdounin
Date: 2012-03-22 10:43:33 +0000 (Thu, 22 Mar 2012)
New Revision: 4554
URL: http://trac.nginx.org/nginx/changeset/4554/nginx

Log:
Fixed off-by-one in xslt parameter parsing.

The problem was introduced in 0.7.44 (r2589) during conversion to complex
values.  Previously string.len included space for terminating NUL, but
with complex values it doesn't.


Modified:
   trunk/src/http/modules/ngx_http_xslt_filter_module.c

Modified: trunk/src/http/modules/ngx_http_xslt_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_xslt_filter_module.c	2012-03-22 10:42:27 UTC (rev 4553)
+++ trunk/src/http/modules/ngx_http_xslt_filter_module.c	2012-03-22 10:43:33 UTC (rev 4554)
@@ -585,7 +585,7 @@
                        "xslt filter param: \"%s\"", string.data);
 
         p = string.data;
-        last = string.data + string.len - 1;
+        last = string.data + string.len;
 
         while (p && *p) {
 


From mdounin at mdounin.ru  Thu Mar 22 10:44:01 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 22 Mar 2012 10:44:01 +0000
Subject: [nginx] svn commit:  r4555 - trunk/src/http/modules
Message-ID: <20120322104401.2CA0F3F9D85@mail.nginx.com>

Author: mdounin
Date: 2012-03-22 10:44:00 +0000 (Thu, 22 Mar 2012)
New Revision: 4555
URL: http://trac.nginx.org/nginx/changeset/4555/nginx

Log:
Added xslt_param and xslt_string_param directives.

Based on patch by Samuel Behan.


Modified:
   trunk/src/http/modules/ngx_http_xslt_filter_module.c

Modified: trunk/src/http/modules/ngx_http_xslt_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_xslt_filter_module.c	2012-03-22 10:43:33 UTC (rev 4554)
+++ trunk/src/http/modules/ngx_http_xslt_filter_module.c	2012-03-22 10:44:00 UTC (rev 4555)
@@ -14,6 +14,7 @@
 #include <libxslt/xslt.h>
 #include <libxslt/xsltInternals.h>
 #include <libxslt/transform.h>
+#include <libxslt/variables.h>
 #include <libxslt/xsltutils.h>
 
 #if (NGX_HAVE_EXSLT)
@@ -27,38 +28,47 @@
 
 
 typedef struct {
-    u_char              *name;
-    void                *data;
+    u_char                    *name;
+    void                      *data;
 } ngx_http_xslt_file_t;
 
 
 typedef struct {
-    ngx_array_t          dtd_files;    /* ngx_http_xslt_file_t */
-    ngx_array_t          sheet_files;  /* ngx_http_xslt_file_t */
+    ngx_array_t                dtd_files;    /* ngx_http_xslt_file_t */
+    ngx_array_t                sheet_files;  /* ngx_http_xslt_file_t */
 } ngx_http_xslt_filter_main_conf_t;
 
 
 typedef struct {
-    xsltStylesheetPtr    stylesheet;
-    ngx_array_t          params;       /* ngx_http_complex_value_t */
+    u_char                    *name;
+    ngx_http_complex_value_t   value;
+    ngx_uint_t                 quote;        /* unsigned  quote:1; */
+} ngx_http_xslt_param_t;
+
+
+typedef struct {
+    xsltStylesheetPtr          stylesheet;
+    ngx_array_t                params;       /* ngx_http_xslt_param_t */
 } ngx_http_xslt_sheet_t;
 
 
 typedef struct {
-    xmlDtdPtr            dtd;
-    ngx_array_t          sheets;       /* ngx_http_xslt_sheet_t */
-    ngx_hash_t           types;
-    ngx_array_t         *types_keys;
+    xmlDtdPtr                  dtd;
+    ngx_array_t                sheets;       /* ngx_http_xslt_sheet_t */
+    ngx_hash_t                 types;
+    ngx_array_t               *types_keys;
+    ngx_array_t               *params;       /* ngx_http_xslt_param_t */
 } ngx_http_xslt_filter_loc_conf_t;
 
 
 typedef struct {
-    xmlDocPtr            doc;
-    xmlParserCtxtPtr     ctxt;
-    ngx_http_request_t  *request;
-    ngx_array_t          params;
+    xmlDocPtr                  doc;
+    xmlParserCtxtPtr           ctxt;
+    xsltTransformContextPtr    transform;
+    ngx_http_request_t        *request;
+    ngx_array_t                params;
 
-    ngx_uint_t           done;         /* unsigned  done:1; */
+    ngx_uint_t                 done;         /* unsigned  done:1; */
 } ngx_http_xslt_filter_ctx_t;
 
 
@@ -76,7 +86,7 @@
 static ngx_buf_t *ngx_http_xslt_apply_stylesheet(ngx_http_request_t *r,
     ngx_http_xslt_filter_ctx_t *ctx);
 static ngx_int_t ngx_http_xslt_params(ngx_http_request_t *r,
-    ngx_http_xslt_filter_ctx_t *ctx, ngx_array_t *params);
+    ngx_http_xslt_filter_ctx_t *ctx, ngx_array_t *params, ngx_uint_t final);
 static u_char *ngx_http_xslt_content_type(xsltStylesheetPtr s);
 static u_char *ngx_http_xslt_encoding(xsltStylesheetPtr s);
 static void ngx_http_xslt_cleanup(void *data);
@@ -85,6 +95,8 @@
     void *conf);
 static char *ngx_http_xslt_stylesheet(ngx_conf_t *cf, ngx_command_t *cmd,
     void *conf);
+static char *ngx_http_xslt_param(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf);
 static void ngx_http_xslt_cleanup_dtd(void *data);
 static void ngx_http_xslt_cleanup_stylesheet(void *data);
 static void *ngx_http_xslt_filter_create_main_conf(ngx_conf_t *cf);
@@ -117,6 +129,20 @@
       0,
       NULL },
 
+    { ngx_string("xslt_param"),
+      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
+      ngx_http_xslt_param,
+      NGX_HTTP_LOC_CONF_OFFSET,
+      0,
+      NULL },
+
+    { ngx_string("xslt_string_param"),
+      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
+      ngx_http_xslt_param,
+      NGX_HTTP_LOC_CONF_OFFSET,
+      0,
+      (void *) 1 },
+
     { ngx_string("xslt_types"),
       NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_1MORE,
       ngx_http_types_slot,
@@ -469,13 +495,31 @@
 
     for (i = 0; i < conf->sheets.nelts; i++) {
 
-        if (ngx_http_xslt_params(r, ctx, &sheet[i].params) != NGX_OK) {
+        ctx->transform = xsltNewTransformContext(sheet[i].stylesheet, doc);
+        if (ctx->transform == NULL) {
             xmlFreeDoc(doc);
             return NULL;
         }
 
-        res = xsltApplyStylesheet(sheet[i].stylesheet, doc, ctx->params.elts);
+        if (conf->params
+            && ngx_http_xslt_params(r, ctx, conf->params, 0) != NGX_OK)
+        {
+            xsltFreeTransformContext(ctx->transform);
+            xmlFreeDoc(doc);
+            return NULL;
+        }
 
+        if (ngx_http_xslt_params(r, ctx, &sheet[i].params, 1) != NGX_OK) {
+            xsltFreeTransformContext(ctx->transform);
+            xmlFreeDoc(doc);
+            return NULL;
+        }
+
+        res = xsltApplyStylesheetUser(sheet[i].stylesheet, doc,
+                                      ctx->params.elts, NULL, NULL,
+                                      ctx->transform);
+
+        xsltFreeTransformContext(ctx->transform);
         xmlFreeDoc(doc);
 
         if (res == NULL) {
@@ -565,25 +609,66 @@
 
 static ngx_int_t
 ngx_http_xslt_params(ngx_http_request_t *r, ngx_http_xslt_filter_ctx_t *ctx,
-    ngx_array_t *params)
+    ngx_array_t *params, ngx_uint_t final)
 {
-    u_char                    *p, *last, *value, *dst, *src, **s;
-    size_t                     len;
-    ngx_uint_t                 i;
-    ngx_str_t                  string;
-    ngx_http_complex_value_t  *param;
+    u_char                 *p, *last, *value, *dst, *src, **s;
+    size_t                  len;
+    ngx_uint_t              i;
+    ngx_str_t               string;
+    ngx_http_xslt_param_t  *param;
 
     param = params->elts;
 
     for (i = 0; i < params->nelts; i++) {
 
-        if (ngx_http_complex_value(r, &param[i], &string) != NGX_OK) {
+        if (ngx_http_complex_value(r, &param[i].value, &string) != NGX_OK) {
             return NGX_ERROR;
         }
 
         ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                        "xslt filter param: \"%s\"", string.data);
 
+        if (param[i].name) {
+
+            ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
+                           "xslt filter param name: \"%s\"", param[i].name);
+
+            if (param[i].quote) {
+                if (xsltQuoteOneUserParam(ctx->transform, param[i].name,
+                                          string.data)
+                    != 0)
+                {
+                    ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                                "xsltQuoteOneUserParam(\"%s\", \"%s\") failed",
+                                param[i].name, string.data);
+                    return NGX_ERROR;
+                }
+
+                continue;
+            }
+
+            s = ngx_array_push(&ctx->params);
+            if (s == NULL) {
+                return NGX_ERROR;
+            }
+
+            *s = param[i].name;
+
+            s = ngx_array_push(&ctx->params);
+            if (s == NULL) {
+                return NGX_ERROR;
+            }
+
+            *s = string.data;
+
+            continue;
+        }
+
+        /*
+         * parse param1=value1:param2=value2 syntax as used by parameters
+         * specified in xslt_stylesheet directives
+         */
+
         p = string.data;
         last = string.data + string.len;
 
@@ -641,13 +726,15 @@
         }
     }
 
-    s = ngx_array_push(&ctx->params);
-    if (s == NULL) {
-        return NGX_ERROR;
+    if (final) {
+        s = ngx_array_push(&ctx->params);
+        if (s == NULL) {
+            return NGX_ERROR;
+        }
+
+        *s = NULL;
     }
 
-    *s = NULL;
-
     return NGX_OK;
 }
 
@@ -768,7 +855,7 @@
     ngx_pool_cleanup_t                *cln;
     ngx_http_xslt_file_t              *file;
     ngx_http_xslt_sheet_t             *sheet;
-    ngx_http_complex_value_t          *param;
+    ngx_http_xslt_param_t             *param;
     ngx_http_compile_complex_value_t   ccv;
     ngx_http_xslt_filter_main_conf_t  *xmcf;
 
@@ -837,7 +924,7 @@
     }
 
     if (ngx_array_init(&sheet->params, cf->pool, n - 2,
-                       sizeof(ngx_http_complex_value_t))
+                       sizeof(ngx_http_xslt_param_t))
         != NGX_OK)
     {
         return NGX_CONF_ERROR;
@@ -850,11 +937,12 @@
             return NGX_CONF_ERROR;
         }
 
+        ngx_memzero(param, sizeof(ngx_http_xslt_param_t));
         ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
 
         ccv.cf = cf;
         ccv.value = &value[i];
-        ccv.complex_value = param;
+        ccv.complex_value = &param->value;
         ccv.zero = 1;
 
         if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
@@ -866,6 +954,48 @@
 }
 
 
+static char *
+ngx_http_xslt_param(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
+{
+    ngx_http_xslt_filter_loc_conf_t  *xlcf = conf;
+
+    ngx_http_xslt_param_t            *param;
+    ngx_http_compile_complex_value_t  ccv;
+    ngx_str_t                        *value;
+
+    value = cf->args->elts;
+
+    if (xlcf->params == NULL) {
+        xlcf->params = ngx_array_create(cf->pool, 2,
+                                        sizeof(ngx_http_xslt_param_t));
+        if (xlcf->params == NULL) {
+            return NGX_CONF_ERROR;
+        }
+    }
+
+    param = ngx_array_push(xlcf->params);
+    if (param == NULL) {
+        return NGX_CONF_ERROR;
+    }
+
+    param->name = value[1].data;
+    param->quote = (cmd->post == NULL) ? 0 : 1;
+
+    ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
+
+    ccv.cf = cf;
+    ccv.value = &value[2];
+    ccv.complex_value = &param->value;
+    ccv.zero = 1;
+
+    if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
+        return NGX_CONF_ERROR;
+    }
+
+    return NGX_CONF_OK;
+}
+
+
 static void
 ngx_http_xslt_cleanup_dtd(void *data)
 {
@@ -925,6 +1055,7 @@
      *     conf->sheets = { NULL };
      *     conf->types = { NULL };
      *     conf->types_keys = NULL;
+     *     conf->params = NULL;
      */
 
     return conf;
@@ -945,6 +1076,10 @@
         conf->sheets = prev->sheets;
     }
 
+    if (conf->params == NULL) {
+        conf->params = prev->params;
+    }
+
     if (ngx_http_merge_types(cf, &conf->types_keys, &conf->types,
                              &prev->types_keys, &prev->types,
                              ngx_http_xslt_default_types)


From mdounin at mdounin.ru  Thu Mar 22 10:45:08 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 22 Mar 2012 10:45:08 +0000
Subject: [nginx] svn commit:  r4556 - trunk/src/os/win32
Message-ID: <20120322104508.B0C3C3F9D85@mail.nginx.com>

Author: mdounin
Date: 2012-03-22 10:45:08 +0000 (Thu, 22 Mar 2012)
New Revision: 4556
URL: http://trac.nginx.org/nginx/changeset/4556/nginx

Log:
Win32: added missing call to srand().

Found by Veracode.


Modified:
   trunk/src/os/win32/ngx_win32_init.c

Modified: trunk/src/os/win32/ngx_win32_init.c
===================================================================
--- trunk/src/os/win32/ngx_win32_init.c	2012-03-22 10:44:00 UTC (rev 4555)
+++ trunk/src/os/win32/ngx_win32_init.c	2012-03-22 10:45:08 UTC (rev 4556)
@@ -228,6 +228,8 @@
         ngx_sprintf((u_char *) ngx_unique, "%P%Z", ngx_pid);
     }
 
+    srand((unsigned) ngx_time());
+
     return NGX_OK;
 }
 


From mdounin at mdounin.ru  Thu Mar 22 11:57:19 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Thu, 22 Mar 2012 11:57:19 +0000
Subject: [nginx] svn commit:  r4557 - trunk/src/core
Message-ID: <20120322115719.31AD63FA3F0@mail.nginx.com>

Author: mdounin
Date: 2012-03-22 11:57:18 +0000 (Thu, 22 Mar 2012)
New Revision: 4557
URL: http://trac.nginx.org/nginx/changeset/4557/nginx

Log:
Resolver: added missing sanity checking when creating name queries.

Found by Veracode.


Modified:
   trunk/src/core/ngx_resolver.c

Modified: trunk/src/core/ngx_resolver.c
===================================================================
--- trunk/src/core/ngx_resolver.c	2012-03-22 10:45:08 UTC (rev 4556)
+++ trunk/src/core/ngx_resolver.c	2012-03-22 11:57:18 UTC (rev 4557)
@@ -1840,7 +1840,7 @@
             len++;
 
         } else {
-            if (len == 0) {
+            if (len == 0 || len > 255) {
                 return NGX_DECLINED;
             }
 
@@ -1851,6 +1851,10 @@
         p--;
     }
 
+    if (len == 0 || len > 255) {
+        return NGX_DECLINED;
+    }
+
     *p = (u_char) len;
 
     return NGX_OK;


From mdounin at mdounin.ru  Tue Mar 27 16:37:43 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Tue, 27 Mar 2012 16:37:43 +0000
Subject: [nginx] svn commit:  r4558 - trunk/src/os/unix
Message-ID: <20120327163746.395C33FA4C8@mail.nginx.com>

Author: mdounin
Date: 2012-03-27 16:37:43 +0000 (Tue, 27 Mar 2012)
New Revision: 4558
URL: http://trac.nginx.org/nginx/changeset/4558/nginx

Log:
Added explicit include of time.h.

Most of the systems have it included due to namespace pollution, but
relying on this is a bad idea.  Explicit include is required for at least
Debian GNU/Hurd.


Modified:
   trunk/src/os/unix/ngx_freebsd_config.h
   trunk/src/os/unix/ngx_posix_config.h
   trunk/src/os/unix/ngx_solaris_config.h

Modified: trunk/src/os/unix/ngx_freebsd_config.h
===================================================================
--- trunk/src/os/unix/ngx_freebsd_config.h	2012-03-22 11:57:18 UTC (rev 4557)
+++ trunk/src/os/unix/ngx_freebsd_config.h	2012-03-27 16:37:43 UTC (rev 4558)
@@ -23,6 +23,7 @@
 #include <grp.h>
 #include <dirent.h>
 #include <glob.h>
+#include <time.h>
 #include <sys/param.h>          /* ALIGN() */
 #include <sys/mount.h>          /* statfs() */
 

Modified: trunk/src/os/unix/ngx_posix_config.h
===================================================================
--- trunk/src/os/unix/ngx_posix_config.h	2012-03-22 11:57:18 UTC (rev 4557)
+++ trunk/src/os/unix/ngx_posix_config.h	2012-03-27 16:37:43 UTC (rev 4558)
@@ -45,6 +45,7 @@
 #include <grp.h>
 #include <dirent.h>
 #include <glob.h>
+#include <time.h>
 #if (NGX_HAVE_SYS_PARAM_H)
 #include <sys/param.h>          /* statfs() */
 #endif

Modified: trunk/src/os/unix/ngx_solaris_config.h
===================================================================
--- trunk/src/os/unix/ngx_solaris_config.h	2012-03-22 11:57:18 UTC (rev 4557)
+++ trunk/src/os/unix/ngx_solaris_config.h	2012-03-27 16:37:43 UTC (rev 4558)
@@ -29,6 +29,7 @@
 #include <grp.h>
 #include <dirent.h>
 #include <glob.h>
+#include <time.h>
 #include <sys/statvfs.h>        /* statvfs() */
 
 #include <sys/filio.h>          /* FIONBIO */


From mdounin at mdounin.ru  Tue Mar 27 16:42:34 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Tue, 27 Mar 2012 16:42:34 +0000
Subject: [nginx] svn commit:  r4559 - in trunk/src: http os/unix os/win32
Message-ID: <20120327164234.BBF8F3FA4BD@mail.nginx.com>

Author: mdounin
Date: 2012-03-27 16:42:34 +0000 (Tue, 27 Mar 2012)
New Revision: 4559
URL: http://trac.nginx.org/nginx/changeset/4559/nginx

Log:
Fixed unconditional MAX_PATH usage (ticket #22).

POSIX doesn't require it to be defined, and Debian GNU/Hurd doesn't define
it.  Note that if there is no MAX_PATH defined we have to use realpath()
with NULL argument and free() the result.


Modified:
   trunk/src/http/ngx_http_variables.c
   trunk/src/os/unix/ngx_files.h
   trunk/src/os/win32/ngx_files.h

Modified: trunk/src/http/ngx_http_variables.c
===================================================================
--- trunk/src/http/ngx_http_variables.c	2012-03-27 16:37:43 UTC (rev 4558)
+++ trunk/src/http/ngx_http_variables.c	2012-03-27 16:42:34 UTC (rev 4559)
@@ -1273,10 +1273,13 @@
 ngx_http_variable_realpath_root(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data)
 {
+    u_char                    *real;
     size_t                     len;
     ngx_str_t                  path;
     ngx_http_core_loc_conf_t  *clcf;
-    u_char                     real[NGX_MAX_PATH];
+#if (NGX_HAVE_MAX_PATH)
+    u_char                     buffer[NGX_MAX_PATH];
+#endif
 
     clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module);
 
@@ -1298,7 +1301,15 @@
         }
     }
 
-    if (ngx_realpath(path.data, real) == NULL) {
+#if (NGX_HAVE_MAX_PATH)
+    real = buffer;
+#else
+    real = NULL;
+#endif
+
+    real = ngx_realpath(path.data, real);
+
+    if (real == NULL) {
         ngx_log_error(NGX_LOG_CRIT, r->connection->log, ngx_errno,
                       ngx_realpath_n " \"%s\" failed", path.data);
         return NGX_ERROR;
@@ -1308,6 +1319,9 @@
 
     v->data = ngx_pnalloc(r->pool, len);
     if (v->data == NULL) {
+#if !(NGX_HAVE_MAX_PATH)
+        ngx_free(real);
+#endif
         return NGX_ERROR;
     }
 
@@ -1318,6 +1332,10 @@
 
     ngx_memcpy(v->data, real, len);
 
+#if !(NGX_HAVE_MAX_PATH)
+    ngx_free(real);
+#endif
+
     return NGX_OK;
 }
 

Modified: trunk/src/os/unix/ngx_files.h
===================================================================
--- trunk/src/os/unix/ngx_files.h	2012-03-27 16:37:43 UTC (rev 4558)
+++ trunk/src/os/unix/ngx_files.h	2012-03-27 16:42:34 UTC (rev 4559)
@@ -200,14 +200,25 @@
 #endif
 
 
-#define ngx_realpath(p, r)       realpath((char *) p, (char *) r)
+#define ngx_realpath(p, r)       (u_char *) realpath((char *) p, (char *) r)
 #define ngx_realpath_n           "realpath()"
 #define ngx_getcwd(buf, size)    (getcwd((char *) buf, size) != NULL)
 #define ngx_getcwd_n             "getcwd()"
 #define ngx_path_separator(c)    ((c) == '/')
 
+
+#if defined(PATH_MAX)
+
+#define NGX_HAVE_MAX_PATH        1
 #define NGX_MAX_PATH             PATH_MAX
 
+#else
+
+#define NGX_MAX_PATH             4096
+
+#endif
+
+
 #define NGX_DIR_MASK_LEN         0
 
 

Modified: trunk/src/os/win32/ngx_files.h
===================================================================
--- trunk/src/os/win32/ngx_files.h	2012-03-27 16:37:43 UTC (rev 4558)
+++ trunk/src/os/win32/ngx_files.h	2012-03-27 16:42:34 UTC (rev 4559)
@@ -183,6 +183,7 @@
 #define ngx_getcwd_n                "GetCurrentDirectory()"
 #define ngx_path_separator(c)       ((c) == '/' || (c) == '\\')
 
+#define NGX_HAVE_MAX_PATH           1
 #define NGX_MAX_PATH                MAX_PATH
 
 #define NGX_DIR_MASK                (u_char *) "/*"


From mdounin at mdounin.ru  Tue Mar 27 16:44:52 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Tue, 27 Mar 2012 16:44:52 +0000
Subject: [nginx] svn commit:  r4560 - in trunk/auto: . lib/pcre os types
Message-ID: <20120327164452.E06B83F9ED8@mail.nginx.com>

Author: mdounin
Date: 2012-03-27 16:44:52 +0000 (Tue, 27 Mar 2012)
New Revision: 4560
URL: http://trac.nginx.org/nginx/changeset/4560/nginx

Log:
Fixed more gcc46 warnings in configure tests.

Steps to reproduce:

./configure --with-cc="gcc46" --with-cc-opt="-Wall -Werror -O2"


Modified:
   trunk/auto/lib/pcre/conf
   trunk/auto/os/solaris
   trunk/auto/types/typedef
   trunk/auto/types/uintptr_t
   trunk/auto/unix

Modified: trunk/auto/lib/pcre/conf
===================================================================
--- trunk/auto/lib/pcre/conf	2012-03-27 16:42:34 UTC (rev 4559)
+++ trunk/auto/lib/pcre/conf	2012-03-27 16:44:52 UTC (rev 4560)
@@ -98,7 +98,9 @@
         ngx_feature_incs="#include <pcre.h>"
         ngx_feature_path=
         ngx_feature_libs="-lpcre"
-        ngx_feature_test="pcre *re; re = pcre_compile(NULL, 0, NULL, 0, NULL)"
+        ngx_feature_test="pcre *re;
+                          re = pcre_compile(NULL, 0, NULL, 0, NULL);
+                          if (re == NULL) return 1"
         . auto/feature
 
         if [ $ngx_found = no ]; then

Modified: trunk/auto/os/solaris
===================================================================
--- trunk/auto/os/solaris	2012-03-27 16:42:34 UTC (rev 4559)
+++ trunk/auto/os/solaris	2012-03-27 16:44:52 UTC (rev 4560)
@@ -35,7 +35,8 @@
 ngx_feature_libs="-lsendfile"
 ngx_feature_test="int fd = 1; sendfilevec_t vec[1];
                   size_t sent; ssize_t n;
-                  n = sendfilev(fd, vec, 1, &sent)"
+                  n = sendfilev(fd, vec, 1, &sent);
+                  if (n == -1) return 1"
 . auto/feature
 
 

Modified: trunk/auto/types/typedef
===================================================================
--- trunk/auto/types/typedef	2012-03-27 16:42:34 UTC (rev 4559)
+++ trunk/auto/types/typedef	2012-03-27 16:44:52 UTC (rev 4560)
@@ -28,9 +28,8 @@
 $NGX_INCLUDE_INTTYPES_H
 
 int main() {
-    $ngx_try i;
-    i = 0;
-    return 0;
+    $ngx_try i = 0;
+    return (int) i;
 }
 
 END

Modified: trunk/auto/types/uintptr_t
===================================================================
--- trunk/auto/types/uintptr_t	2012-03-27 16:42:34 UTC (rev 4559)
+++ trunk/auto/types/uintptr_t	2012-03-27 16:44:52 UTC (rev 4560)
@@ -15,9 +15,8 @@
 $NGX_INTTYPES_H
 
 int main() {
-    uintptr_t i;
-    i = 0;
-    return 0;
+    uintptr_t i = 0;
+    return (int) i;
 }
 
 END

Modified: trunk/auto/unix
===================================================================
--- trunk/auto/unix	2012-03-27 16:42:34 UTC (rev 4559)
+++ trunk/auto/unix	2012-03-27 16:44:52 UTC (rev 4560)
@@ -33,12 +33,12 @@
 ngx_feature_incs="#include <poll.h>"
 ngx_feature_path=
 ngx_feature_libs=
-ngx_feature_test="int  n, dp; struct pollfd  pl;
-                  dp = 0;
+ngx_feature_test="int  n; struct pollfd  pl;
                   pl.fd = 0;
                   pl.events = 0;
                   pl.revents = 0;
-                  n = poll(&pl, 1, 0)"
+                  n = poll(&pl, 1, 0);
+                  if (n == -1) return 1"
 . auto/feature
 
 if [ $ngx_found = no ]; then
@@ -57,7 +57,8 @@
                   dvp.dp_fds = NULL;
                   dvp.dp_nfds = 0;
                   dvp.dp_timeout = 0;
-                  n = ioctl(dp, DP_POLL, &dvp)"
+                  n = ioctl(dp, DP_POLL, &dvp);
+                  if (n == -1) return 1"
 . auto/feature
 
 if [ $ngx_found = yes ]; then
@@ -357,7 +358,7 @@
                   ti.tcpi_rttvar = 0;
                   ti.tcpi_snd_cwnd = 0;
                   ti.tcpi_rcv_space = 0;
-                  getsockopt(0, IPPROTO_TCP, TCP_INFO, NULL, &optlen)"
+                  getsockopt(0, IPPROTO_TCP, TCP_INFO, &ti, &optlen)"
 . auto/feature
 
 
@@ -512,7 +513,8 @@
 ngx_feature_incs=
 ngx_feature_path=
 ngx_feature_libs=
-ngx_feature_test="char buf[1]; ssize_t n; n = pread(0, buf, 1, 0)"
+ngx_feature_test="char buf[1]; ssize_t n; n = pread(0, buf, 1, 0);
+                  if (n == -1) return 1"
 . auto/feature
 
 
@@ -522,7 +524,8 @@
 ngx_feature_incs=
 ngx_feature_path=
 ngx_feature_libs=
-ngx_feature_test="char buf[1]; ssize_t n; n = pwrite(1, buf, 1, 0)"
+ngx_feature_test="char buf[1]; ssize_t n; n = pwrite(1, buf, 1, 0);
+                  if (n == -1) return 1"
 . auto/feature
 
 
@@ -596,7 +599,8 @@
 ngx_feature_incs="#include <stdlib.h>"
 ngx_feature_path=
 ngx_feature_libs=
-ngx_feature_test="void *p; int n; n = posix_memalign(&p, 4096, 4096)"
+ngx_feature_test="void *p; int n; n = posix_memalign(&p, 4096, 4096);
+                  if (n != 0) return 1"
 . auto/feature
 
 
@@ -607,7 +611,8 @@
                   #include <malloc.h>"
 ngx_feature_path=
 ngx_feature_libs=
-ngx_feature_test="void *p; p = memalign(4096, 4096)"
+ngx_feature_test="void *p; p = memalign(4096, 4096);
+                  if (p == NULL) return 1"
 . auto/feature
 
 
@@ -694,10 +699,12 @@
 ngx_feature="struct msghdr.msg_control"
 ngx_feature_name="NGX_HAVE_MSGHDR_MSG_CONTROL"
 ngx_feature_run=no
-ngx_feature_incs="#include <sys/socket.h>"
+ngx_feature_incs="#include <sys/socket.h>
+                  #include <stdio.h>"
 ngx_feature_path=
 ngx_feature_libs=
-ngx_feature_test="struct msghdr  msg; msg.msg_control = NULL"
+ngx_feature_test="struct msghdr  msg; msg.msg_control = NULL;
+                  printf(\"%d\", (int) msg.msg_control)"
 . auto/feature
 
 
@@ -705,40 +712,47 @@
 ngx_feature_name="NGX_HAVE_FIONBIO"
 ngx_feature_run=no
 ngx_feature_incs="#include <sys/ioctl.h>
+                  #include <stdio.h>
                   $NGX_INCLUDE_SYS_FILIO_H"
 ngx_feature_path=
 ngx_feature_libs=
-ngx_feature_test="int i; i = FIONBIO"
+ngx_feature_test="int i = FIONBIO; printf(\"%d\", i)"
 . auto/feature
 
 
 ngx_feature="struct tm.tm_gmtoff"
 ngx_feature_name="NGX_HAVE_GMTOFF"
 ngx_feature_run=no
-ngx_feature_incs="#include <time.h>"
+ngx_feature_incs="#include <time.h>
+                  #include <stdio.h>"
 ngx_feature_path=
 ngx_feature_libs=
-ngx_feature_test="struct tm  tm; tm.tm_gmtoff = 0"
+ngx_feature_test="struct tm  tm; tm.tm_gmtoff = 0;
+                  printf(\"%d\", (int) tm.tm_gmtoff)"
 . auto/feature
 
 
 ngx_feature="struct dirent.d_namlen"
 ngx_feature_name="NGX_HAVE_D_NAMLEN"
 ngx_feature_run=no
-ngx_feature_incs="#include <dirent.h>"
+ngx_feature_incs="#include <dirent.h>
+                  #include <stdio.h>"
 ngx_feature_path=
 ngx_feature_libs=
-ngx_feature_test="struct dirent  dir; dir.d_namlen = 0"
+ngx_feature_test="struct dirent  dir; dir.d_namlen = 0;
+                  printf(\"%d\", (int) dir.d_namlen)"
 . auto/feature
 
 
 ngx_feature="struct dirent.d_type"
 ngx_feature_name="NGX_HAVE_D_TYPE"
 ngx_feature_run=no
-ngx_feature_incs="#include <dirent.h>"
+ngx_feature_incs="#include <dirent.h>
+                  #include <stdio.h>"
 ngx_feature_path=
 ngx_feature_libs=
-ngx_feature_test="struct dirent  dir; dir.d_type = DT_REG"
+ngx_feature_test="struct dirent  dir; dir.d_type = DT_REG;
+                  printf(\"%d\", (int) dir.d_type)"
 . auto/feature
 
 


From mdounin at mdounin.ru  Wed Mar 28 01:56:49 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Wed, 28 Mar 2012 01:56:49 +0000
Subject: [nginx] svn commit:  r4561 - trunk/src/http/modules
Message-ID: <20120328015649.EA3213FA0D6@mail.nginx.com>

Author: mdounin
Date: 2012-03-28 01:56:49 +0000 (Wed, 28 Mar 2012)
New Revision: 4561
URL: http://trac.nginx.org/nginx/changeset/4561/nginx

Log:
Xslt: parser options now set with xmlCtxtUseOptions().

Note that "ctxt->loadsubset = 1" previously used isn't really correct as
ctxt->loadsubset is a bitfield now.  The use of xmlCtxtUseOptions() with
XML_PARSE_DTDLOAD is believed to be a better way to do the same thing.

Patch by Laurence Rowe.


Modified:
   trunk/src/http/modules/ngx_http_xslt_filter_module.c

Modified: trunk/src/http/modules/ngx_http_xslt_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_xslt_filter_module.c	2012-03-27 16:44:52 UTC (rev 4560)
+++ trunk/src/http/modules/ngx_http_xslt_filter_module.c	2012-03-28 01:56:49 UTC (rev 4561)
@@ -362,15 +362,14 @@
                           "xmlCreatePushParserCtxt() failed");
             return NGX_ERROR;
         }
+        xmlCtxtUseOptions(ctxt, XML_PARSE_NOENT|XML_PARSE_DTDLOAD
+                                               |XML_PARSE_NOWARNING);
 
         ctxt->sax->externalSubset = ngx_http_xslt_sax_external_subset;
         ctxt->sax->setDocumentLocator = NULL;
-        ctxt->sax->warning = NULL;
         ctxt->sax->error = ngx_http_xslt_sax_error;
         ctxt->sax->fatalError = ngx_http_xslt_sax_error;
         ctxt->sax->_private = ctx;
-        ctxt->replaceEntities = 1;
-        ctxt->loadsubset = 1;
 
         ctx->ctxt = ctxt;
         ctx->request = r;


From mdounin at mdounin.ru  Wed Mar 28 01:56:57 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 28 Mar 2012 05:56:57 +0400
Subject: [PATCH 1 of 4] Set parser options with xmlCtxtUseOptions
In-Reply-To: <28a1437a8b4492530a24.1332365311@Laurence-Rowes-Mac-mini.local>
References: <patchbomb.1332365310@Laurence-Rowes-Mac-mini.local>
	<28a1437a8b4492530a24.1332365311@Laurence-Rowes-Mac-mini.local>
Message-ID: <20120328015657.GT13466@mdounin.ru>

Hello!

On Wed, Mar 21, 2012 at 09:28:31PM +0000, Laurence Rowe wrote:

> # HG changeset patch
> # User Laurence Rowe <laurence at lrowe.co.uk>
> # Date 1331326885 0
> # Node ID 28a1437a8b4492530a24d51b85aca7923627c330
> # Parent  aed89f1bb357e289086f9a768e2e9958c4076b7c
> Set parser options with xmlCtxtUseOptions
> 
> In the current version of libxml2, loadsubset is a bit field. The
> previous value of 1 does not match any of the loadsubset bit field
> options (XML_DETECT_IDS=2, XML_COMPLETE_ATTRS=4, XML_SKIP_IDS=8) but
> from reading the source, XML_DETECT_IDS is not tested for directly in
> any relevant files, with SAX2.c using the test (ctxt->loadsubset !=
> 0). The loadsubset value is set to XML_DETECT_IDS by the
> xmlCtxtUseOptions call with XML_PARSE_DTDLOAD.
> 
> diff --git a/src/http/modules/ngx_http_xslt_filter_module.c b/src/http/modules/ngx_http_xslt_filter_module.c
> --- a/src/http/modules/ngx_http_xslt_filter_module.c
> +++ b/src/http/modules/ngx_http_xslt_filter_module.c
> @@ -362,15 +362,14 @@
>                            "xmlCreatePushParserCtxt() failed");
>              return NGX_ERROR;
>          }
> +        xmlCtxtUseOptions(ctxt, XML_PARSE_NOENT|XML_PARSE_DTDLOAD
> +                                               |XML_PARSE_NOWARNING);
>  
>          ctxt->sax->externalSubset = ngx_http_xslt_sax_external_subset;
>          ctxt->sax->setDocumentLocator = NULL;
> -        ctxt->sax->warning = NULL;
>          ctxt->sax->error = ngx_http_xslt_sax_error;
>          ctxt->sax->fatalError = ngx_http_xslt_sax_error;
>          ctxt->sax->_private = ctx;
> -        ctxt->replaceEntities = 1;
> -        ctxt->loadsubset = 1;
>  
>          ctx->ctxt = ctxt;
>          ctx->request = r;

This one committed, thanks.

Maxim Dounin


From mdounin at mdounin.ru  Wed Mar 28 02:02:23 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 28 Mar 2012 06:02:23 +0400
Subject: [PATCH 2 of 4] Set done flag on module context to stop further
	chunk parsing
In-Reply-To: <151124d060d3f725c02b.1332365312@Laurence-Rowes-Mac-mini.local>
References: <patchbomb.1332365310@Laurence-Rowes-Mac-mini.local>
	<151124d060d3f725c02b.1332365312@Laurence-Rowes-Mac-mini.local>
Message-ID: <20120328020222.GU13466@mdounin.ru>

Hello!

On Wed, Mar 21, 2012 at 09:28:32PM +0000, Laurence Rowe wrote:

> # HG changeset patch
> # User Laurence Rowe <laurence at lrowe.co.uk>
> # Date 1332363776 0
> # Node ID 151124d060d3f725c02b656d39c10575ff009cdb
> # Parent  28a1437a8b4492530a24d51b85aca7923627c330
> Set done flag on module context to stop further chunk parsing.
> 
> As HTML parsing will recover from non-fatal errors, the return value of
> (x|ht)mlParseChunk cannot be used.
> 
> diff --git a/src/http/modules/ngx_http_xslt_filter_module.c b/src/http/modules/ngx_http_xslt_filter_module.c
> --- a/src/http/modules/ngx_http_xslt_filter_module.c
> +++ b/src/http/modules/ngx_http_xslt_filter_module.c
> @@ -378,7 +378,7 @@
>      err = xmlParseChunk(ctx->ctxt, (char *) b->pos, (int) (b->last - b->pos),
>                          (b->last_buf) || (b->last_in_chain));
>  
> -    if (err == 0) {
> +    if (ctx->done == 0) {
>          b->pos = b->last;
>          return NGX_OK;
>      }
> @@ -463,6 +463,8 @@
>  
>      ngx_log_error(NGX_LOG_ERR, ctx->request->connection->log, 0,
>                    "libxml2 error: \"%*s\"", n + 1, buf);
> +
> +    ctx->done = 1; /* stop further chunk parsing */
>  }

I believe this doesn't cover all possible fatal errors as returned 
by xmlParseChunk().  At least this one isn't covered:

...
    if (ctxt == NULL)
        return(XML_ERR_INTERNAL_ERROR);
...

While it probably not very common, I still think we need a better 
way to differentiate recoverable errors in html parsing.

Maxim Dounin


From mdounin at mdounin.ru  Wed Mar 28 02:02:38 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 28 Mar 2012 06:02:38 +0400
Subject: [PATCH 4 of 4] Handle empty response body
In-Reply-To: <1a21700079c90be39eaf.1332365314@Laurence-Rowes-Mac-mini.local>
References: <patchbomb.1332365310@Laurence-Rowes-Mac-mini.local>
	<1a21700079c90be39eaf.1332365314@Laurence-Rowes-Mac-mini.local>
Message-ID: <20120328020237.GV13466@mdounin.ru>

Hello!

On Wed, Mar 21, 2012 at 09:28:34PM +0000, Laurence Rowe wrote:

> # HG changeset patch
> # User Laurence Rowe <laurence at lrowe.co.uk>
> # Date 1332363819 0
> # Node ID 1a21700079c90be39eaf5f6c5d7aef65be43ff95
> # Parent  65fd4892a78371e863d43e31d4430cdb7333a35d
> Handle empty response body
> 
> Responses, and especially proxied responses, may have empty bodies. These
> should pass through the filter without causing an internal server error or
> crashing nginx.
> 
> Example config:
> 
>     location = /testme {
>         default_type text/xml;
>         return 204;
>         xslt_stylesheet example.xsl;
>     }

This should return 500 as of now, due to malformed xml on input, 
and this is correct behaviour.  Do you see any other problems?

> diff --git a/src/http/modules/ngx_http_xslt_filter_module.c b/src/http/modules/ngx_http_xslt_filter_module.c
> --- a/src/http/modules/ngx_http_xslt_filter_module.c
> +++ b/src/http/modules/ngx_http_xslt_filter_module.c
> @@ -284,10 +284,17 @@
>  
>          if (cl->buf->last_buf || cl->buf->last_in_chain) {
>  
> +            if (ctx->ctxt == NULL) {

This might not happen before this patch due to 
ngx_http_xslt_add_chunk() call just before, which will either 
create ctx->ctxt or fail.

> +                /* empty body */
> +                return ngx_http_next_header_filter(r);

And both the code and comment is incorrect here: 

1. Calling header filter is not enough to provide empty body.

2. I think this should really produce 500, at least in case of 
normal xml parsing.

> +            }
> +
>              ctx->doc = ctx->ctxt->myDoc;
>  
>  #if (NGX_HTTP_XSLT_REUSE_DTD)
> -            ctx->doc->extSubset = NULL;
> +            if (ctx->doc) {
> +                ctx->doc->extSubset = NULL;
> +            }
>  #endif

This wasn't possible either, as ctxt->myDoc was always available 
on successfull parsing.  If it's not available now it some cases 
due to relaxed error handling - this have to be addressed either 
before or with the change which introduce the problem.

>  
>              wellFormed = ctx->ctxt->wellFormed;
> @@ -377,6 +384,10 @@
>  
>      if (ctx->ctxt == NULL) {
>  
> +        if (b->last == b->pos) {
> +            return NGX_OK;
> +        }
> +
>          if (ctx->html_parser) {
>              if (r->headers_out.charset.len) {
>                  enc = xmlParseCharEncoding(

Maxim Dounin


From mikegagnon at gmail.com  Wed Mar 28 04:05:25 2012
From: mikegagnon at gmail.com (Mike Gagnon)
Date: Tue, 27 Mar 2012 21:05:25 -0700
Subject: Socket communication inside load balancer
Message-ID: <CAG23spNgugMc3zb83UjQqCDu4hG6T_nqP-R9Af5MZb6ExRxbWg@mail.gmail.com>

I am working on a new load-balancing module, similar to upstream-fair, but
with different requirements (and hence different features). One of my
planned features is that the module will send an alert when it detects an
overload.

Within my load balancing function (my implementation of
r->upstream->peer.get) I would like to send an alert to an external
process. I am considering sending it over either a socket connection or
perhaps a named pipe.

Does anyone have any tips on how I could accomplish this? Any pointers to
nginx code or 3rd party module code that performs similar communication?

Thanks,
Mike Gagnon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120327/a106adbb/attachment-0001.html>

From ru at nginx.com  Wed Mar 28 06:50:23 2012
From: ru at nginx.com (ru at nginx.com)
Date: Wed, 28 Mar 2012 06:50:23 +0000
Subject: [nginx] svn commit:  r4562 - trunk/src/http/modules
Message-ID: <20120328065023.9B5883F9F0F@mail.nginx.com>

Author: ru
Date: 2012-03-28 06:50:23 +0000 (Wed, 28 Mar 2012)
New Revision: 4562
URL: http://trac.nginx.org/nginx/changeset/4562/nginx

Log:
Fixed calculation of range boundaries.


Modified:
   trunk/src/http/modules/ngx_http_split_clients_module.c

Modified: trunk/src/http/modules/ngx_http_split_clients_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_split_clients_module.c	2012-03-28 01:56:49 UTC (rev 4561)
+++ trunk/src/http/modules/ngx_http_split_clients_module.c	2012-03-28 06:50:23 UTC (rev 4562)
@@ -97,7 +97,7 @@
         ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                        "http split: %uD %uD", hash, part[i].percent);
 
-        if (hash < part[i].percent) {
+        if (hash < part[i].percent || part[i].percent == 0) {
             *v = part[i].value;
             return NGX_OK;
         }
@@ -111,8 +111,9 @@
 ngx_conf_split_clients_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
 {
     char                                *rv;
+    uint32_t                             sum, last;
     ngx_str_t                           *value, name;
-    ngx_uint_t                           i, sum, last;
+    ngx_uint_t                           i;
     ngx_conf_t                           save;
     ngx_http_variable_t                 *var;
     ngx_http_split_clients_ctx_t        *ctx;
@@ -175,19 +176,15 @@
     for (i = 0; i < ctx->parts.nelts; i++) {
         sum = part[i].percent ? sum + part[i].percent : 10000;
         if (sum > 10000) {
-           ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
-                              "percent sum is more than 100%%");
-           return NGX_CONF_ERROR;
+            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                               "percent sum is more than 100%%");
+            return NGX_CONF_ERROR;
         }
 
         if (part[i].percent) {
-            part[i].percent = (uint32_t)
-                                 (last + 0xffffffff / 10000 * part[i].percent);
-        } else {
-            part[i].percent = 0xffffffff;
+            last += part[i].percent * (uint64_t) 0xffffffff / 10000;
+            part[i].percent = last;
         }
-
-        last = part[i].percent;
     }
 
     return rv;


From ru at nginx.com  Wed Mar 28 09:29:09 2012
From: ru at nginx.com (ru at nginx.com)
Date: Wed, 28 Mar 2012 09:29:09 +0000
Subject: [nginx] svn commit:  r4563 - trunk/src/http/modules
Message-ID: <20120328092909.ADC433FB456@mail.nginx.com>

Author: ru
Date: 2012-03-28 09:29:09 +0000 (Wed, 28 Mar 2012)
New Revision: 4563
URL: http://trac.nginx.org/nginx/changeset/4563/nginx

Log:
Replaced ngx_http_realip_from_t with ngx_in_cidr_t.


Modified:
   trunk/src/http/modules/ngx_http_realip_module.c

Modified: trunk/src/http/modules/ngx_http_realip_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_realip_module.c	2012-03-28 06:50:23 UTC (rev 4562)
+++ trunk/src/http/modules/ngx_http_realip_module.c	2012-03-28 09:29:09 UTC (rev 4563)
@@ -16,13 +16,7 @@
 
 
 typedef struct {
-    in_addr_t          mask;
-    in_addr_t          addr;
-} ngx_http_realip_from_t;
-
-
-typedef struct {
-    ngx_array_t       *from;     /* array of ngx_http_realip_from_t */
+    ngx_array_t       *from;     /* array of ngx_in_cidr_t */
     ngx_uint_t         type;
     ngx_uint_t         hash;
     ngx_str_t          header;
@@ -114,9 +108,9 @@
     ngx_list_part_t             *part;
     ngx_table_elt_t             *header;
     struct sockaddr_in          *sin;
+    ngx_in_cidr_t               *from;
     ngx_connection_t            *c;
     ngx_http_realip_ctx_t       *ctx;
-    ngx_http_realip_from_t      *from;
     ngx_http_realip_loc_conf_t  *rlcf;
 
     ctx = ngx_http_get_module_ctx(r, ngx_http_realip_module);
@@ -317,7 +311,7 @@
     ngx_int_t                rc;
     ngx_str_t               *value;
     ngx_cidr_t               cidr;
-    ngx_http_realip_from_t  *from;
+    ngx_in_cidr_t           *from;
 
     value = cf->args->elts;
 
@@ -332,7 +326,7 @@
 
     if (rlcf->from == NULL) {
         rlcf->from = ngx_array_create(cf->pool, 2,
-                                      sizeof(ngx_http_realip_from_t));
+                                      sizeof(ngx_in_cidr_t));
         if (rlcf->from == NULL) {
             return NGX_CONF_ERROR;
         }


From mdounin at mdounin.ru  Wed Mar 28 12:38:04 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Wed, 28 Mar 2012 12:38:04 +0000
Subject: [nginx] svn commit:  r4564 - trunk/auto
Message-ID: <20120328123804.915073FA6F8@mail.nginx.com>

Author: mdounin
Date: 2012-03-28 12:38:03 +0000 (Wed, 28 Mar 2012)
New Revision: 4564
URL: http://trac.nginx.org/nginx/changeset/4564/nginx

Log:
Configure: fixed msghdr.msg_control test on 64bit platforms.

Broken by r4560.


Modified:
   trunk/auto/unix

Modified: trunk/auto/unix
===================================================================
--- trunk/auto/unix	2012-03-28 09:29:09 UTC (rev 4563)
+++ trunk/auto/unix	2012-03-28 12:38:03 UTC (rev 4564)
@@ -703,8 +703,8 @@
                   #include <stdio.h>"
 ngx_feature_path=
 ngx_feature_libs=
-ngx_feature_test="struct msghdr  msg; msg.msg_control = NULL;
-                  printf(\"%d\", (int) msg.msg_control)"
+ngx_feature_test="struct msghdr  msg;
+                  printf(\"%d\", (int) sizeof(msg.msg_control))"
 . auto/feature
 
 


From mdounin at mdounin.ru  Wed Mar 28 13:04:40 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Wed, 28 Mar 2012 13:04:40 +0000
Subject: [nginx] svn commit:  r4565 - trunk/src/os/win32
Message-ID: <20120328130440.2D9503FA750@mail.nginx.com>

Author: mdounin
Date: 2012-03-28 13:04:39 +0000 (Wed, 28 Mar 2012)
New Revision: 4565
URL: http://trac.nginx.org/nginx/changeset/4565/nginx

Log:
Fixed win32 build after realpath changes in r4559.


Modified:
   trunk/src/os/win32/ngx_files.c
   trunk/src/os/win32/ngx_files.h

Modified: trunk/src/os/win32/ngx_files.c
===================================================================
--- trunk/src/os/win32/ngx_files.c	2012-03-28 12:38:03 UTC (rev 4564)
+++ trunk/src/os/win32/ngx_files.c	2012-03-28 13:04:39 UTC (rev 4565)
@@ -426,11 +426,11 @@
 }
 
 
-char *
+u_char *
 ngx_realpath(u_char *path, u_char *resolved)
 {
     /* STUB */
-    return (char *) path;
+    return path;
 }
 
 

Modified: trunk/src/os/win32/ngx_files.h
===================================================================
--- trunk/src/os/win32/ngx_files.h	2012-03-28 12:38:03 UTC (rev 4564)
+++ trunk/src/os/win32/ngx_files.h	2012-03-28 13:04:39 UTC (rev 4565)
@@ -177,7 +177,7 @@
 #define ngx_filename_cmp(s1, s2, n) _strnicmp((char *) s1, (char *) s2, n)
 
 
-char *ngx_realpath(u_char *path, u_char *resolved);
+u_char *ngx_realpath(u_char *path, u_char *resolved);
 #define ngx_realpath_n              ""
 #define ngx_getcwd(buf, size)       GetCurrentDirectory(size, (char *) buf)
 #define ngx_getcwd_n                "GetCurrentDirectory()"


From mdounin at mdounin.ru  Wed Mar 28 13:29:29 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Wed, 28 Mar 2012 13:29:29 +0000
Subject: [nginx] svn commit:  r4566 - trunk/docs/xml/nginx
Message-ID: <20120328132929.C6F7B3FA3DD@mail.nginx.com>

Author: mdounin
Date: 2012-03-28 13:29:29 +0000 (Wed, 28 Mar 2012)
New Revision: 4566
URL: http://trac.nginx.org/nginx/changeset/4566/nginx

Log:
nginx-1.1.18-RELEASE


Modified:
   trunk/docs/xml/nginx/changes.xml

Modified: trunk/docs/xml/nginx/changes.xml
===================================================================
--- trunk/docs/xml/nginx/changes.xml	2012-03-28 13:04:39 UTC (rev 4565)
+++ trunk/docs/xml/nginx/changes.xml	2012-03-28 13:29:29 UTC (rev 4566)
@@ -9,6 +9,89 @@
 <title lang="en">nginx changelog</title>
 
 
+<changes ver="1.1.18" date="28.03.2012">
+
+<change type="change">
+<para lang="ru">
+?????? keepalive ?????????? ?? ????????? ??? Safari ?? ?????????.
+</para>
+<para lang="en">
+keepalive connections are no longer disabled for Safari by default.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+?????????? $connection_requests.
+</para>
+<para lang="en">
+the $connection_requests variable.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+?????????? $tcpinfo_rtt, $tcpinfo_rttvar, $tcpinfo_snd_cwnd ?
+$tcpinfo_rcv_space.
+</para>
+<para lang="en">
+$tcpinfo_rtt, $tcpinfo_rttvar, $tcpinfo_snd_cwnd and
+$tcpinfo_rcv_space variables.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+????????? worker_cpu_affinity ?????? ???????? ?? FreeBSD.
+</para>
+<para lang="en">
+the "worker_cpu_affinity" directive now works on FreeBSD.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+????????? xslt_param ? xslt_string_param.<br/>
+??????? Samuel Behan.
+</para>
+<para lang="en">
+the "xslt_param" and "xslt_string_param" directives.<br/>
+Thanks to Samuel Behan.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? configure.<br/>
+??????? Piotr Sikora.
+</para>
+<para lang="en">
+in configure tests.<br/>
+Thanks to Piotr Sikora.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ?????? ngx_http_xslt_filter_module.
+</para>
+<para lang="en">
+in the ngx_http_xslt_filter_module.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+nginx ?? ????????? ?? Debian GNU/Hurd.
+</para>
+<para lang="en">
+nginx could not be built on Debian GNU/Hurd.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.1.17" date="15.03.2012">
 
 <change type="security">


From mdounin at mdounin.ru  Wed Mar 28 13:29:55 2012
From: mdounin at mdounin.ru (mdounin at mdounin.ru)
Date: Wed, 28 Mar 2012 13:29:55 +0000
Subject: [nginx] svn commit:  r4567 - tags
Message-ID: <20120328132955.537983FA750@mail.nginx.com>

Author: mdounin
Date: 2012-03-28 13:29:55 +0000 (Wed, 28 Mar 2012)
New Revision: 4567
URL: http://trac.nginx.org/nginx/changeset/4567/nginx

Log:
release-1.1.18 tag


Added:
   tags/release-1.1.18/


From ru at nginx.com  Thu Mar 29 19:47:27 2012
From: ru at nginx.com (ru at nginx.com)
Date: Thu, 29 Mar 2012 19:47:27 +0000
Subject: [nginx] svn commit:  r4568 - trunk/src/http/modules
Message-ID: <20120329194727.B3C703FA941@mail.nginx.com>

Author: ru
Date: 2012-03-29 19:47:27 +0000 (Thu, 29 Mar 2012)
New Revision: 4568
URL: http://trac.nginx.org/nginx/changeset/4568/nginx

Log:
Corrected spelling of error message (ticket #136).


Modified:
   trunk/src/http/modules/ngx_http_ssi_filter_module.c

Modified: trunk/src/http/modules/ngx_http_ssi_filter_module.c
===================================================================
--- trunk/src/http/modules/ngx_http_ssi_filter_module.c	2012-03-28 13:29:55 UTC (rev 4567)
+++ trunk/src/http/modules/ngx_http_ssi_filter_module.c	2012-03-29 19:47:27 UTC (rev 4568)
@@ -714,7 +714,7 @@
 
                 if (ctx->params.nelts > NGX_HTTP_SSI_MAX_PARAMS) {
                     ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
-                                  "too many SSI command paramters: \"%V\"",
+                                  "too many SSI command parameters: \"%V\"",
                                   &ctx->command);
                     goto ssi_error;
                 }


From piotr.sikora at frickle.com  Fri Mar 30 12:50:48 2012
From: piotr.sikora at frickle.com (Piotr Sikora)
Date: Fri, 30 Mar 2012 14:50:48 +0200
Subject: [PATCH] ngx_http_upstream state machine fixes
In-Reply-To: <20120321153851.GS88788@mdounin.ru>
References: <CAB4Tn6MzNVGXFW8s3ybtOgdGjaHrDvSQtrJFxijGcMz8QdCw=A@mail.gmail.com>
	<20120321153851.GS88788@mdounin.ru>
Message-ID: <36EE45F319C44D18B753541B2A8005C9@Desktop>

Hi,

> The major question seems to be "what should be done if we've not
> yet sent full request, but already got response"?  Right now line
> is drawn at "if we've got response header, we stop sending
> request".  And if we move the line - we should clearly understand
> where the new line is and why.

While current approach seems acceptable for single request-response 
connections, I'm pretty sure that it can be source of corruption for 
keepalived connections (in case when
backend doesn't read rest of the request after sending response or when 
backend wants to read rest of the request that wasn't completely sent).

I see two possible solutions: either kill connection after such "incomplete" 
request is made or make sure to send whole request even if we receive 
response while doing so.

Opinions?

Best regards,
Piotr Sikora < piotr.sikora at frickle.com >


From mdounin at mdounin.ru  Sat Mar 31 14:48:05 2012
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sat, 31 Mar 2012 18:48:05 +0400
Subject: [PATCH] ngx_http_upstream state machine fixes
In-Reply-To: <36EE45F319C44D18B753541B2A8005C9@Desktop>
References: <CAB4Tn6MzNVGXFW8s3ybtOgdGjaHrDvSQtrJFxijGcMz8QdCw=A@mail.gmail.com>
	<20120321153851.GS88788@mdounin.ru>
	<36EE45F319C44D18B753541B2A8005C9@Desktop>
Message-ID: <20120331144805.GA13466@mdounin.ru>

Hello!

On Fri, Mar 30, 2012 at 02:50:48PM +0200, Piotr Sikora wrote:

> Hi,
> 
> >The major question seems to be "what should be done if we've not
> >yet sent full request, but already got response"?  Right now line
> >is drawn at "if we've got response header, we stop sending
> >request".  And if we move the line - we should clearly understand
> >where the new line is and why.
> 
> While current approach seems acceptable for single request-response
> connections, I'm pretty sure that it can be source of corruption for
> keepalived connections (in case when
> backend doesn't read rest of the request after sending response or
> when backend wants to read rest of the request that wasn't
> completely sent).
> 
> I see two possible solutions: either kill connection after such
> "incomplete" request is made or make sure to send whole request even
> if we receive response while doing so.
> 
> Opinions?

I tend to think that connection should be closed in this case.  
And it's actually likely to be closed by an upstream server 
anyway.

Maxim Dounin