plans for chunked encoding

Maxim Dounin mdounin at mdounin.ru
Fri Nov 23 02:21:20 UTC 2012 

Hello!

On Thu, Nov 22, 2012 at 09:47:42PM +0100, Luka Perkov wrote:

> Hi Maxim,
> 
> On Thu, Nov 22, 2012 at 08:24:29PM +0400, Maxim Dounin wrote:
> ...
> 
> > > $ curl -v -X POST -d "@/dev/null" --header "Transfer-Encoding: chunked" http://127.0.0.1:80/
> > > 
> > > Returns 404 in headers but 500 in html body. Is this how it's supposed to be ?
> 
> ...
> 
> > If you indeed see 500 in body it might be something to do with 
> > either your module or nginx config you use.
> 
> Here is full information from my side... First the nginx logs:
> 
> 2012/11/22 21:46:00 [debug] 1237#0: *1 http cl:-1 max:1048576
> 2012/11/22 21:46:00 [debug] 1237#0: *1 rewrite phase: 2
> 2012/11/22 21:46:00 [debug] 1237#0: *1 post rewrite phase: 3
> 2012/11/22 21:46:00 [debug] 1237#0: *1 generic phase: 4
> 2012/11/22 21:46:00 [debug] 1237#0: *1 generic phase: 5
> 2012/11/22 21:46:00 [debug] 1237#0: *1 access phase: 6
> 2012/11/22 21:46:00 [debug] 1237#0: *1 post access phase: 7
> 2012/11/22 21:46:00 [debug] 1237#0: *1 content phase: 8
> 2012/11/22 21:46:00 [debug] 1237#0: *1 open index "/etc/nginx/html/index.html"
> 2012/11/22 21:46:00 [debug] 1237#0: *1 stat() "/etc/nginx/html/index.html" failed (2: No such file or directory)
> 2012/11/22 21:46:00 [debug] 1237#0: *1 http index check dir: "/etc/nginx/html"
> 2012/11/22 21:46:00 [error] 1237#0: *1 "/etc/nginx/html/index.html" is not found (2: No such file or directory), client: 127.0.0.1, server: , request: "POST / HTTP/1.1", host: "127.0.0.1"
> 2012/11/22 21:46:00 [debug] 1237#0: *1 http finalize request: 404, "/?" a:1, c:1
> 2012/11/22 21:46:00 [debug] 1237#0: *1 http special response: 404, "/?"
> 2012/11/22 21:46:00 [debug] 1237#0: *1 http set discard body
> 2012/11/22 21:46:00 [debug] 1237#0: *1 http chunked byte: 0D s:0
> 2012/11/22 21:46:00 [error] 1237#0: *1 client sent invalid chunked body, client: 127.0.0.1, server: , request: "POST / HTTP/1.1", host: "127.0.0.1"
> 2012/11/22 21:46:00 [debug] 1237#0: *1 HTTP/1.1 404 Not Found^M
> Server: nginx/1.3.9^M
> Date: Thu, 22 Nov 2012 20:46:00 GMT^M
> Content-Type: text/html^M
> Content-Length: 192^M
> Connection: keep-alive^M
> Keep-Alive: timeout=130^M

[...]

Ok, I see what goes on.  In your case (likely due to different 
curl version) request body is corrupted from start and nginx is 
able to detect the corruption while discarding request body.

This results in 500 response body due to the followin code in 
ngx_http_special_response_handler():

    if (ngx_http_discard_request_body(r) != NGX_OK) {
        error = NGX_HTTP_INTERNAL_SERVER_ERROR;
    }

Two patches below fixes this, as well as couple of other problems 
with chunked body discarding identified while testing.

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1353632132 -14400
# Node ID 7885ce53454c5970f2479763645c4885b2bd3937
# Parent  c10bf2cacc89dc3a4bab1eea0cd742dae2034fd5
Request body: fixed discard of chunked request body.

Even if there is no preread data, make sure to always call
ngx_http_discard_request_body_filter() in case of chunked request
body to initialize r->headers_in.content_length_n for later use.

diff --git a/src/http/ngx_http_request_body.c b/src/http/ngx_http_request_body.c
--- a/src/http/ngx_http_request_body.c
+++ b/src/http/ngx_http_request_body.c
@@ -459,7 +459,7 @@ ngx_http_discard_request_body(ngx_http_r
 
     size = r->header_in->last - r->header_in->pos;
 
-    if (size) {
+    if (size || r->headers_in.chunked) {
         rc = ngx_http_discard_request_body_filter(r, r->header_in);
 
         if (rc != NGX_OK) {
# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1353632948 -14400
# Node ID 66e38e53aedec0618931c748ade5f83a3489e840
# Parent  7885ce53454c5970f2479763645c4885b2bd3937
Request body: improved handling of incorrect chunked request body.

While discarding chunked request body in some cases after detecting
request body corruption no error was returned, while it was possible
to correctly return 400 Bad Request.  If error is detected too late,
make sure to properly close connection.

Additionally, in ngx_http_special_response_handler() don't return body
of 500 Internal Server Error to a client if ngx_http_discard_request_body()
fails, but disable keepalive and continue.

diff --git a/src/http/ngx_http_request_body.c b/src/http/ngx_http_request_body.c
--- a/src/http/ngx_http_request_body.c
+++ b/src/http/ngx_http_request_body.c
@@ -471,12 +471,18 @@ ngx_http_discard_request_body(ngx_http_r
         }
     }
 
-    if (ngx_http_read_discarded_request_body(r) == NGX_OK) {
+    rc = ngx_http_read_discarded_request_body(r);
+
+    if (rc == NGX_OK) {
         r->lingering_close = 0;
         return NGX_OK;
     }
 
-    /* == NGX_AGAIN */
+    if (rc >= NGX_HTTP_SPECIAL_RESPONSE) {
+        return rc;
+    }
+
+    /* rc == NGX_AGAIN */
 
     r->read_event_handler = ngx_http_discarded_request_body_handler;
 
@@ -533,6 +539,12 @@ ngx_http_discarded_request_body_handler(
         return;
     }
 
+    if (rc >= NGX_HTTP_SPECIAL_RESPONSE) {
+        c->error = 1;
+        ngx_http_finalize_request(r, NGX_ERROR);
+        return;
+    }
+
     /* rc == NGX_AGAIN */
 
     if (ngx_handle_read_event(rev, 0) != NGX_OK) {
@@ -606,8 +618,7 @@ ngx_http_read_discarded_request_body(ngx
         rc = ngx_http_discard_request_body_filter(r, &b);
 
         if (rc != NGX_OK) {
-            r->connection->error = 1;
-            return NGX_OK;
+            return rc;
         }
     }
 }
diff --git a/src/http/ngx_http_special_response.c b/src/http/ngx_http_special_response.c
--- a/src/http/ngx_http_special_response.c
+++ b/src/http/ngx_http_special_response.c
@@ -421,7 +421,7 @@ ngx_http_special_response_handler(ngx_ht
     r->expect_tested = 1;
 
     if (ngx_http_discard_request_body(r) != NGX_OK) {
-        error = NGX_HTTP_INTERNAL_SERVER_ERROR;
+        r->keepalive = 0;
     }
 
     if (clcf->msie_refresh



-- 
Maxim Dounin
http://nginx.com/support.html

More information about the nginx-devel mailing list