[PATCH] sub_filter bug

[Maxim Dounin]

Hello!

On Thu, May 22, 2014 at 04:53:40PM +0900, anthrax at unixuser.org wrote:

> Hi,
> 
> Could you take a look at the following patch?
> 
> 
> # HG changeset patch
> # User KAWAHARA Masashi <anthrax at unixuser.org>
> # Date 1400671481 -32400
> #      Wed May 21 20:24:41 2014 +0900
> # Node ID 3949e591694fff37a436a076f4d4006c802587c7
> # Parent  1209b8a7b077c7bbf161e502964308736c84db6f
> Bugfix: When both sub_filter and SSI were used and the include file was
>        not terminated by newline, it was transmitted incorrectly.
> ex.
> conf:
>   ssi on
>   sub_filter '//check-string' '//replace-string'
> 
> contents:
>   index.shtml:
>     <!--#include virtual="/test.html"-->//check-string
> 
>   test.html:  (!! saved no newline at end of line !!)
>     include-text/
> 
> output:
>   include-test//replace-string
> 
> correct output:
>   include-test///replace-string

The problem is certainly valid, thanks.  The sub filter doesn't 
output partial match at the end of a subrequest response (if any), 
and this is certainly bad.

The patch looks wrong though, it breaks cross-buffer matching 
within a single responses, and doesn't work for more than one 
character partial matches.

Here are suggested patch (tests can be found at 
http://hg.nginx.org/nginx-tests/rev/811fbc213fd8):

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1400778091 -14400
#      Thu May 22 21:01:31 2014 +0400
# Node ID 2fa527ce5797db76f393c4e1862175381276e7f3
# Parent  7deb014514861b296adaa57af5c955459d7e25ff
Sub filter: fixed subrequests handling.

In particular, properly output partial match at the end of a subrequest
response (much like we do at the end of a response), and reset/set the
last_in_chain flag as appropriate.

Reported by KAWAHARA Masashi.

diff --git a/src/http/modules/ngx_http_sub_filter_module.c b/src/http/modules/ngx_http_sub_filter_module.c
--- a/src/http/modules/ngx_http_sub_filter_module.c
+++ b/src/http/modules/ngx_http_sub_filter_module.c
@@ -305,6 +305,7 @@ ngx_http_sub_body_filter(ngx_http_reques
                 b->last = ctx->copy_end;
                 b->shadow = NULL;
                 b->last_buf = 0;
+                b->last_in_chain = 0;
                 b->recycled = 0;
 
                 if (b->in_file) {
@@ -374,7 +375,9 @@ ngx_http_sub_body_filter(ngx_http_reques
             continue;
         }
 
-        if (ctx->buf->last_buf && ctx->looked.len) {
+        if (ctx->looked.len
+            && (ctx->buf->last_buf || ctx->buf->last_in_chain))
+        {
             cl = ngx_chain_get_free_buf(r->pool, &ctx->free);
             if (cl == NULL) {
                 return NGX_ERROR;
@@ -394,7 +397,7 @@ ngx_http_sub_body_filter(ngx_http_reques
             ctx->looked.len = 0;
         }
 
-        if (ctx->buf->last_buf || ctx->buf->flush
+        if (ctx->buf->last_buf || ctx->buf->flush || ctx->buf->sync
             || ngx_buf_in_memory(ctx->buf))
         {
             if (b == NULL) {
@@ -414,6 +417,7 @@ ngx_http_sub_body_filter(ngx_http_reques
             }
 
             b->last_buf = ctx->buf->last_buf;
+            b->last_in_chain = ctx->buf->last_in_chain;
             b->flush = ctx->buf->flush;
             b->shadow = ctx->buf;
 

-- 
Maxim Dounin
http://nginx.org/