CPS-chained subrequests with I/O interceptions no longer work in nginx 0.8.21 (Was Re: Custom event + timer regressions caused by the new release)

[agentzh]

On Wed, Oct 28, 2009 at 6:10 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> This isn't really natural (and I talked to Igor about this
> recently, probably this should be changed to simplify things), but
> that's how it works now.
>

So how to detect the end of a subrequest's output stream in a
subrequest's output filter? Like in the context of the "addition"
module? (The "addition" module explicitly disallowed any uses in a
subrequest.)

>
> One obvious error I see is that your code tries to do something
> once it passed buffer with last_buf set downstream.  This isn't
> going to work.
>

Oh, not really.

It passed buffer with last_buf set downstream, but it does bot do
anything with the *current* request object. Rather, it tries to do
something with its *parent* request object if its parent request has
not sent "last buf" yet. Well, I know it is not that obvious here ;)

In other words, it *reopens* the continuation and resumes the
execution of its parent request's content handler's tasks. And that's
exactly why I use the term "continuation" and "CPS" all over this
thread. It's a CPS chain that backtracks, rather than a strictly
linear subrequest chain that never goes back. (The latter will be
illustrated using a patched "addition" module below.)

>
> You was asked to produce reduced code for reason.  Please do so.
> Your module is huge enough to be hard to analyse, and there are
> too many ways to screw things up.
>

I'll try to produce a standalone module to demonstrate this issue.
It's nontrivial and will take some time :)

> Alternatively you may try to reproduce problem with SSI and/or
> addition filter.
>

The "addition" module does not really work in subrequests yet. In
fact, it explicitly checks r != r->main in its header filter. I've
patched it to make it work recursively in subrequests and could not
reproduce the hang:

    addition_types text/plain;
    location /main {
        echo main;
        add_after_body /sub;
    }
    location /sub {
        echo sub;
        add_after_body /sub2;
    }
    location /sub2 {
        proxy_pass 'http://127.0.0.1:$server_port/foo';
    }
    location /foo {
        echo foo;
    }

I'm getting the expected response of /main without hanging:

  main
  sub
  foo

So I'm guessing it's the subsubrequests I issued on the parent request
of the current subrequest that makes a difference.

Could anyone confirm the justification of following modification of
nginx 0.8.21?

--- nginx-0.8.20/src/http/ngx_http_request.c    2009-10-02
19:30:47.000000000 +0800
+++ nginx-0.8.21/src/http/ngx_http_request.c    2009-10-22
17:48:42.000000000 +0800
@@ -2235,6 +2248,8 @@
           ngx_log_debug2(NGX_LOG_DEBUG_HTTP, wev->log, 0,
                          "http writer done: \"%V?%V\"", &r->uri, &r->args);

+           r->write_event_handler = ngx_http_request_empty_handler;
+
           ngx_http_finalize_request(r, rc);
       }

It causes all these troubles on my side.

Thanks!
-agentzh