Subrequests from body filters

[Maxim Dounin]

Hello!

On Mon, Mar 25, 2013 at 07:41:16PM +0400, Marat Dakota wrote:

> >> >> >> But is it ok to call next body filter in subrequest's body filter to
> >> >> >> produce output to main request?
> >> >> >> I mean ngx_http_next_body_filter(r->main, out).
> >> >> >
> >> >> > No.  You should call next body filter of the request you are
> >> >> > working with.  It's postpone filter responsibility to manage
> >> >> > subrequests output, and if you try to do this yourself instead -
> >> >> > result will be undefined.
> >> >>
> >> >> It seems to work as expected for me. How can I cause problems with this?
> >> >
> >> > Undefined behaviour sometimes appear to work as expected.  This
> >> > doesn't mean it's correct though.
> >> >
> >> > Depending on the exact place in a filter chain where you did it
> >> > and various other factors like timings, results may vary from
> >> > "nothing bad might happen, as r == r->main anyway" to "response
> >> > will completely incorrect as wrong filters will be applied to the
> >> > response body".
> >> >
> >> > Most trivial thing to test is probably a subrequest order, which
> >> > likely will be wrong in your case if first subrequest will take
> >> > longer to handle than second one.
> >>
> >> Subrequests order doesn't matter much for me. I feed my library (the
> >> one I write a Nginx module for) with a subrequests results in a
> >> whatever order and my library returns next chunk of response only when
> >> it is ready.
> >>
> >> My library has just one function to call. This function returns the
> >> next chunk of data (if any) to send as a response and/or a list of
> >> subrequests to make. In every call to subrequest body filter I pass
> >> subrequest's response to my library and get a new list of subrequests
> >> (if any) and a new chunk of final response (if any). And so on, until
> >> my library says it's done.
> >>
> >> And if I really do something wrong in terms of Nginx architecture,
> >> please, could you give me more details about how to achieve my goals
> >> correctly?
> >
> > I don't really see why you don't call ngx_http_next_body_filter(r,
> > out), which is perfectly correct.
> 
> Because I'm getting a mess with the chunk order.
> 
> Let's suppose I've made two subrequests in a handler.
> 
> ngx_http_subrequest(...); // first
> ngx_http_subrequest(...); // second
> 
> If I call ngx_http_next_body_filter(r, out) in a subrequest body
> filter, I get the following.
> Let's suppose we've received a response from the second subrequest
> before any data from the first subrequest. And my library said I
> should send "aaa" as a response (and I passed "aaa" to
> ngx_http_next_body_filter).
> Then, we've got a response from the first subrequest and my library
> said I should send "bbb" as a response. I would expect "aaabbb" as the
> final result. But real result would be "bbbaaa" (because
> ngx_http_subrequest for first subrequest is earlier).
> 
> How to deal with this?

So the part of code in your body filter which tried to call 
ngx_http_output_filter(r->main, out) is actually not a part of a 
body filter, but a original request response generation, right?

For such a case I would recommend looking at 
NGX_HTTP_SUBREQUEST_IN_MEMORY and NGX_HTTP_SUBREQUEST_WAITED 
functionality, and using post subrequest handler to trigger 
parent request content generation.

-- 
Maxim Dounin
http://nginx.org/en/donation.html