Parallel subrequests for multi-source long polling?

[agentzh]

On Thu, Nov 19, 2009 at 4:54 PM, shaun <nginx-forum at nginx.us> wrote:
>
> Well, I've got everything *almost* working.

Yay!

> Not calling ngx_http_finalize_request on the parent request cost me a couple of hours of debugging mysterious hangs when there were multiple subrequests, but after doing that the hang goes away and the output looks good.

Nice :)

>  So far, it works perfectly if all the requests return immediately, if all the requests error out and if all the requests but one error out.  The only remaining issue I'm having is when there are 2 or more subrequests the main request does return until both subrequests finish.  In my test case, I have one subrequest return after 0.1 seconds and the other after 10 seconds.  It returns the output from the fast request, as expected, but it returns it after 10 seconds.  Very strange.

Yes, I was aware of this and wanted to say you need to cancel the
pending subrequests when one subrequest succeeds.

It's weird that the client sees the response header and body after the
slowest subrequest finishes. It seems that the response has been
buffered in the last few output filters somehow. Please ensure that
you have set b->flush and b->last_buf in your output chain link. These
flags should defeat buffering in most cases.

And still, we have to cancel the pending subrequests, but close the
connection seems a bit overkill especially in the context of HTTP
keepalive.

>
> I'm guessing I need to do something special to cancel the pending subrequests, or possibly force the completion of the parent request regardless of the state of the rest of subrequests.

Indeed.

> I tried calling ngx_http_finalize_request with various rc values on the pending subrequests, hoping to find something that would force the completion, but nothing seemed to work

What rc values have you tried? Can you publish your code?

> (I can get the parent request to close immediately, but that doesn't really help since it hasn't sent the good request's response yet).
>

Try forcibly flushing the buffered response headers and body on the
parent request :)

Cheers,
-agentzh