Parallel subrequests for multi-source long polling?

[agentzh]

On Wed, Nov 18, 2009 at 2:30 PM, shaun <nginx-forum at nginx.us> wrote:
> Here's a description of what I'm hoping to accomplish:

Here's a proposal for your consideration:

1. You need a content handler and an output filter to work together.
2. Use a content handler to issue all the subrequests (to all your
upstream backends). We assume the content handler lives in the main
request.
3. When starting each subrequest, the content handler feeds a
"post_subrequest" handle and the main request's ctx object into the
ngx_http_subrequest call. Also, the content handler registers a ctx
object to the subrequest object to inform the subrequest's output
filter to buffer the response.
4. In your output filter, first check if there's a ctx object
associated with the current request. If yes, then you're filtering one
of the subrequests that your content handler starts. So now simply
buffer the response chain link in this ctx object (in meomory or on
disk, it's up to you).
5. In your "post_subrequest" handle, check the main request's ctx
object (it should the the last argument passed in). We assume there's
a "success" flag and a "failures" counter in that ctx object (both are
zeroed initially), as well as the total number of subrequests. If
"success" flag is not set and if the current subrequest succeeds,
simply return the buffered response contents in your subrequest's ctx
object, and set a "success" flag in your main request's ctx object. If
the current subrequest fails, then increment the "failures" counter.
If the "failures" counter reaches the total number of the subrequests
(which means all tests are failed), then returns the final failure
(and set the main request's status code if appropriate and send
headers for the main request.)

Your main request, i.e., the content handler should not send headers
or any contents by itself and should return NGX_DONE.

I think this model should work after my various experiments in the
"echo" module, but I could be wrong ;) Feel free to tell us your
findings on your way :)

Good luck!
-agentzh