Parallel subrequests for multi-source long polling?

[shaun]

Here's a description of what I'm hoping to accomplish:
A request comes in and, based on the content of the GET arguments or POST data, one or more backends will be selected.  The request (with a bit of modification) will then be proxied to all of the selected backends.  These backends might return data immediately or could hang waiting for data for a indeterminate amount of time.  I'd like to return response from whichever backend responds first and then cancel or ignore the responses from the rest of the pending requests.  In the case of errors (502's, 500's, whatever), the an error wouldn't be returned unless all the backends have failed.

I wrote module a while ago that handles the much simpler single backend case using the built in upstream functionality, but that certainly isn't going to work for multiple requests.

As far as I can tell, I should be able to get most of what I need using subrequests, but after a day of hacking around on it, I haven't had much luck getting anything working.  Part of the problem is likely that I'm trying to use them from a handler, rather than a filter, and capturing the output instead of letting nginx just return it inline.   I haven't seen any code that looks applicable to whatt I'm trying to do, so I have no idea if my approach is appropriate or not. 

Anyway, any suggestions are welcome.  I'd like to do this using as much of the higher level nginx pieces as possible, but if that isn't an option I can probably just dig in to the connection or event level code and try to hook in there.  That seems horribly ugly, though, and I'd prefer to do it the right way if there is a right way.

Thanks!
Shaun Lindsay

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,23767,23767#msg-23767