About cleanup of subrequests

[Chieu]

Thanks for your apply.

I tried the way to use single pool for each subrequest. But I found
it's can't work for lots of things.

The memory I want to free is the data fethed from upstream,that is
from proxy module.

I tried to use ngx_pfree to free the memory, but I can't find the
callback which i can register to do this work.
so, I want to know is there any callback handler to set to ngx_pfree
the output data.

Before,I used ngx_pfree in ngx_http_post_subrequest_t->handler. But
I'm wrong because the output data may not be sent over.

To Agentzh, I found in echo module ,that pr->write_event_handler is
set in  ngx_http_post_subrequest_t->handler.
So, I just guess maybe I can set the pr->write_event_handler, which do
ngx_pfree, in  ngx_http_post_subrequest_t->handler, just like the echo
module did.

2010/11/17 agentzh <agentzh at gmail.com>:
> On Wed, Nov 17, 2010 at 4:33 PM, Chieu <lfchieu at gmail.com> wrote:
>> To agentzh,
>>
>
> I've cc'd the nginx-devel mailing list, BTW.
>
>> I had read your lua, echo module.
>>
>> I think the location.capture of lua module, and echo_location_async of
>> echo module may cause this problem, too
>>
>
> By design (well, I mean by Igor Sysoev's design), nginx subrequests do
> share the same memory pool with the main request (see the definition
> of the ngx_http_subrequest function). So I think an assumption here is
> that a main request does not take a *lot* of subrequests, at least
> usually ;)
>
> If it is your user data in each subrequest that takes up too much
> room, you can explicitly free them by making an ngx_pfree call, as
> long as those chunks are big enough (nginx's memory pool will ignore
> small chunks and thus save some CPU cycles).
>
>> For example:
>>
>>  location /main {
>>      echo_reset_timer;
>>
>>      # subrequests in parallel
>>      echo_location_async /sub1;
>>      echo_location_async /sub2;
>>     ...
>>      echo_location_async /subn;
>>
>>      echo "took $echo_timer_elapsed sec for total.";
>>  }
>>
>> If there are many "echo_location_async /subX", echo subrequest will
>> occupy some memory. Lots of memory will not be free in time.
>> So,if  /main is requested a lot , the system memory will be run out.
>>
>
> Fortunately in almost all of *our* web apps, "n" in your example is
> quite small, usually 2 or 3, and 5 at most :)
>
>> Am I right? And what's your opinion?
>>
>
> I think in theory you can explicitly force each subrequest created by
> yourself to use a separate memory pool such that when a subrequest
> finalizes, it can free up its own pool as soon as possible. But I
> haven't done that myself and it's very likely some parts of the nginx
> core relies on the assumption that subrequest's memory chunks do have
> an identical lifetime as all of its parent requests. I'm not sure. And
> I myself, for example, have used this assumption in our ngx_lua module
> to capture subrequest response headers for the ngx.location.capture
> Lua interface ;)
>
> Cheers,
> -agentzh
>