High memory consumption when proxying to a Comet server

[Rogério Schneider]

Maxim, thanks.

I have tried to reduce memory usage by reducing the default values for:
  proxy_buffers
from "8 4k" to "2 4k".

With this one would expect the memory usage to be 4 times lower, but
that was not what I saw.
I saw the memory usage stay in the same growth step.

May I mention that for the fact I am using Comet, I turned proxy buffering off?
  proxy_buffering         off;
I think this is important to note and it can explain why tuning the
proxy buffers down did not lower the memory consumption.

Could I debug Nginx procs to see where the memory is being allocated
and for what purpose?
How could I debug it? Do you have some guideline?

Many thanks.

Regards,
Rogério Schneider

On Sun, Apr 11, 2010 at 9:49 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> Hello!
>
> On Sun, Apr 11, 2010 at 09:16:33PM -0300, Rogério Schneider wrote:
>
>> When proxying to a Comet server, I was able to notice that Nginx is
>> eating and not releasing a massive amount of memory.
>>
>> As we all know, Comet servers can have this "streaming" mode, where we
>> receive a chunked HTTP request that never ends.
>>
>> Is there some problem that avoids Nginx to release memory when in
>> proxy mode, before the connection ends at all?
>
> [...]
>
> nginx does memory allocations on per-request basis, and frees
> memory when request ends.  Though after sending response headers
> possible allocations are limited by various *_buffers settings.
>
> For proxied requests you may want to make sure your proxy_buffers
> aren't set too high, see here:
>
> http://wiki.nginx.org/NginxHttpProxyModule#proxy_buffers
>
> Maxim Dounin
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://nginx.org/mailman/listinfo/nginx
>


-- 
Rogério Schneider

+55 (21) 9856 7847

MSN: stockrt at hotmail.com
GTalk: stockrt at gmail.com
Skype: stockrt
http://stockrt.github.com