High memory consumption when proxying to a Comet server
mdounin at mdounin.ru
Mon Apr 12 05:57:26 MSD 2010
On Sun, Apr 11, 2010 at 10:08:26PM -0300, Rogério Schneider wrote:
> Maxim, thanks.
> I have tried to reduce memory usage by reducing the default values for:
> from "8 4k" to "2 4k".
> With this one would expect the memory usage to be 4 times lower, but
> that was not what I saw.
> I saw the memory usage stay in the same growth step.
No, changing 8 buffers to 2 buffers doesn't mean memory usage will
be 4 times less. 1) this isn't the only allocations made, and 2)
not all requests allocate all proxy buffers, only ones with big
responses and slow clients. But see below.
> May I mention that for the fact I am using Comet, I turned proxy buffering off?
> proxy_buffering off;
> I think this is important to note and it can explain why tuning the
> proxy buffers down did not lower the memory consumption.
With proxy_buffering off; setting proxy_buffers indeed doesn't
matter at all.
Next candidates is gzip_buffers (only matters if you are using
gzip, up to 32 * 4k by default; and keep in mind that about 200k
or so will be allocated per request anyway with gzip enabled).
Remaining ones are output_buffers and fastcgi_buffers, these
shouldn't matter in your case anyway.
> Could I debug Nginx procs to see where the memory is being allocated
> and for what purpose?
> How could I debug it? Do you have some guideline?
You may switch on debug log, it will be possible to trace big
allocations there. See here:
p.s. Please do not top-post. Thank you.
More information about the nginx