High memory consumption when proxying to a Comet server

[Maxim Dounin]

Hello!

On Mon, Apr 12, 2010 at 05:17:10PM -0300, Rogério Schneider wrote:

> > As a workaround you may want to increase proxy_buffer_size to
> > reduce number of such allocation (and/or just drop connections
> > periodically).  Correct fix would be to make these buffers
> > reusable once sent to client.
> 
> Maxim, thanks for the review of the logs.
> 
> Increasing proxy_buffer_size wouldn't end up on malloc()ing more
> memory at the end of a period?

It's size of single proxy buffer used for passing data with 
proxy_buffering off.  It will be allocated once during request 
start.  And by allocating bigger buffer you reduce number of 
chunked encoding markers needed to pass big responses...

> I think on sending small chunks (say, 500 bytes) each minute,
> during hours, to the clients.

... but once your chunks are small enough to fit into your 
proxy_buffer_size - there is no difference.

> For each small chunk, I think, we will be malloc()ing one buffer, or
> am I wrong?

Not exactly.  It allocates 2 buffer structures (52 bytes each on 
i386) and 18 bytes to hold chunk-size and initial CRLF (final CRLF 
uses static string).  This gives something about 122 bytes per 
chunk, i.e. about 30 chunks per 4k malloc'ed from system (+- 
alignment and allocation overheads).

And here is the numbers in your logs:

$ egrep 'malloc:|recv:' zz2 | uniq -c
...
   1 2010/04/12 00:06:04 [debug] 32748#0: *55 malloc: 096E9878:4096
  31 2010/04/12 00:06:04 [debug] 32748#0: *55 recv: fd:16 1024 of 1024
   1 2010/04/12 00:06:04 [debug] 32748#0: *55 malloc: 096F5160:4096
  30 2010/04/12 00:06:04 [debug] 32748#0: *55 recv: fd:16 1024 of 1024
   1 2010/04/12 00:06:04 [debug] 32748#0: *55 malloc: 096F6168:4096
...

Numbers vary slightly due to process not being stable enough, but 
it doesn't really matter.

Maxim Dounin