question about proxy_buffers

[Maxim Dounin]

Hello!

On Fri, Jun 12, 2009 at 12:30:58AM +0200, Jérôme Loyet wrote:

[...]

> >> I'll set up small buffers but a large number:
> >> proxy_buffers 2048 8k;
> >>
> >> Is it too much ?
> >
> > Each connection that uses proxy can use up to 16M of memory with
> > this setting (even after actual connection to backend has been
> > already closed).  With 1024 client connections this means up to
> > 16G of memory, probably a bit too many.
> 
> Do you mean that proxy_buffers are dedicated to each connection ?
> I thought it was shared by all connections.

Directive proxy_buffers specify maximum number and size of buffers 
that can be used by one (each) connection.  Not every connection 
uses so many buffers as they are allocated on demand, but every 
connection can do so.

Original documentation clearly states "... number and size of 
buffers for one connection ..." (in russian):

http://sysoev.ru/nginx/docs/http/ngx_http_proxy_module.html#proxy_buffers

Probably this was lost in translation (or was clarified later).  
Feel free to fix wiki.

Maxim Dounin

> 
> >
> > And keep in mind that:
> >
> > - Doing reconfiguration will spawn new worker processes and memory
> >  usage by nginx nearly doubles for a while.
> >
> > - You also need some memory for OS's disk cache.
> >
> > Some hints about seveal big vs. many small buffers:
> >
> > - Using bigger buffers may help a bit with CPU usage, but usually
> >  it's not a big deal.
> >
> > - Using bigger buffers means less disk seeks if reply won't fit
> >  into memory and will be buffered to disk.
> >
> > - On the other hand, small buffers allow better granularity (and
> >  less memory wasted for last partially filled buffer).
> >
> > Since there are different load patterns (and different hardware)
> > it's hard to recommend some particular setting, but the about
> > hints should allow you to tune it for your system appropriately.
> >
> > Maxim Dounin
> >
> >
> >>
> >> >
> >> > 2009/6/9 Jérôme Loyet <jerome at loyet.net>:
> >> >> Hi all,
> >> >>
> >> >> I'm not sure how proxy_buffers are used.
> >> >>
> >> >> 1- requests to the backend can use only one buffer. If the reponse is
> >> >> bigger than one buffer, some part of the response is buffered to disk.
> >> >> 2- requests to the backen can use multiple buffers. If the response is
> >> >> bigger than on buffer, nginx will use more buffers (if available of
> >> >> course).
> >> >>
> >> >> So depending on the answer, I would use :
> >> >> 1- big buffers so that 80% of the requests can feet in on buffer. And
> >> >> enough buffers so that 80% of the requests can be buffered
> >> >> 2- small buffers and a lots of buffers
> >> >>
> >> >> don't know what to use :) any clue ?
> >> >>
> >> >> thx
> >> >> ++ jerome
> >> >>
> >> >>
> >> >
> >> >
> >>
> >
> >
>