question about proxy_buffers

[Jérôme Loyet]

2009/6/12 Maxim Dounin <mdounin at mdounin.ru>:
> Hello!
>
> On Fri, Jun 12, 2009 at 12:30:58AM +0200, Jérôme Loyet wrote:
>
> [...]
>
>> >> I'll set up small buffers but a large number:
>> >> proxy_buffers 2048 8k;
>> >>
>> >> Is it too much ?
>> >
>> > Each connection that uses proxy can use up to 16M of memory with
>> > this setting (even after actual connection to backend has been
>> > already closed).  With 1024 client connections this means up to
>> > 16G of memory, probably a bit too many.
>>
>> Do you mean that proxy_buffers are dedicated to each connection ?
>> I thought it was shared by all connections.
>
> Directive proxy_buffers specify maximum number and size of buffers
> that can be used by one (each) connection.  Not every connection
> uses so many buffers as they are allocated on demand, but every
> connection can do so.
>
> Original documentation clearly states "... number and size of
> buffers for one connection ..." (in russian):
>
> http://sysoev.ru/nginx/docs/http/ngx_http_proxy_module.html#proxy_buffers
>
> Probably this was lost in translation (or was clarified later).
> Feel free to fix wiki.
>
OK Thanks you very much. In this case 16Mo (2048 * 8k is way way too much)

I'll analyse my logs to know what is the distribution of my requests
size so that I can do the better compromise. Thx you guys
>
>>
>> >
>> > And keep in mind that:
>> >
>> > - Doing reconfiguration will spawn new worker processes and memory
>> >  usage by nginx nearly doubles for a while.
>> >
>> > - You also need some memory for OS's disk cache.
>> >
>> > Some hints about seveal big vs. many small buffers:
>> >
>> > - Using bigger buffers may help a bit with CPU usage, but usually
>> >  it's not a big deal.
>> >
>> > - Using bigger buffers means less disk seeks if reply won't fit
>> >  into memory and will be buffered to disk.
>> >
>> > - On the other hand, small buffers allow better granularity (and
>> >  less memory wasted for last partially filled buffer).
>> >
>> > Since there are different load patterns (and different hardware)
>> > it's hard to recommend some particular setting, but the about
>> > hints should allow you to tune it for your system appropriately.
>> >
>> > Maxim Dounin
>> >
>> >
>> >>
>> >> >
>> >> > 2009/6/9 Jérôme Loyet <jerome at loyet.net>:
>> >> >> Hi all,
>> >> >>
>> >> >> I'm not sure how proxy_buffers are used.
>> >> >>
>> >> >> 1- requests to the backend can use only one buffer. If the reponse is
>> >> >> bigger than one buffer, some part of the response is buffered to disk.
>> >> >> 2- requests to the backen can use multiple buffers. If the response is
>> >> >> bigger than on buffer, nginx will use more buffers (if available of
>> >> >> course).
>> >> >>
>> >> >> So depending on the answer, I would use :
>> >> >> 1- big buffers so that 80% of the requests can feet in on buffer. And
>> >> >> enough buffers so that 80% of the requests can be buffered
>> >> >> 2- small buffers and a lots of buffers
>> >> >>
>> >> >> don't know what to use :) any clue ?
>> >> >>
>> >> >> thx
>> >> >> ++ jerome
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >>
>> >
>> >
>>
>
>