Memory Management ( > 25GB memory usage)

[Maxim Dounin]

Hello!

On Mon, Jun 03, 2013 at 08:57:21AM -0400, Belly wrote:

> Hello nginx!
> 
> I have one worker-process, which uses over 25GB memory (and doesn't stop to
> do that).
> My configuration is... let's say special:
> 
> So there is nginx, which proxies all requests to the PHP backend and the PHP
> backend sends a large request back to nginx. I set the fastcgi_buffers very
> enormous huge to avoid nginx creating temporary files on my disk - which
> would result in high CPU load.
> 
> Here is my configuration: (reduced to the problem)
> 
> worker_processes 1;
> worker_rlimit_nofile 80000;
> worker_priority -20;
> 
> events {
>         worker_connections 10240;
>         multi_accept on;
> }
> # ... 
>         # fastcgi settings
>         fastcgi_buffers 20480 1k;

Just a side note: each buffer structure takes about 100 bytes of 
memory on 64-bit platforms, and using 1k buffers results in about 
10% overhead just because of this.

>         fastcgi_connect_timeout 30;
>         fastcgi_read_timeout 30;
>         fastcgi_send_timeout 30;
>         fastcgi_keep_conn on;
>         upstream php-backend {
>                 server 127.0.0.1:9000;
>                 keepalive 10000;
>         }
> 
> 
> As you can see the buffers are extreme large, to avoid disk buffering. The
> problem is that nginx doesn't free the buffers. It just eats and eats. I
> know it's my fault and not nginx' fault. What am I doing wrong?
> 
> The response of my php backend could be from 1k to 300mb.

With your settings each connection can allocate up to 20M of 
buffers.  That is, 1500k connections are enough to allocate 25G of 
memory.  So the basic question is - how many connections are open?

With pessimistic assumption of 10k connections as per 
worker_connections, you configuration will result in more than 
200G memory used.

> What is the best setting for my situation?

I would recommend using "fastcgi_max_temp_file_size 0;" if you 
want to disable disk buffering (see [1]), and configuring some 
reasonable number of reasonably sized fastcgi_buffers.  I would 
recommend starting tuning with something like 32 x 64k buffers.

[1] http://nginx.org/r/fastcgi_max_temp_file_size

-- 
Maxim Dounin
http://nginx.org/en/donation.html