excessive RAM consumption - memory leak

[Todd HG]

Igor Sysoev wrote:
> On Thu, Feb 21, 2008 at 08:17:01PM +0100, Todd HG wrote:
> 
>> Of course disabling gzip defeats the purpose of having gzip decrease 
>> MaxRequestsPerChild limit setting. There should be a way to set Nginx to 
>> kill and restart the worker process to free the RAM, and start again at 
>> zero for situations like mine.
> 
> Could you show what does
> 
> ps ax -o pid,ppid,%cpu,vsz,rss,wchan,command|egrep '(nginx|PID)'
> 
> show when nginx grows up ?
> 
> What OS do you use ?
> 
>> Without a solution I need to restart my server about every 24 hours, and 
>> this is a very robust server.
> 
> It's really strange. I run all my sites unattended. The workers are 
> restarted
> only for reconfiguration or online upgrade. For example, this nginx
> runs more than 2 days (static, SSI, gzipping, proxying) without any
> leaks (60-120M is stable state):
> 
>>ps ax -o pid,ppid,%cpu,vsz,lstart,wchan,command|egrep '(nginx|PID)'
>   PID  PPID %CPU   VSZ STARTED                      WCHAN  COMMAND
>  1645     1  0.0 16520 Mon Feb 18 02:16:37 2008     pause  nginx: master 
> proces
> 66458  1645 24.9 78984 Tue Feb 19 18:10:20 2008     kqread nginx: worker 
> proces
> 
> Now it handles 22000 simultaneous connections:
> 
>>fstat | grep 'nginx.*tcp' | awk '{print $3}' | sort | uniq -c
>    8 1645
> 22013 66458
> 
> 16 hours per day it handles 1000-2000 requests per seconds.
> This is 60-100 millions per day.

I agree this is a strange problem. I am running Nginx on Redhat 
Enterprise Server 4. I will post the output of the Niginx master and 
worker processes once the memory reaches it's max again. That should be 
in about 24 hours. Just for reference I am posting my exact current 
configuration below, but of course I've replaced some values to keep 
them private:

user  nobody;
worker_processes  2;

# The worker_processes and worker_connections from the event sections 
allows you to calculate maxclients value:
# max_clients = worker_processes * worker_connections

pid        /usr/local/nginx/logs/nginx.pid;

events {
        worker_connections  12000;
        use epoll;
}

http {
    include       /usr/local/nginx/conf/mime.types;
    default_type  application/octet-stream;

    expires       1M;
    add_header    Cache-Control  must-revalidate;
    add_header    Cache-Control  public;
    server_tokens off;

    client_header_timeout  3m;
    client_body_timeout    3m;
    send_timeout           3m;

    gzip off;

    sendfile       on;
    tcp_nopush     on;
    tcp_nodelay    on;

    keepalive_timeout  5 5;

    server_names_hash_bucket_size 128;

    server {
        listen       ip-address:9000;
        server_name  images.mydomain.com;

    error_page  404              http://www.mydomain.com/e404.php;

   location  / {
         root    /var/www/mydomain;
         expires 30d;
                 valid_referers blocked mydomain.com;
      if ($invalid_referer) {
            # return   404;
      rewrite  ^(.*)$  http://www.mydomain.com/;
      }
    deny ip-address;
                allow all;
       }

    }
}
-- 
Posted via http://www.ruby-forum.com/.