excessive RAM consumption - memory leak

[Igor Sysoev]

On Thu, Feb 21, 2008 at 08:17:01PM +0100, Todd HG wrote:

> >> Igor Sysoev wrote:
> >> > On Wed, Feb 20, 2008 at 11:47:29PM +0100,
> >> 
> >> Is there anywhere I could read more about how Nginx uses 
> >> connection_pool_size, stores connections, if it is in a cache in RAM or 
> >> hard drive, and what else might be stored in RAM by Nginx?
> > 
> > All that you need is to disable gzipping images. It's enough.
> > Other default settings do not allow workers to grow up.
> 
> Of course disabling gzip defeats the purpose of having gzip decrease 
> bandwidth and increase site speed for readers. Right now I only have 
> gzip handling a few million js and css files a day, in addition to tens 
> of millions of images which are not gzipped, but the RAM usage just 
> grows until it is completely consumed. By setting the gzip compression 
> level to 1 the RAM consumption grows more slowly, but eventually eats 
> all the RAM.
> 
> It appears what might be needed is a setting to allow the total number 
> of connections for gzip to be set before Nginx automatically kills and 
> restarts a worker. This would be similar to the Apache 
> MaxRequestsPerChild limit setting. There should be a way to set Nginx to 
> kill and restart the worker process to free the RAM, and start again at 
> zero for situations like mine.

Could you show what does

ps ax -o pid,ppid,%cpu,vsz,rss,wchan,command|egrep '(nginx|PID)'

show when nginx grows up ?

What OS do you use ?

> Without a solution I need to restart my server about every 24 hours, and 
> this is a very robust server.

It's really strange. I run all my sites unattended. The workers are restarted
only for reconfiguration or online upgrade. For example, this nginx
runs more than 2 days (static, SSI, gzipping, proxying) without any
leaks (60-120M is stable state):

>ps ax -o pid,ppid,%cpu,vsz,lstart,wchan,command|egrep '(nginx|PID)'
  PID  PPID %CPU   VSZ STARTED                      WCHAN  COMMAND
 1645     1  0.0 16520 Mon Feb 18 02:16:37 2008     pause  nginx: master proces
66458  1645 24.9 78984 Tue Feb 19 18:10:20 2008     kqread nginx: worker proces

Now it handles 22000 simultaneous connections:

>fstat | grep 'nginx.*tcp' | awk '{print $3}' | sort | uniq -c
   8 1645
22013 66458

16 hours per day it handles 1000-2000 requests per seconds.
This is 60-100 millions per day.


-- 
Igor Sysoev
http://sysoev.ru/en/