Nginx 1.12.1 Memory Consumption

[Maxim Dounin]

Hello!

On Wed, Mar 14, 2018 at 05:05:42PM +0000, Matthew Smith wrote:

> I have encountered what I consider to be an interesting behavior. We have
> Nginx 1.12.1 configured to do SSL termination as well as reverse proxy.
> Whenever there is a traffic spike (300 req/s > 1000 req/s, 3k active
> connections > 20k active connections), there is a corresponding spike in
> Nginx memory consumption. In this case 500M > 8G across 10 worker
> processes. What is interesting is that Nginx never seems to release this
> memory after the traffic returns to normal. Is this expected? What is Nginx
> using this memory for? Is there a configuration that will rotate the
> workers based on some metric in order to return memory to the system?

All memory allocated for request handling nginx returns to 
the allocator once a request is closed.  If you see memory not 
actually released to the system, this may be one of the following:

- System allocator fails to return memory to the system.  Tuning 
  system allocator might help here.

- Given that you are using nginx for SSL termination, this may be 
  an OpenSSL [mis]feature called "buf-freelists".  It implies caching 
  of up to 2 megabytes of allocated buffers per SSL context, and 
  this may be a problem if there are multiple server{} blocks with 
  SSL enabled.  Fix is to recompile OpenSSL with the 
  "no-buf-freelists" option, or upgrade to OpenSSL 1.1.x where 
  this feature is disabled.

-- 
Maxim Dounin
http://mdounin.ru/