Normal memory usage for SSL terminating, reverse proxy nginx?

[onecrazymonkey]

It has been a difficult topic to research. The nginx instance is doing
nothing more than what the subject stated. It reverse proxies to a backend,
load balanced set of web app instances and terminates SSL for a large number
of unique domains each with their own SSL cert. Here's a `ps aux` of nginx
running after a clean start and zero (out of rotation) traffic.

root      20   0  676052 598224   1848 S   0.0 16.5   0:00.06 nginx         
                                  
nginx     20   0  675552 597204   1228 S   0.0 16.5   0:00.44 nginx         
                                  
nginx     20   0  675552 596612    636 S   0.0 16.5   0:00.36 nginx

Looking at that process list, nginx is using about 676mb of RAM for ~400
vhosts each with their own unique SSL cert for a unique domain. Here's an
example of a vhost server config. They're all generated based on the same
base template:

server {
  listen              443 ssl proxy_protocol;
  server_name         <uniquehostname> www.<uniquehostname>;
  access_log          /var/log/nginx/access_vhost_443.log accesslog;
  error_log           /var/log/nginx/error_vhost_443.log warn;
  real_ip_header      proxy_protocol;

  ssl                 on;
  ssl_certificate     /etc/nginx/ssl/<uniquehostname>/<uniquehostname>.crt;
  ssl_certificate_key /etc/nginx/ssl/<uniquehostname>/<uniquehostname>.key;

  ssl_stapling        on;
  ssl_stapling_verify on;

  resolver        internal-dns.vpc  valid=60s;

  set             $internal "upstream-load-balancer.vpc";
  location / {
    if ($denied) {
      return 444;
    }
    proxy_pass          http://$internal;
  }
}

Now, this wouldn't be all that bad. 1.69mb of memory per vhost isn't
horrible, high, but not unsustainable. However, if I do `nginx -s reload` or
restart via systemd service... 

root      20   0 1370188 1.176g   3240 S   0.0 33.4   0:14.98 nginx         
                                  
nginx     20   0 1370192 1.175g   2584 S   0.3 33.4   2:39.95 nginx         
                                  
nginx     20   0 1370192 1.175g   2584 S   1.7 33.4   2:28.42 nginx

It doubles the memory consumption! It never goes up or down drastically
again. It's as if it duplicates and never frees or releases unless you do a
restart.

This was tested on a handful of AWS EC2 instance types using vanilla Centos7
and both nginx 1.6.3 (stable in centos repos) and nginx 1.10.0 (nginx.org
repo).

In summary, my questions are thus:
- Is it normal for nginx to use ~1.7mb per SSL vhost?
- Is there a way to reduce that memory usage?

- Am I the only one that experiences the doubling of nginx memory usage
after a nginx reload?
- Is that a bug?

Thanks!

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,267189,267189#msg-267189