SSL Memory Usage and Fragmentation
Ben Maurer
bmaurer at andrew.cmu.edu
Wed Dec 26 20:34:39 MSK 2007
Hi,
On a production server, I found that nginx appears to leak when using
ssl. With some investigation, it seems that this is actually memory
fragmentation due to the session cache. I made a very simple
configuration for the server:
daemon off;
master_process off;
pid /tmp/x.pid;
error_log /tmp/x.log;
events { use epoll; }
http {
client_body_temp_path /tmp;
proxy_temp_path /tmp;
fastcgi_temp_path /tmp;
access_log /tmp/access.log;
server {
listen localhost:8666;
ssl on;
ssl_certificate /home/bmaurer/x.pem;
ssl_certificate_key /home/bmaurer/x.pem;
root /tmp;
}
}
Then I did a benchmark with the following command:
ab -c500 -n20000 https://localhost:8666/
After doing this, the server uses ~ 30 MB of RSS. Running it once more,
it uses ~ 40 MB of RSS. Valgrind claims that there are no "leaks", it
seems that there's just a really bad case of memory fragmentation.
I tried applying this to the SSL configuration:
ssl_session_cache builtin:2;
Doing so resulted in the memory use of the nginx server staying
relatively low (it appears the memory was reclaimed from the OS after it
was used).
It seems like it might be worth switching to something like the shared
memory cache by default. Keeping the long-lived session cache in a
different pool of memory avoids the risk of large amounts of memory
getting pinned in.
One other thing I noticed while investigating this stuff was that nginx
keeps a 16 KB buffer for each SSL connection for the entire duration of
the connection. I've attached a patch that keeps this buffer alive only
while there's a pending write. Sadly, there are some relatively large
buffers internal to openssl as well, which means the overhead for SSL
keepalive connections is pretty high.
- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nginx-fewer-ssl-buffers.patch
Type: text/x-patch
Size: 1481 bytes
Desc: not available
URL: <http://nginx.org/pipermail/nginx/attachments/20071226/19a6d57b/attachment.bin>
More information about the nginx
mailing list