Occupancy of SSL connections?
nginx-forum at nginx.us
Thu Nov 12 21:11:08 MSK 2009
Thanks for your response.
Maxim Dounin Wrote:
> Are you sure you measure allocated memory?
Yes, I'm looking at VIRT in top rather than RES.
> 16k connections
> openssl (at least 0.9.8, but you claim numbers are
> the same...)
> should allocate about 1.2G (~80k per connection)
> on ssl handshake.
That's odd, because I don't see that. I'm using some slightly hacky code derived from some openssl examples, and here's what it looks like:
/* Build our SSL context*/
for (loop = 0; loop < 20000; loop++)
printf("Loop %d\n", loop);
/* Connect the TCP socket*/
/* Connect the SSL socket */
berr_exit("SSL connect error");
/* Now make our HTTP request */
/* Shutdown the socket */
The difference in occupancy comes from whether or not I comment out the http_request code. Would you expect the openssl calls above to have incurred the server-side occupancy hit?
> Numbers you see in resident memory (RES in top)
> may be quite
> different, but it just means that relevant memory
> wasn't yet
> touched. And grow on request processing is
Yes, these are easily confused but I'm familiar with the problem.
> > - 16K connections with the outstanding GET
> request now takes ~1GB (down from estimated 1.6GB
> > So there's still a massive occupancy cost of
> having an outstanding GET request in the HTTPS
> case, which is surprising. Looks like I'd better
> roll my sleeves up and dig into the code.
> For outstanding GET requests there is 16k ssl
> buffer allocated by
> nginx on first output (see
> src/event/ngx_event_openssl.c) and
> freed when connection goes to keepalive state.
> This gives 256M
> for 16k connections.
Ok, that's useful. I'll look at the code, but do you think it's feasible either to
- shrink this (how small could it be? what would happen if it was too small?), or
- free it in between getting a request in and sending the response?
That wouldn't account for all the difference I see, but 256M would go some way to reducing the occupancy hit.
> Also there is a bunch of memory in nginx (output
> buffers, large
> client header buffers, gzip buffers, various
> request-related data
> and so on) which are freed when connection goes to
> state. Depending on your settings, particular
> request and already
> sent response this may contribute various numbers
> to keepalive vs.
> outstanding request cases. This should be almost
> the same in
> non-https case though, at least with sendfile not
Yes, that's what I would expect, and the occupancy in the non-https case is pretty good. So I'm hoping not to have to touch that, and that this is something https-specific.
Posted at Nginx Forum: http://forum.nginx.org/read.php?2,21531,22405#msg-22405
More information about the nginx