Fwd: Automatic pooling of upstream keepalive connections (patch proposal)

[Greg Vishnepolsky]

Hello,
Please consider the following problem and solution proposal.

I'm using nginx in a transparent proxy configuration, sort of like this:
...
    location / {
        ...
        proxy_set_header Host $host;
        proxy_pass https://$host;
    }

This works OK, except for a pretty bad performance issue, where (in this
case) nginx doesn't do the following:
    1) SSL session reuse
    2) Caching of upstream keepalive connections

If an explicit upstream configuration is defined per proxied host, like
this:
    upstream some.proxied.host.com {
        server some.domain.com:443;
        keepalive 10;
    }
Only then will SSL session reuse and upstream keepalive work as expected.

Unfortunately, this "trick" can't be used for the aforementioned
"transparent proxy" configuration, since I don't know in advance which
hosts are going to be proxied.
In order to allow for these very important optimizations when proxy_pass is
used with "unknown" hosts, I believe a patch is needed. An additional
configuration such as this should be considered:
    server {
        ...
        proxy_upstream_default_keepalive on;
        proxy_upstream_default_keepalive_max_hosts 10;
        proxy_upstream_default_keepalive_max_connections 100;
        ....
    }
These are to be configurations of the ngx_http_proxy_module. In the
proposed patch, upstreams are created dynamically for new hosts (up to
"max_hosts" at the same time). For each upstream the number of simultaneous
keepalive'd connections would be "max_connections".
The following patch does not currently solve the SSL session reuse problem,
but it does handle the keepalive pooling problem. Here it is on github:
https://gist.github.com/gregvish/6511822/raw/1f7de28c8ac7bf133376eccee9bb4fc65a8a2917/default_keepalive_patch
It was taken against the 1.4.2 version source code.

Most of the new code was added to the "ngx_http_upstream_keepalive_module".
This new code is called by the function
"ngx_http_upstream_resolve_handler", after the call to
"ngx_http_upstream_create_round_robin_peer". The new function I added,
"ngx_http_upstream_default_keepalive_adapt_peer" converts the peer that was
created by "create_round_robin_peer" into a "keepalive'd" peer (if the
"conversion" fails, the peer remains a regular round robin one, and
continues to work).

Inside the new code in the "keepalive_module", new hosts are added to a
data structure, where uniqueness is established by a tuple of (host, port).
If all the allotted entries are taken, a "garbage collection" occurs.
During the collection, cache entries are removed where no keepalive'd
connections are currently established.

I did some testing, and it works as expected. Didn't see memory leaks or
unexpected behavior.

As for the SSL session reuse in this case,
"ngx_http_upstream_create_round_robin_peer" allocates the
"ngx_http_upstream_rr_peers_t" struct (which holds the SSL session) in the
request pool (r->pool), and not somewhere "persistent". A similar patch can
be added there, so there is a cache of these structures as well. This
should enable the session reuse in this case too. I didn't write this patch
yet, since I believe the keepalive'd connections have a greater performance
impact.

P.S.
I'd love to receive any feedback that you are willing to give me. Perhaps
you can think of a far better way to do this, or improve the code in some
way.
Also I'd like to know if you'd consider this patch to be added to an
official release, and what needs to be changed so this can happen.

Thanks,
Greg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20130911/14e39cc5/attachment-0001.html>