limiting upstream backend connections

[Jeroen Ooms]

I am hosting a CPU intensive service and use nginx as a reverse proxy
cache / load balancer to my backend(s). Basic config looks like:

upstream myservice {
    server 127.0.0.1:80;
    server 123.123.123.123:80;
}

limit_req_zone $binary_remote_addr  zone=softlimit:10m   rate=1r/s;

location /R {
    proxy_pass  http://myservice/R;
    proxy_cache mycache;
    proxy_cache_methods POST;
    proxy_cache_key "$request_uri|$request_body";
    limit_req zone=softlimit  burst=10;
}

So I use limit_req to make sure that no single user can do more than 1
req per sec to my service.

However, actually more important is that the upstream backend servers
are protected against overload. Some of the requests might take 30 or
60 seconds to return. I want to limit the total number of active
concurrent connections to the upstream back-end at, say, 10. Requests
that hit the cache and never make it to the backend should not be
counted towards this. Only active backend connections.

I had a look at limit_conn, but it looks like this limits the client
connections rather than the back-end connections, is that correct?

Thanks for advice,

Jeroen