Strange $upstream_response_time latency spikes with reverse proxy

Jay Oster jay at
Tue Mar 19 22:42:27 UTC 2013

Hi Andrei!

On Tue, Mar 19, 2013 at 2:49 AM, Andrei Belov <defan at> wrote:

> Hello Jay,
> If I understand you right, issue can be repeated in the following cases:
> 1) client and server are on different EC2 instances, public IPs are used;
> 2) client and server are on different EC2 instances, private IPs are used;
> 3) client and server are on a single EC2 instance, public IP is used.
> And there are no problems when:
> 1) client and server are on a single EC2 instance, either loopback or
> private IP is used.
> Please correct me if I'm wrong.

If by "client" you mean nginx, and by "server" you mean our upstream HTTP
service ... That is exactly correct. You could also throw in another
permutation by changing where ApacheBench is run, but it doesn't change the
occurrence of dropped packets; only increases average latency when AB and
nginx are on separate EC2 instances.

> What about EC2 security group - do the client and the server use the same
> group?
> How many rules are present in this group? Have you tried to either decrease
> a number of rules used, or create "pass any to any" simple configuration?

That's a great point! We have been struggling with the number of firewall
rules as a separate matter, in fact. There may be some relation here. Thank
you for reminding me.

> And just to clarify the things - under "external IP address" do you mean
> EC2
> instance's public IP, or maybe Elastic IP?

I'm talking about the instance public IPs. Elastic IPs are only used for
client access to nginx. And specifically only for managing DNS. Between
nginx and upstream servers, the public IPs are used.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the nginx mailing list