nginx not responding to some SYN packets

[Radek Burkat]

Have a machine running the latest devel version nginx-0.7.33 (tried 0.6.35
with same results) for serving small (less than 10K images) and am seeing on
tcpdump that some SYN packets are not responded to right away.The browser
does retransmit these image requests every second and on the 2nd or 3rd
resent SYN, I finally start seeing and ACK, and the images load.
It is very indeterministic as to when it happens and can only reproduce it
some of the time.  When it does occur the outcome is a page with some images
loaded and others (whose SYN packets are not ACKs) are not loaded.....a few
seconds later they load.
Typically the system has ~2000 active connections, most in keep alive.  The
load is around 100-200 req/sec.

I have tries all sorts of settings and configurations suggested in the
maillist but I still dont have the solution for this issue.  from 1 to 4
workers, changing the connection counts, different even handlers, kernel
buffers, etc.
It just seems so anecdotal to just change a bunch of settings without being
able to what is happening internally.
I'd like to be able to debug a little deeper to find out what is happening
to these packets.

How would I go about debugging what is the cause of it.  Is it the interface
driver, kernel, or nginx?  What kind of tools and debugging options can I
try next?

Thanks, Radek


System Details
model name : Intel(R) Xeon(R) CPU X3210 @ 2.13GHz
Linux n06 2.6.18-92.1.22.el5 #1 SMP Tue Dec 16 12:03:43 EST 2008 i686 i686
i386 GNU/Linux
eth0
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Current message level: 0x000000ff (255)
Link detected: yes
driver: tg3
version: 3.86
firmware-version: 5721-v3.61, ASFIPMI v6.21
bus-info: 0000:05:00.0

avg-cpu: %user %nice %system %iowait %steal %idle
0.10 0.00 0.20 2.43 0.00 97.27
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm
%util
sda 0.00 0.00 27.40 0.00 443.20 0.00 16.18 0.10 3.50 3.32 9.10
no overruns or errors on interface.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://nginx.org/pipermail/nginx/attachments/20090203/e116c6a6/attachment.html>