<html><head></head><body><br>
A test with debug logging is scheduled for upcoming Monday. I hope to be able to deliver the logs by then. <br>
<br>
Thanks, <br>
<br>
B.<br>
<br><br><div class="gmail_quote">On November 24, 2015 4:32:11 PM GMT+01:00, "Valentin V. Bartenev" <vbart@nginx.com> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<pre class="k9mail">On Tuesday 24 November 2015 15:10:12 Bart Warmerdam wrote:<br /><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"> <br /> Hello,<br /> <br /> On a system with a load of about 500-600 URI/sec I see some unexpected<br /> behaviour when using "aio threads" option in the configuration.<br /> <br /> System setup:<br /> The system runs on RHEL6.6 with 3 workers running nginx 1.9.6 with<br /> thread support. Content is cached and populated by a proxied-upstream.<br /> The cache location is a tmpfs file system with more then enough space<br /> at all times. Proxy buffer size 8k. The output buffer is default (no<br /> config item, so 2 32k). Keepalive timeout 75s. Sendfile is enabled.<br /> <br /> Seen behaviour:<br /> On the WAF in front of this system I see occasional hangs on resources<br /> (mainly larger files like js, jpeg, ..). Seen in the WAF log is that<br /> this WAF waits for the transfer to be
completed until nginx closes the<br /> connection at the keepalive time of 75s. In the nginx access.log I see<br /> the entry served from cache (upstream server '-') with the correct<br /> content length. In the tcp dump I see the response of this call to<br /> contain a content-length header with the correct length, a server time<br /> header over 1 minute older then the tcpdump timestamp (all servers are<br /> ntp-connected). The served jpeg is half-way in its cache lifetime at<br /> that time and there are previous served entries from cache without<br /> incomplete transfers. In the tcp dump the jpeg file starts to differ<br /> from the original after 32168 bytes and misses 8192 bytes after which<br /> the remaining content is served (which is identical to original). From<br /> the tcpdump I can extract the file which is missing 8192 bytes.<br /> <br /> We have also a dump when during the proxied call this same behaviour<br /> was seen. The upstream call is started to get a jpeg
from the origin.<br /> After a few packets the data is sent to the WAF. The complete upstream<br /> file is retrieved (can be validated in the tcpdump that the jpeg is<br /> complete and correctly retrieved), but not all the data is sent to the<br /> listening socket to the WAF.<br /> <br /> <br /> If I change the setup to "aio on" or "aio off" this behaviour is not<br /> seen. This is the only change in the configuration between the tests.<br /> It looks like this behaviour only affects bigger files. I have not seen<br /> this effect on small files or proxied responses.<br /> <br /> <br /> Does anyone have the same experience with this option. And what is the<br /> best way to proceed in tracing this?<br /> <br /></blockquote>[..]<br /><br />Could you provide the debug log?<br /><a href="http://nginx.org/en/docs/debugging_log.html">http://nginx.org/en/docs/debugging_log.html</a><br /><br /> wbr, Valentin V. Bartenev<br /><br /><hr /><br />nginx-devel mailing list<br
/>nginx-devel@nginx.org<br /><a href="http://mailman.nginx.org/mailman/listinfo/nginx-devel">http://mailman.nginx.org/mailman/listinfo/nginx-devel</a><br /></pre></blockquote></div><br>
-- <br>
Sent from my Android device with K-9 Mail. Please excuse my brevity.</body></html>