<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-AU" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We have been doing some testing with Nginx as a reverse proxy. We have been comparing it to a number of solutions which it easily beats, like IIS and Apache with mod_proxy etc. However, as an experiment we have been comparing it to an adapted
NIO server written in Java. This seems to be out performing Nginx in the reverse proxy role by a factor of 3 times. We are convinced our configuration is wrong. Both run on the same box (at different times) with the same sysctl settings (see below). We also
saw some spikes, up to 3 seconds per request at times, and some at 10 over a 1 million request test of 1000 concurrent clients.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We are using a fairly straight forward configuration for Nginx. Since we have two processors on the box we tried worker_processes of 4 with worker_connections of 6000, then we tried worker_processes of 40 with worker_connections of 5000.
No change. We need to be able to support responsive Ajax requests with strategies like HTTP streaming and long polling in our setup.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Any ideas what we can do to boost our throughput and latency?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[root@dc1dmzngx02 apachebench]# uname -a<o:p></o:p></p>
<p class="MsoNormal">Linux dc1dmzngx02 2.6.32-220.13.1.el6.x86_64 #1 SMP Tue Apr 17 23:56:34 BST 2012 x86_64 x86_64 x86_64 GNU/Linux<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[root@dc1dmzngx02 apachebench]# cat /etc/sysctl.conf<o:p></o:p></p>
<p class="MsoNormal"># Kernel sysctl configuration file for Red Hat Linux<o:p></o:p></p>
<p class="MsoNormal">#<o:p></o:p></p>
<p class="MsoNormal"># For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and<o:p></o:p></p>
<p class="MsoNormal"># sysctl.conf(5) for more details.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># High perf config<o:p></o:p></p>
<p class="MsoNormal">net.core.somaxconn = 12048<o:p></o:p></p>
<p class="MsoNormal">net.core.rmem_default = 262144<o:p></o:p></p>
<p class="MsoNormal">net.core.wmem_default = 262144<o:p></o:p></p>
<p class="MsoNormal">net.core.rmem_max = 16777216<o:p></o:p></p>
<p class="MsoNormal">net.core.wmem_max = 16777216<o:p></o:p></p>
<p class="MsoNormal">net.ipv4.tcp_rmem = 4096 4096 16777216<o:p></o:p></p>
<p class="MsoNormal">net.ipv4.tcp_wmem = 4096 4096 16777216<o:p></o:p></p>
<p class="MsoNormal">net.ipv4.tcp_mem = 786432 2097152 3145728<o:p></o:p></p>
<p class="MsoNormal">net.ipv4.tcp_max_syn_backlog = 16384<o:p></o:p></p>
<p class="MsoNormal">net.core.netdev_max_backlog = 20000<o:p></o:p></p>
<p class="MsoNormal">net.ipv4.tcp_fin_timeout = 15<o:p></o:p></p>
<p class="MsoNormal">net.ipv4.tcp_max_syn_backlog = 16384<o:p></o:p></p>
<p class="MsoNormal">net.ipv4.tcp_tw_reuse = 1<o:p></o:p></p>
<p class="MsoNormal">net.ipv4.tcp_tw_recycle = 1<o:p></o:p></p>
<p class="MsoNormal">net.ipv4.tcp_max_orphans = 131072<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># Controls IP packet forwarding<o:p></o:p></p>
<p class="MsoNormal">net.ipv4.ip_forward = 0<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># Controls source route verification<o:p></o:p></p>
<p class="MsoNormal">net.ipv4.conf.default.rp_filter = 1<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># Do not accept source routing<o:p></o:p></p>
<p class="MsoNormal">net.ipv4.conf.default.accept_source_route = 0<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># Controls the System Request debugging functionality of the kernel<o:p></o:p></p>
<p class="MsoNormal">kernel.sysrq = 0<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># Controls whether core dumps will append the PID to the core filename.<o:p></o:p></p>
<p class="MsoNormal"># Useful for debugging multi-threaded applications.<o:p></o:p></p>
<p class="MsoNormal">kernel.core_uses_pid = 1<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># Controls the use of TCP syncookies<o:p></o:p></p>
<p class="MsoNormal">net.ipv4.tcp_syncookies = 1<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># Disable netfilter on bridges.<o:p></o:p></p>
<p class="MsoNormal">net.bridge.bridge-nf-call-ip6tables = 0<o:p></o:p></p>
<p class="MsoNormal">net.bridge.bridge-nf-call-iptables = 0<o:p></o:p></p>
<p class="MsoNormal">net.bridge.bridge-nf-call-arptables = 0<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># Controls the maximum size of a message, in bytes<o:p></o:p></p>
<p class="MsoNormal">kernel.msgmnb = 65536<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># Controls the default maxmimum size of a mesage queue<o:p></o:p></p>
<p class="MsoNormal">kernel.msgmax = 65536<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># Controls the maximum shared segment size, in bytes<o:p></o:p></p>
<p class="MsoNormal">kernel.shmmax = 68719476736<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># Controls the maximum number of shared memory segments, in pages<o:p></o:p></p>
<p class="MsoNormal">kernel.shmall = 4294967296<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
<p class="MsoNormal">Niall<o:p></o:p></p>
</div>
</body>
</html>