Nginx setting up >25.000 concurrent connections per second

[MagicBear]

increase the worker number to 12, and I get such results, I think that
may be the maximum.
cat logger | sed  's/||/ /g' | awk '{print $3}'| sed 's/\.[0-9]\+//g'
| sort | uniq -c
  58423 1317950330
  85703 1317950331
 116036 1317950332
 115995 1317950333
 116070 1317950334
 120604 1317950335
 119080 1317950336
 118695 1317950337
 118231 1317950338
 114383 1317950339
 104594 1317950340
 103047 1317950341
 105614 1317950342
 100386 1317950343
  90679 1317950344
  94728 1317950345
 100741 1317950346
 100206 1317950347
 100959 1317950348
  99431 1317950349
 104943 1317950350
 104868 1317950351
 100532 1317950352
 101507 1317950353
 106315 1317950354
 110642 1317950355
 108740 1317950356
 105454 1317950357
 104623 1317950358
 101233 1317950359


2011/10/7 MagicBear <magicbearmo at gmail.com>:
> and here is my sysctl
>
> # Avoid a smurf attack
> net.ipv4.icmp_echo_ignore_broadcasts = 1
>
> # Turn on protection for bad icmp error messages
> net.ipv4.icmp_ignore_bogus_error_responses = 1
>
> # Turn on and log spoofed, source routed, and redirect packets
> #net.ipv4.conf.all.log_martians = 1
> #net.ipv4.conf.default.log_martians = 1
>
> # No source routed packets here
> net.ipv4.conf.all.accept_source_route = 0
> net.ipv4.conf.default.accept_source_route = 0
>
> # Turn on reverse path filtering
> net.ipv4.conf.all.rp_filter = 1
> net.ipv4.conf.default.rp_filter = 1
>
> # Make sure no one can alter the routing tables
> net.ipv4.conf.all.accept_redirects = 0
> net.ipv4.conf.default.accept_redirects = 0
> net.ipv4.conf.all.secure_redirects = 0
> net.ipv4.conf.default.secure_redirects = 0
>
> # Don't act as a router
> net.ipv4.ip_forward = 0
> net.ipv4.conf.all.send_redirects = 0
> net.ipv4.conf.default.send_redirects = 0
>
> net.core.somaxconn=32768
>
> net.ipv4.ip_local_port_range=4096 65535
>
> net.core.netdev_max_backlog = 32768
> net.ipv4.tcp_max_syn_backlog = 32768
> net.ipv4.tcp_max_orphans = 262144
>
> #for GigaEthernet
> net.core.rmem_max = 16777216
> net.core.wmem_max = 16777216
> net.ipv4.tcp_rmem = 4096 87380 16777216
> net.ipv4.tcp_wmem = 4096 65536 16777216
> net.ipv4.tcp_syncookies = 1
>
> net.ipv4.tcp_mem = 50576   64768   98152
> net.core.netdev_max_backlog = 2500
> net.ipv4.netfilter.ip_conntrack_max = 1048576
> net.ipv4.netfilter.ip_conntrack_tcp_timeout_established=1200
>
> net.nf_conntrack_max=237680
> net.netfilter.nf_conntrack_max=237680
> net.netfilter.nf_conntrack_tcp_timeout_established=1200
>
> # fs.file-max = 131072
> # Setting the Minimum System Page Cache
> vm.min_free_kbytes=1024
> #Managing the Swap Space
> vm.swappiness=10
>
>
>
>
>
>
> nginx config
> worker_processes  8;
>
> worker_rlimit_nofile 131072;
> events {
>    worker_connections  65536;
>    use epoll;
> }
>
>
>
> 2011/10/7 MagicBear <magicbearmo at gmail.com>:
>> here is my server results.
>> using 3 of ab, each open 10000 concurrent connections.
>>
>> cat logger | sed  's/||/ /g' | awk '{print $3}'| sed 's/\.[0-9]\+//g'
>> | sort | uniq -c
>>  66776 1317949624
>>  91383 1317949625
>>  92828 1317949626
>>  93364 1317949627
>>  91456 1317949628
>>  93498 1317949629
>>  92916 1317949630
>>  91795 1317949631
>>  91921 1317949632
>>  92935 1317949633
>>  93000 1317949634
>>  89737 1317949635
>>  91141 1317949636
>>  93217 1317949637
>>  93490 1317949638
>>  93069 1317949639
>>  88566 1317949640
>>  93721 1317949641
>>  93860 1317949642
>>  90619 1317949643
>>  93118 1317949644
>>  93011 1317949645
>>  94501 1317949646
>>  93367 1317949647
>>  92656 1317949648
>>  91941 1317949649
>>
>> using 60% of cpu.
>> Server Environment:
>> 4x AMD Quad-Core 8360 SE  (total 16 cores)
>> 32G DDR2
>> SATA3 SSD (r/w 550MB/s)
>> 4x1Gbps Ethernet
>>
>> 2011/10/7 Bradley Falzon <brad at teambrad.net>:
>>> On Fri, Oct 7, 2011 at 5:00 AM, atadmin <nginx-forum at nginx.us> wrote:
>>>>
>>>> # Turn on syncookies for SYN flood attack protection
>>>> net.ipv4.tcp_syncookies = 0
>>>
>>> I've never tested the performance benefit (and the costs) of having sync
>>> cookies enabled or not, but that command there suggests you have turned sync
>>> cookies off (you probably want it enabled - again it comes at a cost I
>>> haven't personally investigated).
>>>
>>> Also, you have mentioned the state of iptables connection tracking yet. That
>>> could be a problem if you believe the bottleneck is the server and haven't
>>> checked that already. You probably want to disable nf_conntrack and rewrite
>>> your iptables rules (or just disable firewalling completely).
>>>
>>> --
>>> Bradley Falzon
>>> brad at teambrad.net
>>>
>>> _______________________________________________
>>> nginx mailing list
>>> nginx at nginx.org
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>
>>
>>
>>
>> --
>> MagicBear
>>
>
>
>
> --
> MagicBear
>



-- 
MagicBear