<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">The ultimate bottleneck in any setup
like this is usually raw cpu power. A single virtual core doesn't
look like it'll hack it. You've got 35 php processes serving 250
users, and I think it's just spread a bit thin.<br>
<br>
Apart from adding cores, there are 2 things I'd suggest looking at<br>
<br>
- are you using an opcode cacher? APC ( install via pecl to get
the latest ) works really well with php in fpm... allocate plenty
of memory to it too<br>
- check the bandwidth at the network interface. The usual 100Mbit
connection can easily get swamped by a graphics rich site -
especially with 250 concurrent users. If this is a problem, then
look at using a CDN to ease things.<br>
<br>
hth,<br>
<br>
Steve<br>
<br>
On 11/10/13 21:07, Nikolaos Milas wrote:<br>
</div>
<blockquote cite="mid:5257B1BA.1050702@noa.gr" type="cite">Hello,
<br>
<br>
I am trying to migrate a Joomla 2.5.8 website from Apache to NGINX
1.4.2 with php-fpm 5.3.3 (and MySQL 5.5.34) on a CentOS 6.4 x86_64
Virtual Machine (running under KMS).
<br>
<br>
The goal is to achieve better peak performance: This site has
occasional high peaks; while the normal traffic is ~10 req/sec, it
may reach > 3000 req/sec for periods of a few hours (due to the
type of services the site provides - it is a non-profit, real-time
seismicity-related site - so php caching should not be more than
10 seconds).
<br>
<br>
The new VM (using Nginx) currently is in testing mode and it only
has 1-core CPU / 3 GB of RAM. We tested performance with
loadimpact and the results are attached.
<br>
<br>
You can see at the load graph that as the load approaches 250
clients, the response time increases very much and is already
unacceptable (this happens consistently). I expected better
performance, esp. since caching is enabled. Despite many efforts,
I cannot find the cause of the bottleneck, and how to deal with
it. We would like to achieve better scaling, esp. since NGINX is
famous for its scaling capabilities. Having very little experience
with Nginx, I would like to ask for your assistance for a better
configuration.
<br>
<br>
When this performance deterioration occurs, we don't see very high
CPU load (Unix load peaks 2.5), neither RAM exhaustion (System RAM
usage appears to be below 30%). [Monitoring is through Nagios.]
<br>
<br>
Can you please guide me on how to correct this issue? Any and all
suggestions will be appreciated.
<br>
<br>
Current configuration, based on info available on the Internet, is
as follows (replaced true domain/website name and public IP
address(es)):
<br>
<br>
=================== Nginx.conf ===================
<br>
<br>
user nginx;
<br>
worker_processes 1;
<br>
<br>
error_log /var/log/nginx/error.log warn;
<br>
pid /var/run/nginx.pid;
<br>
<br>
worker_rlimit_nofile 200000;
<br>
<br>
events {
<br>
worker_connections 8192;
<br>
multi_accept on;
<br>
use epoll;
<br>
}
<br>
<br>
http {
<br>
include /etc/nginx/mime.types;
<br>
default_type application/octet-stream;
<br>
server_names_hash_bucket_size 64;
<br>
<br>
log_format main '$remote_addr - $remote_user [$time_local]
"$request" '
<br>
'$status $body_bytes_sent "$http_referer" '
<br>
'"$http_user_agent"
"$http_x_forwarded_for"';
<br>
<br>
log_format cache '$remote_addr - $remote_user [$time_local]
"$request" '
<br>
'$status $upstream_cache_status
$body_bytes_sent "$http_referer" '
<br>
'"$http_user_agent"
"$http_x_forwarded_for"';
<br>
<br>
fastcgi_cache_path /var/cache/nginx levels=1:2
keys_zone=microcache:5m max_size=1000m;
<br>
<br>
access_log /var/log/nginx/access.log main;
<br>
<br>
sendfile on;
<br>
<br>
tcp_nopush on;
<br>
tcp_nodelay on;
<br>
keepalive_timeout 2;
<br>
<br>
types_hash_max_size 2048;
<br>
server_tokens off;
<br>
<br>
keepalive_requests 30;
<br>
<br>
open_file_cache max=5000 inactive=20s;
<br>
open_file_cache_valid 30s;
<br>
open_file_cache_min_uses 2;
<br>
open_file_cache_errors on;
<br>
<br>
gzip on;
<br>
gzip_static on;
<br>
gzip_disable "msie6";
<br>
gzip_http_version 1.1;
<br>
gzip_vary on;
<br>
gzip_comp_level 6;
<br>
gzip_proxied any;
<br>
gzip_types text/plain text/css application/json
application/x-javascript text/xml application/xml
application/xml+rss text/javascript application/javascript
text/x-js;
<br>
gzip_buffers 16 8k;
<br>
<br>
include /etc/nginx/conf.d/*.conf;
<br>
}
<br>
<br>
==================================================
<br>
<br>
================ website config ==================
<br>
<br>
server {
<br>
listen 80;
<br>
server_name <a class="moz-txt-link-abbreviated" href="http://www.example.com">www.example.com</a>;
<br>
access_log /var/webs/wwwexample/log/access_log main;
<br>
error_log /var/webs/wwwexample/log/error_log warn;
<br>
root /var/webs/wwwexample/www/;
<br>
<br>
index index.php index.html index.htm index.cgi default.html
default.htm default.php;
<br>
location / {
<br>
try_files $uri $uri/ /index.php?$args;
<br>
}
<br>
<br>
location /nginx_status {
<br>
stub_status on;
<br>
access_log off;
<br>
allow 10.10.10.0/24;
<br>
deny all;
<br>
}
<br>
<br>
location ~*
/(images|cache|media|logs|tmp)/.*\.(php|pl|py|jsp|asp|sh|cgi)$ {
<br>
return 403;
<br>
error_page 403 /403_error.html;
<br>
}
<br>
<br>
location ~ /\.ht {
<br>
deny all;
<br>
}
<br>
<br>
location /administrator {
<br>
allow 10.10.10.0/24;
<br>
deny all;
<br>
}
<br>
<br>
location ~ \.php$ {
<br>
<br>
# Setup var defaults
<br>
set $no_cache "";
<br>
# If non GET/HEAD, don't cache & mark user as
uncacheable for 1 second via cookie
<br>
if ($request_method !~ ^(GET|HEAD)$) {
<br>
set $no_cache "1";
<br>
}
<br>
# Drop no cache cookie if need be
<br>
# (for some reason, add_header fails if included in prior
if-block)
<br>
if ($no_cache = "1") {
<br>
add_header Set-Cookie "_mcnc=1; Max-Age=2; Path=/";
<br>
add_header X-Microcachable "0";
<br>
}
<br>
# Bypass cache if no-cache cookie is set
<br>
if ($http_cookie ~* "_mcnc") {
<br>
set $no_cache "1";
<br>
}
<br>
# Bypass cache if flag is set
<br>
fastcgi_no_cache $no_cache;
<br>
fastcgi_cache_bypass $no_cache;
<br>
fastcgi_cache microcache;
<br>
fastcgi_cache_key $scheme$host$request_uri$request_method;
<br>
fastcgi_cache_valid 200 301 302 10s;
<br>
fastcgi_cache_use_stale updating error timeout
invalid_header http_500;
<br>
fastcgi_pass_header Set-Cookie;
<br>
fastcgi_pass_header Cookie;
<br>
fastcgi_ignore_headers Cache-Control Expires Set-Cookie;
<br>
<br>
try_files $uri =404;
<br>
include /etc/nginx/fastcgi_params;
<br>
fastcgi_param PATH_INFO $fastcgi_script_name;
<br>
fastcgi_intercept_errors on;
<br>
<br>
fastcgi_buffer_size 128k;
<br>
fastcgi_buffers 256 16k;
<br>
fastcgi_busy_buffers_size 256k;
<br>
fastcgi_temp_file_write_size 256k;
<br>
fastcgi_read_timeout 240;
<br>
<br>
fastcgi_pass unix:/tmp/php-fpm.sock;
<br>
<br>
fastcgi_index index.php;
<br>
include /etc/nginx/fastcgi_params;
<br>
fastcgi_param SCRIPT_FILENAME
$document_root$fastcgi_script_name;
<br>
<br>
}
<br>
<br>
location ~* \.(ico|pdf|flv)$ {
<br>
expires 1d;
<br>
}
<br>
<br>
location ~* \.(js|css|png|jpg|jpeg|gif|swf|xml|txt)$ {
<br>
expires 1d;
<br>
}
<br>
<br>
}
<br>
==================================================
<br>
<br>
================= php-fpm.conf ===================
<br>
include=/etc/php-fpm.d/*.conf
<br>
[global]
<br>
pid = /var/run/php-fpm/php-fpm.pid
<br>
error_log = /var/log/php-fpm/error.log
<br>
<br>
daemonize = no
<br>
==================================================
<br>
<br>
============== php-fpm.d/www.conf ================
<br>
<br>
[www]
<br>
listen = /tmp/php-fpm.sock
<br>
listen.allowed_clients = 127.0.0.1
<br>
user = nginx
<br>
group = nginx
<br>
<br>
pm = dynamic
<br>
pm.max_children = 1024
<br>
pm.start_servers = 5
<br>
pm.min_spare_servers = 5
<br>
pm.max_spare_servers = 35
<br>
<br>
slowlog = /var/log/php-fpm/www-slow.log
<br>
<br>
php_flag[display_errors] = off
<br>
php_admin_value[error_log] = /var/log/php-fpm/www-error.log
<br>
php_admin_flag[log_errors] = on
<br>
php_admin_value[memory_limit] = 128M
<br>
<br>
php_value[session.save_handler] = files
<br>
php_value[session.save_path] = /var/lib/php/session
<br>
<br>
==================================================
<br>
<br>
================ mysql my.cnf ====================
<br>
<br>
[mysqld]
<br>
datadir=/var/lib/mysql
<br>
socket=/var/lib/mysql/mysql.sock
<br>
symbolic-links=0
<br>
user=mysql
<br>
<br>
query_cache_limit = 2M
<br>
query_cache_size = 200M
<br>
query_cache_type=1
<br>
thread_cache_size=128
<br>
key_buffer = 100M
<br>
join_buffer = 2M
<br>
table_cache= 150M
<br>
sort_buffer= 2M
<br>
read_rnd_buffer_size=10M
<br>
tmp_table_size=200M
<br>
max_heap_table_size=200M
<br>
<br>
[mysqld_safe]
<br>
log-error=/var/log/mysqld.log
<br>
pid-file=/var/run/mysqld/mysqld.pid
<br>
<br>
==================================================
<br>
<br>
=============== mysqltuner report ================
<br>
<br>
>> MySQLTuner 1.2.0 - Major Hayden <a class="moz-txt-link-rfc2396E" href="mailto:major@mhtx.net"><major@mhtx.net></a>
<br>
<br>
-------- General Statistics
--------------------------------------------------
<br>
[--] Skipped version check for MySQLTuner script
<br>
[OK] Currently running supported MySQL version 5.5.34
<br>
[OK] Operating on 64-bit architecture
<br>
<br>
-------- Storage Engine Statistics
-------------------------------------------
<br>
[--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
<br>
[--] Data in MyISAM tables: 9M (Tables: 80)
<br>
[--] Data in InnoDB tables: 1M (Tables: 65)
<br>
[--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 17)
<br>
[--] Data in MEMORY tables: 0B (Tables: 4)
<br>
[!!] Total fragmented tables: 66
<br>
<br>
-------- Security Recommendations
-------------------------------------------
<br>
[OK] All database users have passwords assigned
<br>
<br>
-------- Performance Metrics
-------------------------------------------------
<br>
[--] Up for: 12h 51m 16s (21K q [0.471 qps], 1K conn, TX: 10M, RX:
1M)
<br>
[--] Reads / Writes: 55% / 45%
<br>
[--] Total buffers: 694.0M global + 21.4M per thread (151 max
threads)
<br>
[!!] Maximum possible memory usage: 3.8G (135% of installed RAM)
<br>
[OK] Slow queries: 0% (0/21K)
<br>
[OK] Highest usage of available connections: 23% (36/151)
<br>
[OK] Key buffer size / total MyISAM indexes: 150.0M/5.1M
<br>
[OK] Key buffer hit rate: 99.3% (51K cached / 358 reads)
<br>
[OK] Query cache efficiency: 80.9% (10K cached / 13K selects)
<br>
[OK] Query cache prunes per day: 0
<br>
[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 55
sorts)
<br>
[OK] Temporary tables created on disk: 8% (5 on disk / 60 total)
<br>
[OK] Thread cache hit rate: 98% (36 created / 1K connections)
<br>
[OK] Table cache hit rate: 20% (192 open / 937 opened)
<br>
[OK] Open file limit used: 0% (210/200K)
<br>
[OK] Table locks acquired immediately: 99% (4K immediate / 4K
locks)
<br>
[!!] Connections aborted: 8%
<br>
[OK] InnoDB data size / buffer pool: 1.1M/128.0M
<br>
<br>
==================================================
<br>
<br>
Please advise.
<br>
<br>
Thanks and Regards,
<br>
Nick
<br>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
nginx mailing list
<a class="moz-txt-link-abbreviated" href="mailto:nginx@nginx.org">nginx@nginx.org</a>
<a class="moz-txt-link-freetext" href="http://mailman.nginx.org/mailman/listinfo/nginx">http://mailman.nginx.org/mailman/listinfo/nginx</a></pre>
</blockquote>
<br>
</body>
</html>