Strange nginx issue

[Steve Wilson]

I've just done a drupal7 site under nginx+php-fpm on debian. 

One thing I noticed was that the php process wasn't closing fast enough,
this was tracked down to an issue with mysql. Connections were sitting
idle for a long time which basically exhausted the fpm workers on both
the web servers. 

We have 2 mysql nodes doing replication between them and I think the
binlog commit was holding this up, adding
"innodb_flush_log_at_trx_commit = 0" to the my.cnf stopped this problem
from occuring. 

Page caching is stored in mysql too, moving this to memcached helped
massively and reduced the daily binlogs from 5G/day down to a few
hundred meg. 

I'm not sure if this is a strange setup but we have nginx terminating
ssl which proxies to varnish which then has 2 additional nginx nodes
serving drupal7, these use a 2 node mysql cluster and 2 memcached nodes
for caching pages etc. 

Steve. 

On 07/04/2014 15:34, Geo P.C. wrote: 

> We are facing a strange issue on our servers. We have servers with 1GB RAM and some drupal sites are running on it. 
> 
> Generally all sites are loading fine but sometimes we are unable to access any sites. After waiting for 10mts we are getting a 502 gateway timeout error. In middle when we restart either nginx or php5-fpm it will load. 
> 
> Our configurations are as follows: 
> 
> /etc/nginx/nginx.conf: 
> 
> user www-data; 
> 
> worker_processes 1; 
> 
> pid /run/nginx.pid; 
> 
> worker_rlimit_nofile 400000; 
> 
> events { 
> 
> worker_connections 10000; 
> 
> multi_accept on; 
> 
> use epoll; 
> 
> } 
> 
> http { 
> 
> access_log off; 
> 
> sendfile on; 
> 
> tcp_nopush on; 
> 
> tcp_nodelay on; 
> 
> keepalive_timeout 2; 
> 
> types_hash_max_size 2048; 
> 
> server_tokens off; 
> 
> keepalive_requests 100000; 
> 
> reset_timedout_connection on; 
> 
> port_in_redirect off; 
> 
> client_max_body_size 10m; 
> 
> proxy_connect_timeout 600s; 
> 
> proxy_send_timeout 600s; 
> 
> proxy_read_timeout 600s; 
> 
> fastcgi_send_timeout 600s; 
> 
> fastcgi_read_timeout 600s; 
> 
> open_file_cache max=200000 inactive=20s; 
> 
> open_file_cache_valid 30s; 
> 
> open_file_cache_min_uses 2; 
> 
> open_file_cache_errors on; 
> 
> /etc/php5/fpm/pool.d/www.conf 
> 
> pm.max_children = 5 
> 
> pm.start_servers = 2 
> 
> pm.min_spare_servers = 1 
> 
> pm.max_spare_servers = 3 
> 
> ;pm.process_idle_timeout = 10s; 
> 
> ;pm.max_requests = 200 
> 
> request_terminate_timeout = 300s 
> 
> And please see the added contents in /etc/sysctl.conf 
> 
> ########################## 
> 
> fs.file-max = 150000 
> 
> net.core.netdev_max_backlog=32768 
> 
> net.core.optmem_max=20480 
> 
> #net.core.rmem_default=65536 
> 
> #net.core.rmem_max=16777216 
> 
> net.core.somaxconn=50000 
> 
> #net.core.wmem_default=65536 
> 
> #net.core.wmem_max=16777216 
> 
> net.ipv4.tcp_fin_timeout=120 
> 
> #net.ipv4.tcp_keepalive_intvl=30 
> 
> #net.ipv4.tcp_keepalive_probes=3 
> 
> #net.ipv4.tcp_keepalive_time=120 
> 
> net.ipv4.tcp_max_orphans=262144 
> 
> net.ipv4.tcp_max_syn_backlog=524288 
> 
> net.ipv4.tcp_max_tw_buckets=524288 
> 
> #net.ipv4.tcp_mem=1048576 1048576 2097152 
> 
> #net.ipv4.tcp_no_metrics_save=1 
> 
> net.ipv4.tcp_orphan_retries=0 
> 
> #net.ipv4.tcp_rmem=4096 16384 16777216 
> 
> #net.ipv4.tcp_synack_retries=2 
> 
> net.ipv4.tcp_syncookies=1 
> 
> #net.ipv4.tcp_syn_retries=2 
> 
> #net.ipv4.tcp_wmem=4096 32768 16777216 
> 
> ########################## 
> 
> Can anyone please help us on it. 
> 
> Thanks 
> 
> Geo 
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx [1]

 

Links:
------
[1] http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20140407/e9881785/attachment-0001.html>