nginx high i/o
Stefanita Rares Dumitrescu
katmai at keptprivate.com
Sat Nov 21 16:19:16 MSK 2009
fastcgi_max_temp_file_size 0;
This directive turns off fastcgi buffering according to the source code.
i did this, and the buffer errors are gone. however, the pages load very
very slow. take about 5-10 seconds to load the page.
could it be my php-fpm.conf ?
the thing is why i am going nuts about this is because the same exact
config (same nginx/php-fpm), i am running on a centos server, and i am
able to push over 200 mb/s and everything works perfectly fine.
i switched over to freebsd to squeeze some more performance out of the
server though.
also i have noticed that when more and more connections are established
to the server, my php-cgi processes are frozen in "sbwait" state:
32514 nobody 1 4 0 60816K 14252K sbwait 4 0:12 0.00% php-cgi
32562 nobody 1 4 0 60816K 14212K sbwait 2 0:11 0.00% php-cgi
32577 nobody 1 4 0 60816K 14240K sbwait 6 0:11 0.00% php-cgi
32479 nobody 1 4 0 60816K 14244K sbwait 0 0:11 0.00% php-cgi
32475 nobody 1 4 0 60816K 14256K sbwait 0 0:11 0.00% php-cgi
32497 nobody 1 4 0 60816K 14228K sbwait 0 0:11 0.00% php-cgi
32527 nobody 1 4 0 60816K 14208K sbwait 1 0:11 0.00% php-cgi
32564 nobody 1 4 0 60816K 14192K sbwait 4 0:10 0.00% php-cgi
posted php-fpm.conf below.
<?xml version="1.0" ?>
<configuration>
All relative paths in this config are relative to php's install
prefix
<section name="global_options">
Pid file
<value
name="pid_file">/usr/local/kphp/logs/php-fpm.pid</value>
Error log file
<value
name="error_log">/usr/local/kphp/logs/php-fpm.log</value>
Log level
<value name="log_level">notice</value>
When this amount of php processes exited with SIGSEGV
or SIGBUS ...
<value name="emergency_restart_threshold">10</value>
... in a less than this interval of time, a graceful
restart will be initiated.
Useful to work around accidental curruptions in
accelerator's shared memory.
<value name="emergency_restart_interval">1m</value>
Time limit on waiting child's reaction on signals from
master
<value name="process_control_timeout">5s</value>
Set to 'no' to debug fpm
<value name="daemonize">yes</value>
</section>
<workers>
<section name="pool">
Name of pool. Used in logs and stats.
<value name="name">nginx</value>
Address to accept fastcgi requests on.
Valid syntax is 'ip.ad.re.ss:port' or just
'port' or '/path/to/unix/socket'
<value name="listen_address">127.0.0.1:8000</value>
<value name="listen_options">
Set listen(2) backlog
<value name="backlog">-1</value>
Set permissions for unix socket, if one
used.
In Linux read/write permissions must be
set in order to allow connections from web server.
Many BSD-derrived systems allow
connections regardless of permissions.
<value name="owner"></value>
<value name="group"></value>
<value name="mode">0666</value>
</value>
Additional php.ini defines, specific to this
pool of workers.
<value name="php_defines">
<!-- <value
name="sendmail_path">/usr/sbin/sendmail -t -i</value> -->
<!-- <value name="display_errors">0</value>
-->
</value>
Unix user of processes
<value name="user">nobody</value>
Unix group of processes
<value name="group">nobody</value>
Process manager settings
<value name="pm">
Sets style of controling worker process
count.
Valid values are 'static' and 'apache-like'
<value name="style">static</value>
Sets the limit on the number of
simultaneous requests that will be served.
Equivalent to Apache MaxClients directive.
Equivalent to PHP_FCGI_CHILDREN
environment in original php.fcgi
Used with any pm_style.
<value name="max_children">128</value>
Settings group for 'apache-like' pm style
<value name="apache_like">
Sets the number of server
processes created on startup.
Used only when 'apache-like'
pm_style is selected
<value
name="StartServers">32</value>
Sets the desired minimum number
of idle server processes.
Used only when 'apache-like'
pm_style is selected
<value
name="MinSpareServers">32</value>
Sets the desired maximum number
of idle server processes.
Used only when 'apache-like'
pm_style is selected
<value
name="MaxSpareServers">64</value>
</value>
</value>
The timeout (in seconds) for serving a single
request after which the worker process will be terminated
Should be used when 'max_execution_time' ini
option does not stop script execution for some reason
'0s' means 'off'
<value name="request_terminate_timeout">0s</value>
The timeout (in seconds) for serving of single
request after which a php backtrace will be dumped to slow.log file
'0s' means 'off'
<value name="request_slowlog_timeout">0s</value>
The log file for slow requests
<value name="slowlog">logs/slow.log</value>
Set open file desc rlimit
<value name="rlimit_files">2048</value>
Set max core size rlimit
<value name="rlimit_core">0</value>
Chroot to this directory at the start, absolute
path
<value name="chroot"></value>
Chdir to this directory at the start, absolute path
<value name="chdir"></value>
Redirect workers' stdout and stderr into main
error log.
If not set, they will be redirected to
/dev/null, according to FastCGI specs
<value name="catch_workers_output">yes</value>
How much requests each process should execute
before respawn.
Useful to work around memory leaks in 3rd party
libraries.
For endless request processing please specify 0
Equivalent to PHP_FCGI_MAX_REQUESTS
<value name="max_requests">500</value>
Comma separated list of ipv4 addresses of
FastCGI clients that allowed to connect.
Equivalent to FCGI_WEB_SERVER_ADDRS environment
in original php.fcgi (5.2.2+)
Makes sense only with AF_INET listening socket.
<value name="allowed_clients">127.0.0.1</value>
Pass environment variables like LD_LIBRARY_PATH
All $VARIABLEs are taken from current environment
<value name="environment">
<value name="HOSTNAME">$HOSTNAME</value>
<value
name="PATH">/usr/local/bin:/usr/bin:/bin</value>
<value name="TMP">/tmp</value>
<value name="TMPDIR">/tmp</value>
<value name="TEMP">/tmp</value>
<value name="OSTYPE">$OSTYPE</value>
<value name="MACHTYPE">$MACHTYPE</value>
<value name="MALLOC_CHECK_">2</value>
</value>
</section>
</workers>
</configuration>
Maxim Dounin wrote:
> Hello!
>
> On Sat, Nov 21, 2009 at 01:39:38AM +0100, Stefanita Rares Dumitrescu wrote:
>
>> actually i got a bunch of errors, in error log, as i enabled it to
>> get some idea on what's going on:
>>
>> 2009/11/21 08:47:44 [warn] 24393#0: *57036 an upstream response is
>> buffered to a temporary file
>> /var/tmp/nginx/fastcgi_temp/1/56/0024741561 while reading upstream,
>> client:
>>
>> i am so sleepy right now, could you please tell me where do i get
>> the fastcgi buffers optimizations ? i searched on the wiki but i
>> can't think straight.
>
> Start from tuning fastcgi_buffers:
>
> http://wiki.nginx.org/NginxHttpFcgiModule#fastcgi_buffers
>
> If you haven't had enough memory to deligate it for buffers you
> may consider using fastcgi_max_temp_file_size 0; - this will
> disable disk buffering at cost of not reading data from backend
> connection (i.e. keeping backend busy).
>
> You may also consider investigating why your backend replies are
> so big (fastcgi_buffers should be 64k in total on your platform,
> this is much more than typical generated html...) and moving them
> to nginx itself if possible (e.g. by using X-Accel-Redirect).
>
> Maxim Dounin
>
>>
>>
>> Maxim Dounin wrote:
>>> Hello!
>>>
>>> On Sat, Nov 21, 2009 at 12:21:52AM +0100, Stefanita Rares Dumitrescu wrote:
>>>
>>>> hi guys,
>>>>
>>>> i have posted a question a while ago, and now i am coming back once more.
>>>>
>>>> i have now a dual quad core xeon system, running freebsd 7.2 latest
>>>> patch level, 64 bit, and 16 gb ram. using the latest nginx-devel.
>>>>
>>>> the OS runs on a ssd, the db is set on a normal drive, and the site
>>>> files reside on another ssd.
>>>>
>>>> i had a pretty similar config just instead i was using centos.
>>>>
>>>> the i/o on the main drive (os drive) is constantly at 22 mb/s.
>>>>
>>>> i have disabled nginx logging, and everything has been setup from
>>>> the ground. i have no idea why do i get such high i/o, because i
>>>> have no other apps that could cause this load.
>>> Do you use proxy for big replies and/or have big client uploads?
>>>
>>> If yes, it's probably caused by disk buffering (you should have
>>> warnings in logs, but you have no logs...) and you probably want
>>> to tune your buffer sizes and/or proxy_temp_file_max_size.
>>>
>>> Maxim Dounin
>>>
>>>> could you please point me to some direction ? i don't know what to do.
>>>>
>>>> ad4 ad5 ad6
>>>> KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s
>>>> 54.56 428 22.79 0.00 0 0.00 32.89 4 0.14
>>>> 55.05 492 26.46 0.00 0 0.00 26.38 8 0.21
>>>> 55.24 452 24.40 0.00 0 0.00 36.36 5 0.20
>>>> 53.20 457 23.76 22.31 19 0.42 19.88 8 0.16
>>>> 53.33 464 24.15 0.00 0 0.00 48.47 23 1.11
>>>> 54.14 403 21.32 0.00 0 0.00 16.18 217 3.43
>>>> 52.75 406 20.90 0.00 0 0.00 17.49 178 3.04
>>>> 55.75 402 21.90 0.00 0 0.00 13.42 12 0.16
>>>>
>
More information about the nginx
mailing list