Nginx worker processors D state and high I/O utilization !!

[shahzaib shahzaib]

What should be my next step ? Should i buy SAS Drive with hard-raid-10 ?


On Tue, Jul 16, 2013 at 12:34 PM, Sergey Kandaurov <pluknet at nginx.com>wrote:

> On Jul 16, 2013, at 8:59 AM, shahzaib shahzaib <shahzaib.cb at gmail.com>
> wrote:
> > Hello,
> >
> >       We're using nginx-1.2.8 to serve large static files for video
> streaming. However all nginx worker_processes are in "D" state and HDD I/O
> utilization is 99%.
> > […]
> > [root at DNTX010 ~]# iostat -x -d 3
> > Linux 2.6.32-358.6.2.el6.x86_64 (DNTX010.local)         07/16/2013
>  _x86_64_        (8 CPU)
> >
> > Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s
> avgrq-sz avgqu-sz   await  svctm  %util
> > sda              30.28   177.37  260.32    2.96 38169.26  1442.70
> 150.46     2.29    8.70   3.52  92.78
> >
> > Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s
> avgrq-sz avgqu-sz   await  svctm  %util
> > sda               4.33     0.00  544.00    0.00 34376.00     0.00
>  63.19    43.83   75.25   1.84 100.00
> >
> > Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s
> avgrq-sz avgqu-sz   await  svctm  %util
> > sda               9.00     6.33  547.67    0.67 34637.33    56.00
>  63.27    48.01   86.20   1.82 100.00
> >
> > Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s
> avgrq-sz avgqu-sz   await  svctm  %util
> > sda               0.00     0.67  568.00    2.33 36024.00    29.33
>  63.21    54.98  101.10   1.75 100.00
> >
> > Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s
> avgrq-sz avgqu-sz   await  svctm  %util
> > sda               0.00     4.33  560.33    1.33 35712.00    45.33
>  63.66    37.20   65.06   1.78 100.00
> >
>
>
> You are likely hitting the IOPS limit. 550r/s is quite enough to saturate
> 4xSATA 7200 in RAID10.  There are reads of 64 of something per request
> in average. I'd first look at what does something designated as "rsec/s"
> exactly mean in linux terms here.
>
> --
> Sergey Kandaurov
> pluknet at nginx.com
>
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20130716/5b74ef5f/attachment-0001.html>