slow streaming due to high %utilization !!
shahzaib shahzaib
shahzaib.cb at gmail.com
Wed Jun 5 08:06:05 UTC 2013
Thanks for co-operation B.R but i've got to know that aio doesn't work well
with linux centos and i already used it once with one of our content
servers for serving large-files. I noted that after enabling aio, the
svctime reduced but %util increased upto 20% using command iostat -x -d 3
Regarding gzip compression, i am afraid that our content server is used for
large videos serving as well as conversion which consumes 80% of cpu
resources and gzip compression would overwhelm the cpus.
Also trying to figure out that how nginx works with the terms asynchronous
and synchronous ?
Best Regards.
On Wed, Jun 5, 2013 at 11:46 AM, B.R. <reallfqq-nginx at yahoo.fr> wrote:
> Hello,
>
> On Wed, Jun 5, 2013 at 1:56 AM, shahzaib shahzaib <shahzaib.cb at gmail.com>wrote:
>
>> Hello,
>>
>> We're using nginx-1.2.8 to stream large files size 1G but found
>> the slow stream with high utilization of harddrive using command "iostat -x
>> -d 3"
>>
>> Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s
>> avgrq-sz avgqu-sz await svctm %util
>> sdc 0.00 444.00 78.67 7.00 18336.00 3608.00
>> 256.16 3.28 39.17 11.02 94.40
>> sdb 0.00 0.00 0.00 0.00 0.00
>> 0.00 0.00 0.00 0.00 0.00 0.00
>> sda 0.00 0.00 0.00 0.00 0.00
>> 0.00 0.00 0.00 0.00 0.00 0.00
>>
>> 16G Ram
>> 8X CPU E31240
>> HDD SATA
>>
>> The best way of serving large files is to use asynchronous transfer to
> avoid blocking disk requests.
> To do so, have a look at the aio<http://nginx.org/en/docs/http/ngx_http_core_module.html#aio>directive. The default behavior uses sendfile, which sends file in a
> synchronous fashion.
>
> Note 1: On Linux, aio will work for writing requests only if activated
> alone. To activate asynchronous read requests (which seems coherent to your
> use case, since you stream files - I understand yo usend them, thus you
> read them), you need to activate the directio<http://nginx.org/en/docs/http/ngx_http_core_module.html#directio>directive.
> Refer to the aio documentaiton (link above) for details.
>
> Note 2: Using directio will automatically disable sendfile. If you don't
> use it, you'll need to deactivate sendfile manually.
>
> You might also wanna have a look at your output_buffers (once again see
> the aio directive documentation) to improve large files buffering.
> I read around that buffers large enough (>= 4MiB) are usually advised for
> big files (splitting them in less parts). Maybe you would like to make some
> tests for your specific use case to find the most appropriate values.
> The number of those buffers also have an impact: too much buffers consumes
> RAM for nothing, whereas too little of them will slow down requests by
> creating an artificial bottleneck.
>
> Following are the main nginx configuration :-
>>
>> http {
>> include mime.types;
>> default_type application/octet-stream;
>> #tcp_nopush on;
>> client_body_buffer_size 128k;
>> client_header_buffer_size 128k;
>> client_max_body_size 1200M;
>> keepalive_timeout 10;
>> ignore_invalid_headers on;
>> client_header_timeout 3m;
>> client_body_timeout 3m;
>> send_timeout 3m;
>> reset_timedout_connection on;
>> #gzip on;
>> access_log off;
>> include /usr/local/nginx/conf/vhosts/*;
>>
>> }
>>
>>
>>
> By default, Nginx sets gzip to off. Depending on the content you stream
> and since you seem to have a nice CPU potential (you didn't provide
> information on CPU usage, you'll need to see if you have room for gzip
> compression there), you might wanna activate gzip to reduce the amount of
> data transferred, thus lowering the transmission time at the cost of a more
> sollicitated CPU... if your content provides room for a sensible
> compression efficiency (once again, it depends on your specific use case).
> No need for it if the benefit is 1% ;o)
>
>>
>>
>> Would be very thankful If someone could help me with better configuration
>> of nginx to reduce HDD-utilization.
>>
>> Best Regards..
>>
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
>>
>
> No other idea.
>
> Hope I helped,
> ---
> *B. R.*
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20130605/0d0ff0d2/attachment.html>
More information about the nginx
mailing list