aio info

[Richard Kearsley]

Is aio used while reading files from proxy_cache?
I test aio on a file every 1s (it should be in ram buffers, so no disk io needed)
Usually it takes 1ms (or lower), but sometimes it spikes to over a second if disks are really busy
The max_aio_queue doesn't go above 3-4 during test
Any ideas the cause of spikes?


-----Original Message-----
From: Igor Sysoev [mailto:igor at sysoev.ru] 
Sent: 05 May 2011 08:12
To: nginx at nginx.org
Subject: Re: aio info

On Thu, May 05, 2011 at 05:05:37AM +0000, Richard Kearsley wrote:
> Hi
> A lot of talk about AIO recently - I actually didn't know it existed before, so I am trying it on freebsd8
> 
> # sysctl -a | grep aio
> kern.features.aio: 1
> vfs.aio.max_buf_aio: 16
> vfs.aio.max_aio_queue_per_proc: 256
> vfs.aio.max_aio_per_proc: 32
> vfs.aio.unloadable: 0
> vfs.aio.aiod_lifetime: 30000
> vfs.aio.aiod_timeout: 10000
> vfs.aio.num_buf_aio: 0
> vfs.aio.num_queue_count: 0
> vfs.aio.max_aio_queue: 1024
> vfs.aio.target_aio_procs: 4
> vfs.aio.num_aio_procs: 4
> vfs.aio.max_aio_procs: 32
> 
> Could anyone comment what these mean and how to go about tweaking for best performance (should any be increased?)

$ sysctl -d vfs.aio
vfs.aio: Async IO management
vfs.aio.max_buf_aio: Maximum buf aio requests per process (stored in the process)
vfs.aio.max_aio_queue_per_proc: Maximum queued aio requests per process (stored in the process)
vfs.aio.max_aio_per_proc: Maximum active aio requests per process (stored in the process)
vfs.aio.unloadable: Allow unload of aio (not recommended)
vfs.aio.aiod_lifetime: Maximum lifetime for idle aiod
vfs.aio.aiod_timeout: Timeout value for synchronous aio operations
vfs.aio.num_buf_aio: Number of aio requests presently handled by the buf subsystem
vfs.aio.num_queue_count: Number of queued aio requests
vfs.aio.max_aio_queue: Maximum number of aio requests to queue, globally
vfs.aio.target_aio_procs: Preferred number of ready kernel threads for async IO
vfs.aio.num_aio_procs: Number of presently active kernel threads for async IO
vfs.aio.max_aio_procs: Maximum number of kernel threads to use for handling async IO 

There are two types AIO operations inside FreeBSD kernel - raw file
AIO (buf aio) and usual file AIO that uses in-kernel threads/processes.
nginx uses only usual file AIO. Relevant sysctls are:

vfs.aio.max_aio_queue: 1024
vfs.aio.max_aio_queue_per_proc: 256
vfs.aio.max_aio_per_proc: 32
vfs.aio.num_queue_count: 0

vfs.aio.max_aio_procs: 32
vfs.aio.target_aio_procs: 4
vfs.aio.num_aio_procs: 4

vfs.aio.aiod_lifetime: 30000
vfs.aio.aiod_timeout: 10000

You may want to increase

vfs.aio.max_aio_procs
vfs.aio.max_aio_queue
vfs.aio.max_aio_queue_per_proc
vfs.aio.max_aio_per_proc

> Particularly.. vfs.aio.num_queue_count - does this monitor the amount of io waiting?

Yes.


-- 
Igor Sysoev

_______________________________________________
nginx mailing list
nginx at nginx.org
http://nginx.org/mailman/listinfo/nginx