Is sendfile_max_chunk needed when using aio on FreeBSD 9?

[Maxim Dounin]

Hello!

On Wed, Feb 08, 2012 at 07:49:29AM -0500, Maxim Khitrov wrote:

> Hi all,
> 
> Following Igor's advice [1], I'm using the following configuration for
> file handling on FreeBSD 9.0 amd64:
> 
> sendfile on;
> aio sendfile;
> tcp_nopush on;
> read_ahead 256K;
> 
> My understanding of this setup is that sendfile, which is a
> synchronous operation, is restricted to sending only the bytes that
> are already buffered in memory. Once the data has to be read from
> disk, sendfile returns and nginx issues a 1-byte aio_read operation to
> buffer an additional 256 KB of data.
> 
> The question is whether it is beneficial to use sendfile_max_chunk
> option is this configuration as well? Since sendfile is guaranteed to
> return as soon as it runs out of buffered data, is there any real
> advantage to further restricting how much it can send in a single
> call?

It may make sense as in exreame conditions (i.e. if 
sendfile(NODISKIO) fails to send anything right after aio preread) 
nginx will fallback to normal sendfile() (without SF_NODISKIO).  
On the other hand, if the above happens - it means you have 
problem anyway.

> By the way, is tcp_nopush needed here just to make sure that the first
> packet, which contains headers, doesn't get sent out without any data
> in it? I think this would also prevent transmission of partial packets
> when sendfile runs out of data to send and nginx has to wait for the
> aio_read to finish. Wouldn't it be better in this case to send the
> packet without waiting for disk I/O?

The tcp_nopush is expected to prevent transmission of incomplete 
packets.  I see no problem here.

Maxim Dounin