Lost connection after reading 2147479552 bytes with sendfile

[Maxim Dounin]

Hello!

On Wed, Jul 15, 2015 at 09:48:50AM +0200, Mathias Andre wrote:

> Hi,
> 
> Thanks for the detailed reply!
> 
> * Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> <snip>
>  
> > The 2147479552 is a limit applied by default to allow sendfile() 
> > to work with larger files on Linux up to 2.6.16 (see 
> > src/os/unix/ngx_linux_sendfile_chain.c for some comments).  You can see the 
> > same limit on the first sendfile() call in the Ubuntu log as well.
> 
> Indeed, I had also seen a lot of reference to this "magic" number around,
> so I thought it might be related to it.
>  
> > The strange thing here is that on Scientific Linux 6 the call 
> > pretends it send all the bytes in a single non-blocking call.  
> > This is not nginx expects to ever happen, and this is what causes 
> > the problem to appear.  It would be interesting to dig further to 
> > understand what causes this SL6 behaviour.
> 
> OK, I did write a tiny test program to try and reproduce the problem on
> the SL box: it tries to copy 4GB from an existing file in one sendfile
> call:
> https://gist.github.com/mathiasuk/cf46d0f0caf1dd597e59
> 
> As expected the sendfile calls return 2147479552, and the output file is
> indeed 2147479552 bytes long, so this seems to work.
> Here's the trace:
> https://gist.github.com/mathiasuk/694177cf6446428f9498
> 
> I wonder if this could be because my test uses an output file and not a
> socket. I'll try and investigate some more.

The question is "how this can legitimately happen on a 
non-blocking socket".  The "socket" and "non-blocking" parts are 
both important.

For sure this can happen on a file and/or blocking socket.

-- 
Maxim Dounin
http://nginx.org/