Nginx stops sending file after ~1.5mb ?

[Igor Sysoev]

On Fri, Jan 23, 2009 at 02:35:00PM +0200, Yo'av Moshe wrote:

> Thanks Igor.
> I don't want to be rude, but I find it hard to believe that it's a kernel
> bug.
> 
> First, I'm using the default CentOS 5.2 kernel, it's seems to me pretty odd
> that there's such a bug in it. CentOS is being used on a lot of servers,
> some of them must be running nginx.
> 
> Second, I tried serving the same file with thttpd, and it runs just fine.
> http://195.28.180.147:40/temp/hozer.pdf
> Note it's the exact same file which I cannot get from nginx:
> http://www.noal.org.il/static/temp/hozer.pdf
> 
> How come thttpd can serve the file just fine? If it's a kernel bug, I
> thought it would happen with all of my servers.
> If it's using some different way of sending the files which does not depend
> on that epoll call - is there any way to configure nginx to use that as
> well?

I do not look modern thttpd, but according its change log, it has no
epoll support. You may try different ways in nginx: you need to build it

        --with-rtsig_module
        --with-select_module
        --with-poll_module

and then choose method:

events {
     use  select;
     #use  poll;
     #use  rtsig;
}

Also, epoll has two modes: level triggered (default) and edge
triggered (EPOLLET). nginx uses more effective edge triggered mode,
lighttpd uses level triggered one. The bug may be in ET mode only.

As to probabilty of kernel bugs: I saw them in FreeBSD (kqueue, sendfile),
Linux (epoll), Solaris (event ports), and MacOSX (kqueue, sendfile).

> Can I do anything to confirm that it's a kernel bug? If it is, I should
> probably contact CentOS's maintainers.

Your straces confirms this: nginx added socket to epoll and did not
deleted it, nevertheless epoll does not send event.

> Thank you, again.
> Yo'av.
> 
> On Wed, Jan 21, 2009 at 7:26 AM, Igor Sysoev <is at rambler-co.ru> wrote:
> 
> > On Wed, Jan 21, 2009 at 01:52:53AM +0200, Yo'av Moshe wrote:
> >
> > > Igor, here's the new strace.
> > > http://pastebin.com/m5129c54b
> > >
> > > Does the fact it works in low download rates still makes you think it's a
> > > kernel issue?
> >
> > Yes, this is seems like a kernel issue. Kernel excatly said (with EAGAIN)
> > that there is no place to output data, but never returned a new
> > notification
> > via epoll_wait() about free space:
> >
> > 21420 01:39:43 <... epoll_wait resumed> {{EPOLLOUT, {u32=153137960,
> > u64=13811628390360331048}}}, 512, 60000) = 1
> > 21420 01:39:43 gettimeofday({1232494783, 413764}, NULL) = 0
> > 21420 01:39:43 sendfile64(20, 21, [1645625], 304605) = 40320
> > 21420 01:39:43 sendfile64(20, 21, [1685945], 264285) = -1 EAGAIN (Resource
> > temporarily unavailable)
> > 21420 01:39:43 epoll_wait(11,  <unfinished ...>
> >
> > This is probably some race condition inside kernel, that does not appear on
> > low speed download.
> >
> > > Thank you, really!
> > > And thanks, Kiril and Jeff, for confirming that!
> > >
> > > Yo'av.
> > >
> > >
> > > On Tue, Jan 20, 2009 at 9:39 PM, Kiril Angov <kupokomapa at gmail.com>
> > wrote:
> > >
> > > > wget http://www.noal.org.il/static/temp/barvazi2.pdf
> > > > --2009-01-20 21:36:00--
> > http://www.noal.org.il/static/temp/barvazi2.pdf
> > > > Resolving www.noal.org.il... 195.28.180.147
> > > > Connecting to www.noal.org.il|195.28.180.147|:80... connected.
> > > > HTTP request sent, awaiting response... 200 OK
> > > > Length: 1950230 (1.9M) [application/pdf]
> > > > Saving to: `barvazi2.pdf'
> > > >
> > > > 81% [=====================================>         ] 1,596,665
> > > > --.-K/s  eta 5s
> > > >
> > > >
> > > > wget -c http://www.noal.org.il/static/temp/barvazi2.pdf
> > > > --2009-01-20 21:38:28--
> > > > http://www.noal.org.il/static/temp/barvazi2.pdf
> > > > Resolving www.noal.org.il... 195.28.180.147
> > > > Connecting to www.noal.org.il|195.28.180.147|:80... connected.
> > > > HTTP request sent, awaiting response... 206 Partial Content
> > > > Length: 1950230 (1.9M), 305960 (299K) remaining [application/pdf]
> > > > Saving to: `barvazi2.pdf'
> > > >
> > > > 100%[+++++++++++++++++++++++++++++++++++++++=======>] 1,950,230
> > > > 295K/s   in 1.0s
> > > >
> > > > 2009-01-20 21:38:30 (295 KB/s) - `barvazi2.pdf' saved [1950230/1950230]
> > > >
> > > >
> > > > On Tue, Jan 20, 2009 at 9:31 PM, jeff emminger <jemminger at gmail.com>
> > > > wrote:
> > > > >> Could anyone confirm that this works:
> > > > >> wget --limit-rate=3500
> > http://www.noal.org.il/static/temp/barvazi2.pdf
> > > > >> while this doesn't:
> > > > >> wget http://www.noal.org.il/static/temp/barvazi2.pdf
> > > > >>
> > > > >
> > > > > Confirmed
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Yo'av Moshe
> >
> > --
> > Igor Sysoev
> > http://sysoev.ru/en/
> >
> >
> 
> 
> -- 
> Yo'av Moshe

-- 
Igor Sysoev
http://sysoev.ru/en/