Nginx - Google Summer of Code ideas

[Maxim Dounin]

Hello!

On Fri, Mar 13, 2009 at 02:21:14PM +0000, Valery Kholodkov wrote:

> 
> ----- "Maxim Dounin" <mdounin at mdounin.ru> wrote:
> 
> > Hello!
> > 
> > On Thu, Mar 12, 2009 at 06:40:13PM -0700, Merlin wrote:
> > 
> > > On Thu, Mar 12, 2009 at 4:51 PM, Michael Baudino <
> > > michael.baudino at acrelec.com> wrote:
> > > 
> > > > Hi C.
> > > >
> > > > Participating in the SoC is a definitively good idea.
> > > >
> > > > What about implementing async disk IO ?
> > > >
> > > > --
> > > > Michael Baudino
> > > >
> > > >
> > > >
> > > What's wrong with aio_write() and aio_read()?
> > 
> > Well, you really want to hear?  There is a couple of issues:
> > 
> > 1. There is no good standard method of notification.  Per POSIX 
> > it's uses signals to notify process about completed operations.  
> > Under FreeBSD notifications is possible over kqueue, and probably 
> > other OSes have something too, but it's anyway will require some 
> > non-trivial porting.
> 
> Here I disagree: linux kernel supports notification via eventfd since 2.6.18. eventfd syscall is available as of glibc 2.8, which is a part of e.g. ubuntu intrepid distribution. 
> 
> Personally I've tested this interface and it looks fine to me.

Disagree with what?  I've said there are interfaces, but they 
aren't standard.  You've just added one more example of 
non-standard interface.

> > 2. It's usually implemented as a thread pool within OS kernel (at 
> > least FreeBSD and Linux implementations AFAIK), and usually have 
> > some limits administrator should be aware of (including maximum 
> > number of io requests system may queue).
> 
> It is unclear whether kernel thread pool is a disadvantage or not. I don't think it is reasonable to use disk AIO as a primary mean to serve files, I think it won't make server faster than while using sendfile, but AIO could be serious boost for large out-of-cache files, because it eliminates blocking.
> 
> With such considerations kernel thread pool doesn't seem to be a big hassle.

Yes.  I've mentioned it just to show that AIO interface isn't 
perfect and should be used with care.

I actually think that AIO support in nginx would be really great.  
I've just here to say that it's not trivial task.

Maxim Dounin

p.s. As discussed some time ago on russian mailing list, probably 
the best way will be to combine sendfile() with SF_NODISKIO and 
AIO reading of 1 byte to place some file pages to cache (if 
sendfile() returns EBUSY).  Just mentioning it here to make sure 
this idea (Igor's one AFAIR) won't be lost if someone finally 
start digging into AIO support.