Breaking content generation phase into multiple phases or adding custom events in content phase

[Jeff Kaufman]

The two other things that ngx_pagespeed needs to use OS threads for:

* image optimization: blocking the nginx event loop to recompress
large images gives you very visible stalls

* background optimization: we want to return whatever we have to the
user, and then go on optimizing resources this request referenced in
the background

But yes, this (and parsing all the html as it flows through) does keep
you from hitting C10K.  I'm currently playing with some designs for a
lighter version of PageSpeed that drops optimizations that can't be
implemented efficiently or 100% safely, and I think it should be able
to support nearly as many clients as standard nginx.  It would still
use a background thread for image optimization, though.

On Thu, Jun 18, 2015 at 11:06 PM, Yichun Zhang (agentzh)
<agentzh at gmail.com> wrote:
> Hello!
>
> On Thu, Jun 18, 2015 at 7:06 PM, Jeff Kaufman wrote:
>> ngx_pagespeed does this by giving nginx a pipe to watch, setting up a
>> handler for that pipe, calling an async api that uses threads, then
>> the from the callback writing a byte to the pipe.  Now when the async
>> code finishes we're back on the nginx event loop in the pipe's
>> handler.
>>
>
> Extra OS threads add their own complexity and overhead (killing C10K,
> for example), which I think we should avoid wherever we can (well,
> disk I/O cannot be nonblocking and we should only use thread pools for
> that).
>
> Well, just my 2 cents.
>
> Regards,
> -agentzh
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel