Breaking content generation phase into multiple phases or adding custom events in content phase

Jeff Kaufman jefftk at
Thu Jun 18 11:06:34 UTC 2015

ngx_pagespeed does this by giving nginx a pipe to watch, setting up a
handler for that pipe, calling an async api that uses threads, then
the from the callback writing a byte to the pipe.  Now when the async
code finishes we're back on the nginx event loop in the pipe's

On Wed, Jun 17, 2015 at 11:27 PM, Yichun Zhang (agentzh)
<agentzh at> wrote:
> Hello!
> On Thu, Jun 18, 2015 at 2:17 AM, Kaustubh Deorukhkar wrote:
>> I am working on a custom module where I need to use a third party library
>> and make sync/async calls to APIs. I do not have control over what the
>> library does internally but any async API call on library would call a
>> callback which indicates that content generation phase continue with forming
>> response and sending it back to client.
> If you MUST use this 3rd-party library, then you can check out our
> ngx_drizzle [1] (for nonblocking MySQL communication via libdrizzle)
> and ngx_postgres [2] (for nonblocking PostgreSQL communication via
> libpq) for such 3rd-party library integration examples (both of them
> are production ready for years.
> But in retrospect, it took a *lot* of developer efforts to get them
> exactly right due to the inherent limitations in nginx's upstream
> mechanism and you MAY run into bugs in 3rd-party libraries when using
> edge-triggered (ET) events (well, we had to work around such issues in
> libpq, at least).
> The recommended way is to re-implement the wire protocol for I/O
> directly in Lua atop the cosocket API [3] provided by the ngx_lua
> module (or better, use the openresty bundle directly), in the same
> spirit of the existing lua-resty-mysql [4] and lua-resty-redis [5]
> libraries out there.
>> We want to avoid upstream server model if this is already possible with
>> nginx.
> Both ngx_postgres an ngx_drizzle on based on a good part of the stock
> nginx's upstream mechanism. It's easier to reuse it than coding
> everything from scratch if you stick with that road.
> It's worth mentioning that the cosocket mechanism in ngx_lua is a
> *parallel* implementation to the official upstream thing and overcomes
> all those limitations in "upstream" and makes things much cleaner and
> nicer at least on the Lua land. Still, we inherit most (if not all) of
> the good stuff from the "upstream" facility. You can check out the
> picture below for some ideas:
> Best regards,
> -agentzh
> [1]
> [2]
> [3]
> [4]
> [5]
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at

More information about the nginx-devel mailing list