[ANN] ngx_lua v0.1.5: ability to capture multiple parallel subrequests

[agentzh]

Hi, folks!

I'm pleased to announce that ngx_lua v0.1.5 is now released. You can
download the release tarball from the download page:

    https://github.com/chaoslawful/lua-nginx-module/downloads

This version of the ngx_lua module implements the
ngx.location.capture_multi method that can issue multiple nginx
subrequests at the same time, capture their responses along the way,
and then return the results in the same order that they're specified
in the arguments.

Here is a small example:

    location ~ '^/echo/(\w+)$' {
        echo -n $1;
    }

    location = /main {
        content_by_lua '
            local res1, res2, res3 =
                ngx.location.capture_multi{
                    { "/echo/sun" },
                    { "/echo/moon" },
                    { "/echo/earth" },
                }
            ngx.say(res1.body)
            ngx.say(res2.body)
            ngx.say(res3.body)
        ';
    }

then accessing /main gives

    sun
    moon
    earth

and those three subrequests, GET /echo/sun, GET /echo/moon, and GET
/echo/earth, were issued at the same time.

Below is a more complicated example that fires 2 concurrent MySQL
queries to calculate the count of dogs and the count cats,
respectively, at the same time, and finally output the sum of these 2
counts, i.e., the total count of all the animals.

First of all, we define the remote mysql server upstream in the main http block:

    upstream mysql_node {
        drizzle_server 10.32.136.5:3306
            user=monty password=some_pass dbname=main protocol=mysql;
        drizzle_keepalive max=100 mode=single overflow=ignore;
    }

then we define a general-purpose internal location to proxy SQL
queries to remote MySQL node in our default virtual host server block:

    location = /mysql {
        internal;
        drizzle_query $echo_request_body;
        drizzle_pass mysql_node;
        rds_json on;
    }

after that, we define our main location that does the job:

    location /main {
        content_by_lua '
            local opts1 = {
                method = ngx.HTTP_POST,
                body = "select sum(1) cnt from cats",
            }
            local opts2 = {
                method = ngx.HTTP_POST,
                body = "select sum(1) cnt from dogs",
            }
            local res1, res2 =
                ngx.location.capture_multi{
                    { "/mysql", opts1 },
                    { "/mysql", opts2 }
                }

            local yajl = require("yajl")
            local cats = yajl.to_value(res1.body)
            local dogs = yajl.to_value(res2.body)

            local total = cats[1].cnt + dogs[1].cnt
            ngx.say("for total ", total, " animals.")
        ';
    }

For brevity, I've omitted all the error handling code like checking if
res1.status is 200 and validating cats.errcode has nil values ;)

The nginx modules ngx_drizzle, ngx_rds_json, and the lua-yajl Lua
library are also used in the sample above.

This example can be trivially extended to run tens or even more sql
queries to different MySQL machines and combined with more PostgreSQL
database servers. All in parallel and all are 100% non-blocking on
network traffic.

This v0.1.5 version of ngx_lua also contains

    * optimizations on subrequest buffer management and thus saving
much memory when using ngx.location.capture (and
ngx.location.capture_multi).

    * fixes of the build system that we no longer require OpenSSL and
ngx_lua no longer crash when used with statically linked OpenSSL.
Thanks Marcus Clyne and Vladislav Manchev.

    * fixes of the Lua code cache handling that we no longer clear
pre-loaded standard Lua packages like "table" and "string" when the
Lua code cache is off.

The ngx_lua module embeds the Lua/LuaJIT interpreter into the nginx
core and integrates the powerful Lua threads (aka Lua coroutines) into
the nginx event model by means of nginx subrequests.

You can get the latest source code and full documentation from the
ngx_lua's project page:

   https://github.com/chaoslawful/lua-nginx-module/

Happy Lua and nginx.conf hacking!

Cheers,
-agentzh