Catch ALL requests by LUA-script

[agentzh]

On Mon, May 2, 2011 at 12:11 PM, HowToMeetLadies <nginx-forum at nginx.us> wrote:
> Maybe slightly off-topic but then maybe it is useful for you (andiL) or
> someone else.
>

Your feedback is always welcome! Sorry for the delay on my side for I
was enjoying my vocation in the Fuzhou city in the southern region of
China in the last few days :)

> Before i start, @agentzh: every ?nth request, one of nginx
> (0.8.54.4rc2-jit2beta6) worker processes segfaults, is this behaviour
> related to the pcre issue? have not patched it yet.. ah and thank you
> *very* much for giving nginx this all new powers!! ((:
>

What's your operating system? We used to observe similar periodic
segfaults with exactly the same version of ngx_openresty on RedHat
Enterprise Linux (RHEL) 5.4 in production but now all is fine after we
switched LuaJIT to a more recent git HEAD version which includes an
important work-around for an old libgcc bug explained here in the
lua-l mailing list:

    http://lua-users.org/lists/lua-l/2011-03/msg00576.html

I'll make a new release of ngx_openresty to include a new version of
LuaJIT as well as some other modules' updates like ngx_lua's and
ngx_srcache's. I'll inform you here when I'm done.

Also, it'll be very appreciated if you can obtain a gdb backtrace for
your segfaults so that we can look at the details ;)

> Ok, im a long time nginx/redis/ruby user and this are my first steps
> with lua and a lightweight (M)VC framework around three core "classes",
> which uses haml for templates/layout and markdown for text processing (5
> minute ffi discount binding), drizzle and redis for data and redis also
> for caching. Successfully running a little app with it. yay! *.* For now
> i can give you some snippets just to show some basic aspects, ill
> release a fully working example sometime soon, if the segfaults doesnt
> belong to the pcre-issue ill try to debug that too.
>

Oh, that's awesome :)

>
> Below ill use luasql, instead you can use
> ngx.location.capture("/some-internal-rds-json") and lua-yajl, but this
> approach ends as soon as you want INSERT followed by LAST_INSERT_ID() or
> other complex queries/transactions (as long as libdrizzle has no support
> for multiple queries, maybe this is not the best option to go with, ive
> tried alternatives and workarounds, but for now this is what *I* want).
>

The luasql library will certainly block nginx worker processes. Please
dont' use that.

I know libdrizzle is very limited in terms of functionality and it's
*not* maintained very well. We'll start a completely new nginx
upstream modules for mysql named ngx_mysql by implementing a
non-blocking full-fledged highly-optimized mysql client from scratch.
I'd love to see it happen this year ;)

Another direction we're taking is to introduce the "cosocket"
mechanism into ngx_lua such that one can use a *synchronous* Lua
socket API that is transparently *asynchronous* and *non-blocking* on
the nginx C level. Atop that, we can build pure-Lua non-blocking mysql
client driver and thus putting everything onto the Lua land and no
longer need to mess up the nginx.conf file and nginx subrequests. I'd
also want to see it happen by this Christmas ;)

> At the moment i do not need an ORM for my data, but have already done
> some functions for both rds_json and luasql. redis has some commands
> exposed in nginx.conf for easy access with lua (: The core classes:
>
> Request; which contains several ngx.var wrappers (for nil instead of ''
> for empty keys), present(str), simple cookie/session/login, before/after
> filters, haml/markdown rendering, etc..
>
> MethodRequest < Request; which calls functions for HTTP Methods aka GET,
> POST alike..
> and finally Resource < MethodRequest; which has an
> Rails-ActionController-like interface.
>

The high-level abstraction for ngx_lua is excellent! We're taking a
slightly different approach along the line in our web applications:

    http://agentzh.org/misc/nginx/lzsql-manual.html

We're using a DSL (Domain Specific Language) compiler that emits Lua
code and business logic is mostly implemented in this DSL. It's
targeting our business though, not meant to be useful for general web
application development (yet).

Thanks for sharing!

-agentzh