Catch ALL requests by LUA-script

[agentzh]

On Thu, May 5, 2011 at 8:05 AM, HowToMeetLadies <nginx-forum at nginx.us> wrote:
>
> Heh, I know what delays are and that was no delay.. Here it is best
> weather for bicycling, so i do all the time! ;) One thing on my todo is
> to visit china, my cousin lived there past 5-6 years and i've seen many
> pictures and videos.. Last month he married his wife there, but i could
> not get away :(
>

Oh, I'm so happy to hear that ;) Please bring my best wishes to your cousin!

>
> Sure, thats no problem. Tomorrow ill attach to it and take a sample for
> you. Have also a full dump, as every build has its own /prefix.
>

Looking forward to that :)

>
> Tbh, youre absolutely right. Realized it before i used this and *I* had
> no problems, also do not have much inserts (about 25-40k text each,
> stored in a fraction of a second). Tested it also further with luasql
> all queries, no problems so far but only with circumventions and all
> under pre-calculated conditions with *enough* instances, fast i/o,
> caching etc. This is at no will a reliable solution and much likely a
> show-stopper! So do *not* try at home eh?!!
>

Well, it will only cause problems when you have too few nginx workers
and nontrivial number of concurrent requests and your database is
going mad and becoming slow ;) Anyway, it's good for personal use, not
for production apps ;)

> In the meantime or maybe forever ill stick to redis only (: Exchanges to
> or from intermediates are stressing me, but still better as seeing ruby
> eating up resources and doing "nothing" on ngx_passenger/ree.. sinatra,
> merb, rails, foo, bla -.-" Redis helped there a lot, but even in my
> *experimental* Rack written with D ive done better! ;) But thats true
> another story..
>

When you're using ngx_redis2 + lua-redis-parser, please ensure you're
using a tcp connection pool (provided by ngx_http_upstream_keepalive)
and redis pipelining wherever possible. These features will
significantly improve performance. Also, using multiple instance of
redis servers on your multi-core machines also help a lot due to the
sequential processing nature of a single redis server instance.

Also, when you're benchmarking performance, please ensure that your
error log level is high enough to prevent nginx workers spend too much
cycles on flushing the error.log file, which is certainly very
expensive :)

> As a last resort for mysql i thought of getting an unique id from a
> (lua) location which is then used for insertion and afterwards on
> success, i.e. for redirects. Or have i missed something, what would you
> do?
>

You can certainly generate a globally unique id by means of pure Lua
or by accessing redis, though that's a bit silly ;)

If all you want is to get LAST_INSERT_ID, then ngx_drizzle already
returns that automatically for you when you're doing a SQL insert
query. Consider the following sample nginx.conf snippet:

    location /test {
        echo_location /mysql "drop table if exists foo";
        echo;
        echo_location /mysql "create table foo (id serial not null,
primary key (id), val real);";
        echo;
        echo_location /mysql "insert into foo (val) values (3.1415926);";
        echo;
        echo_location /mysql "select * from foo;";
        echo;
    }
    location /mysql {
        drizzle_pass backend;
        drizzle_module_header off;
        drizzle_query $query_string;
        rds_json on;
    }

Then GET /test gives the following outputs:

    {"errcode":0}
    {"errcode":0}
    {"errcode":0,"insert_id":1,"affected_rows":1}
    [{"id":1,"val":3.1415926}]

Have you noticed the "insert_id" field in the 3rd JSON response? ;)

Full transaction support will land into ngx_drizzle (maybe
ngx_postgres as well). It will be implemented by locking and tagging
database connections in the pool. But there's no ETA yet for that.

>
> You mentioned "cosocket" somewhere (github?) already and IMO that would
> be really amazing! I'm following "agentzh" and "chaoslawful" on github
> and will take a deeper look when i get more time. Will be linking to
> your projects when i release something useful (safe) for it.
>

Oh, I'm feeling honored :)

>
> Looks very promising, i seen that page already on my researches and
> would love to get my hands on it. I do a lot of research, trying as much
> as possible, protocols like XMPP and PSYC and such good things, but
> mostly everything apart from PHP, .Net and Java.. :>
>

LOL

>
> Heh. >.< All the thanks belong to you and who else is also behind that
> projects! Had so much *fun* with it so far, you wont believe it how it
> boosted everything at our groups site, ookeh not *that* much, but enough
> to already left ruby (was riding since little rails). And personally it
> helped me alot to grok lua, as i needed that for deeper reworks of my
> prosody-im fork (another lua project thats awesome, i prefer it over
> ejabberd/erl which indeed is both neat feature-wise, but wont let me
> change (easily) from mnesia to redis or other nice tricks).
>

ngx_openresty still has many rough corners and has a long way to go.
We'll surely try to make it better and better unfailingly :)

Happy hacking!

-agentzh