Accessing binary blob data in MySQL backend

[agentzh]

On Tue, May 31, 2011 at 11:50 PM, Piotr Sikora <piotr.sikora at frickle.com> wrote:
> There is no formal specification, but you can learn everything from
> ngx_drizzle / ngx_postgres sources.
>

There's a specification for the RDS binary format in ngx_drizzle's README file:

    http://github.com/chaoslawful/drizzle-nginx-module

See the Output section there.

> Take a look at those 3 files:
>
> https://github.com/chaoslawful/drizzle-nginx-module/blob/master/t/sanity.t
> https://github.com/chaoslawful/drizzle-nginx-module/blob/master/src/resty_dbd_stream.h
> https://github.com/chaoslawful/drizzle-nginx-module/blob/master/src/ngx_http_drizzle_output.c
>

A generic RDS parser can be built based on these C source and header files.

>> and why the use of the database modules for Nginx for lua-nginx more or
>> less enforces the use of JSON. There might be some valid points for this
>> design decision and I would be interested to hear about them. It is also
>> unclear to me if accessing the RDS data directly from lua-nginx is possible.
>
> Because JSON parsers for Lua were already available ;)
> Anyway, I believe that agentzh was/is working on RDS parser for Lua.
>

Yeah, I'll work on the RDS API for ngx_lua, for example:

    local lua_table = ngx.from_rds(some_rds_binary)
    local rds_binary = ngx.to_rds(some_lua_table)

>> Any ideas on that more than welcome :-)
>
> Not sure if this is helpful, but you can return binary data from PostgreSQL
> (you should be using PostgreSQL instead of MySQL anyway :P) using
> ngx_postgres with "postgres_output" directive. There is a reason why there
> is no "drizzle_output" directive, but I'll let agentzh comment on that.
>

We surely *can* implement a drizzle_output directive though it's much
more difficult than ngx_postgres because ngx_drizzle is designed to
work in a streaming fashion while libpq in the Pg world always buffer
the whole resultset.

It seems that I still do have a lot of interesting stuffs to work on,
huh? ;) Well, the focus in the future will, however, be on developing
a full-fledged non-blocking streaming mysql C driver from scratch for
nginx because both libmysql and libdrizzle are sad in one way or
another.

Cheers,
-agentzh