Making subrequests from handler

[agentzh]

On Sat, Nov 28, 2009 at 8:49 AM, Peter Leonov <pl at inshaker.ru> wrote:
> As far as I remember, NGX_HTTP_SUBREQUEST_IN_MEMORY helps me to get all the
> response body in one memory buffer.

Yup, indeed, but it relies on the content handler being called
recognizing and implementing the r->subrequest_in_memory semantics.

ngx_http_upstream implements it, so the standard "proxy" and
"memcached" modules (as well as my "memc" module) will work as
expected using this approach.

But certain upstream modules like "fastcgi" explicitly disables this.
For instance, quoted from ngx_http_fastcgi_module's handler function:

    if (r->subrequest_in_memory) {
        ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0,
                      "ngx_http_fastcgi_module does not support "
                      "subrequest in memory");
        return NGX_HTTP_INTERNAL_SERVER_ERROR;
    }

Valery Kholodkov's "eval" module uses this "subrequest_in_memory"
trick to intercept a subrequest's response body into an nginx variable
named by the user, like this:

    location /foo {
       eval $res {
           set $memcached_key 'foo';
           memcached_pass 127.0.0.1:11211;
       }
       # now $res has the value of the key 'foo'...
    }

But my "echo" module does not support "subrequest_in_memory" at all,
so the following breaks horribly:

    location /eval {
        eval $res {
            echo hello;
        }
        echo $res;
    }

The $res variable in the example above won't get the value "hello",
rather, the "hello" gets sent to the client directly.

So basically ngx_eval provides a quick check for the
"subrequest_in_memory" for a given module :)

Whenever we know our request is being sent to a location that supports
the in memory semantics, it's certainly much more convenient to
utilize this trick without introducing a full-fledged output filer :)

> Then the single buffer was converted to JS string and passed to the handler.
> After the callback was invoked JS sends a string to the client (some overhead on copying,
> I see).
> This approach was not tested well, so I do not know how stable it is :)
> All back to the 0.6.* ages.
>

I believe this should work for 0.7.x and 0.8.x :)

> P.S. The ngx_http_memc_module is exactly what I was looking for!

Ah, happy to hear to that.

I'm working on the ngx_memc module in a full-time fashion these days
and I'll make the first release soon :)

> Take ngx_http_memc_module, add ngx_http_(perl | lua | js | brainfck)_module, than memcachedb, and you'll get the amazing new… em… something very amazing and very new ;)
>

Indeed! Although my current list is something like C, ngx_memc,
memcached, tokyotyrant, and memcacheq ;)

Happy hacking nginx!

Best,
-agentzh