Making subrequests from handler

[Peter Leonov]

On 28.11.2009, at 14:19, agentzh wrote:

> On Sat, Nov 28, 2009 at 8:49 AM, Peter Leonov <pl at inshaker.ru> wrote:
>> As far as I remember, NGX_HTTP_SUBREQUEST_IN_MEMORY helps me to get all the
>> response body in one memory buffer.
> 
> Yup, indeed, but it relies on the content handler being called
> recognizing and implementing the r->subrequest_in_memory semantics.
> 
> ngx_http_upstream implements it, so the standard "proxy" and
> "memcached" modules (as well as my "memc" module) will work as
> expected using this approach.
> 
> But certain upstream modules like "fastcgi" explicitly disables this.
> For instance, quoted from ngx_http_fastcgi_module's handler function:
> 
>   if (r->subrequest_in_memory) {
>       ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0,
>                     "ngx_http_fastcgi_module does not support "
>                     "subrequest in memory");
>       return NGX_HTTP_INTERNAL_SERVER_ERROR;
>   }
> 
> Valery Kholodkov's "eval" module uses this "subrequest_in_memory"
> trick to intercept a subrequest's response body into an nginx variable
> named by the user, like this:
> 
>   location /foo {
>      eval $res {
>          set $memcached_key 'foo';
>          memcached_pass 127.0.0.1:11211;
>      }
>      # now $res has the value of the key 'foo'...
>   }
> 
> But my "echo" module does not support "subrequest_in_memory" at all,
> so the following breaks horribly:
> 
>   location /eval {
>       eval $res {
>           echo hello;
>       }
>       echo $res;
>   }
> 
> The $res variable in the example above won't get the value "hello",
> rather, the "hello" gets sent to the client directly.
> 
> So basically ngx_eval provides a quick check for the
> "subrequest_in_memory" for a given module :)
> 
> Whenever we know our request is being sent to a location that supports
> the in memory semantics, it's certainly much more convenient to
> utilize this trick without introducing a full-fledged output filer :)
Thanks a lot, now all this become much clear!
Am gonna implement this trick adding buffering filter.


> 
>> Then the single buffer was converted to JS string and passed to the handler.
>> After the callback was invoked JS sends a string to the client (some overhead on copying,
>> I see).
>> This approach was not tested well, so I do not know how stable it is :)
>> All back to the 0.6.* ages.
>> 
> 
> I believe this should work for 0.7.x and 0.8.x :)
It does, you are right. I'v tested tonight.

> 
>> P.S. The ngx_http_memc_module is exactly what I was looking for!
> 
> Ah, happy to hear to that.
> 
> I'm working on the ngx_memc module in a full-time fashion these days
> and I'll make the first release soon :)
It's good to hear!
Saw what you can do in only two months with echo module, that was great!
Can't wait to play with memc module from JS :)

> 
>> Take ngx_http_memc_module, add ngx_http_(perl | lua | js | brainfck)_module, than memcachedb, and you'll get the amazing new… em… something very amazing and very new ;)
>> 
> 
> Indeed! Although my current list is something like C, ngx_memc,
> memcached, tokyotyrant, and memcacheq ;)
Nobody can even imagine this year ago ;)

> 
> Happy hacking nginx!
> 
> Best,
> -agentzh

Thanks,
Peter.