Get $request_body before content handlers?

[Mike Gagnon]

Thanks!

On Sat, Jul 7, 2012 at 8:19 PM, agentzh <agentzh at gmail.com> wrote:

> Hello!
>
> On Sat, Jul 7, 2012 at 4:41 PM, Mike Gagnon <mikegagnon at gmail.com> wrote:
> > I have the impression that $request_body isn't available until the
> > content-handlers phase. Is that correct?
> >
>
> It depends on actual nginx modules. It's the nginx module's
> responsibility to trigger request body reading (or discarding). For
> ngx_proxy, ngx_fastcgi, and many other upstream modules, they run in
> the content phase, so request body is read in that phase.
>
> But in modules like ngx_form_input, it reads the request body at
> rewrite phase, see
>
>     https://github.com/calio/form-input-nginx-module
>
>
Thanks for the pointer! I just looked at the source for
form-input-nginx-module.

It gives me the impression that
the function ngx_http_read_client_request_body() isn't guaranteed to read
the complete request_body before it is time to find the value for a
set_form_input variable. In other words. it seems possible that the
directive "set_form_input $data" might not work if the request_body isn't
fully read before ngx_http_set_form_input() is called.

I gathered this impression because it looks like ctx->done is only set
after request_body is fully read. However, the
function ngx_http_set_form_input() only advances to the
function ngx_http_form_input_arg() if ctx->done is set.

Do you know of my analysis correct?

I would like to ensure that the request_body is fully read before my module
attempts to read it during the rewrite phase.

And in ngx_lua, we can read request body at various phases at our own
> will, like "rewrite", "access", and "content" phases.
>
> > Is it possible to hack nginx such that the rewrite phase is delayed until
> > after the request_body is available? I am writing a module that analyzes
> > requests during the rewrite phase to determine where to route them. The
> goal
> > is to use online machine learning to detect suspicious requests and
> re-route
> > them to a quarantine. http://mikegagnon.com/2012/06/08/beer-garden/
> >
>
> We're just using ngx_lua to do suspicious request filtering in rewrite
> and/or access phases. It's convenient (and also rather efficient) to
> use the various Lua APIs for Nginx provided by ngx_lua. See
>
>     http://wiki.nginx.org/HttpLuaModule
>
> It even supports streaming reading of request body via the "downstream
> cosocket" API, so you can implement "input body filter" in pure Lua.
>
>
Thanks for the Lua tips. Lua has a reputation for speed, but I am worried
it won't be as performant as a well engineered C module. Normally, I
wouldn't be concerned with minor speed improvements but I am specifically
trying building my module to withstand CPU overload attacks. Within my
context, it is critical that my module be able to classify incoming
requests quickly.

Do you think I am under-estimating Lua's performance capability?


> > If I have access to the request_body during the rewrite phase then my
> > request classifier will be better at detecting suspicious requests
> (since it
> > has more information about requests).
> >
>
> It's more appropriate to do that at the access phase, preferably
> running after the nginx "access" phase. (You can just ngx_lua's
> access_by_lua or access_by_lua_file directives.)
>
>
Thanks for the tip!


> Best regards,
> -agentzh
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120708/c431699c/attachment.html>