nginx POST field data processing (Was Re: Introducing Nginx Guts blog)

[agentzh]

On Sun, Jan 23, 2011 at 2:25 AM, Ranier <ranier at cultura.com.br> wrote:
>
> Hi Agentzh,

Please please do not capitalize my nick.

> Yes I checked this module.
>
> IMO, it has drawbacks!

Sure, it does have drawbacks as a standalone nginx module. I
recommended this module just for the sake of source code references. I
believe this can help you devise a satisfying C level utility for
yourself.

I don't think it will be in the nginx core without a decent input
filter mechanism in the core :)

Please also keep in mind, nginx does not read the request body by
default. It's usually up to the content handler (or any other earlier
phase handlers) to determine whether to actually call the
ngx_http_read_client_request_body function on the C level.

> Drawbacks:
> 1. Need set field by field in nginx.conf ($var)

Sure. Because ngx_form_input fits its user interface into the bigger
picture of nginx.conf scripting :)

> 2. Run the whole module, each time you read one field.

You mean reading one field requires parsing through the whole request
body? Yeah, it's currently designed this way, just like nginx's
$arg_FIELD always walk through the whole query string buffer every
time. One may argue that query strings can not be huge while request
bodies can be of megas of bytes, and I agree it's suboptimal for big
chunk of data when there's lots of fields to be processed :)

> 3. Copy data

Putting data into nginx variables requires an additional data copy.
Are you suggesting any other places that introduces unnecessary data
copies?

> 4. More memory for $var
>

Nginx variables do have their own memory footprint and for the sake of
ngx_form_input's UI, it's inevitable :)

> Core support for POST FIELDS:
> 1. No need set $var in nginx.conf
> 2. Read all fields one pass

> 3. Each field in ngx_str_t variable, no need data copy, less memory

POST fields are (usually) encoded as URI components, are you
suggesting in-place URI decoding? (In-place request body processing
may have bad side-effects that you do not want.) Also...how about the
POST body is big enough for nginx to put it onto temporary disk files?

Cheers,
-agentzh