processing request body in files

[Maxim Dounin]

Hello!

On Fri, Aug 08, 2014 at 10:44:59AM -0400, Jeff Kaufman wrote:

> In ngx_pagespeed we want to handle POSTs, processing them in our nginx
> module and performing an action.  This works fine for small POSTs, but
> if they're too big then r->request_body->temp_file is set and the data
> isn't available in memory.  Right now we just log an error, but I'd
> like to fix this.
> 
> To read from that file, is it safe to just call
> ngx_open_tempfile(r->request_body->temp_file->file.name, ...) and then
> ngx_read_file() until that returns 0 for EOF (like standard unix) or
> an error?  Or is there something more complex we need to do?
> 
> (I've looked for existing modules that do this, but haven't found any.
> I'm not finding any modules that currently do that; just ones that
> issue a rename command to move r->request_body->temp_file into the
> final destination.)

The modules which read request body from a file don't do this via 
a file name (in most cases, there is no name at all, as temporary 
files are unlink()'ed after they are open), but rather through 
already opened file descriptor.  E.g., upstream does this (using 
output chain) to send a request body to a backend server.

In either case, reading temporary files might be suboptimal - you 
may consider using request body filters instead, an experimental 
patch can be found, e.g., here:

http://mailman.nginx.org/pipermail/nginx-devel/2013-March/003492.html

-- 
Maxim Dounin
http://nginx.org/