Chunked Transfer

[Maxim Dounin]

Hello!

On Mon, Oct 29, 2012 at 10:23:45AM +0000, Jonathan Matthews wrote:

> On 29 October 2012 10:08, Maxim Dounin <mdounin at mdounin.ru> wrote:
> > Hello!
> >
> > On Fri, Oct 26, 2012 at 04:58:46PM -0400, AJ Weber wrote:
> >
> >> Asking, because the documentation looks like it's a little outdated
> >> on this...
> >>
> >> Is Chunked Transfer still not enabled OOTB?  This would seem like
> >> almost a mandatory feature of HTTP 1.1 to implement, and the only
> >> reference I could find is to separate source code/module/patch that
> >> I would have to download and recompile all of nginx for?
> >>
> >> Has it been implemented or added to the default, pre-compiled
> >> packages and I just can't see it in the nginx -V output?
> >>
> >> I need the ability to upload large content, and this would appear to
> >> be the proper way to do that.
> >
> > I'm working on it, and it's expected to be available later this
> > month.
> >
> > BTW, if you know examples of real-world use of chunked transfer
> > encoding by clients - please let me know.  AFAIK no browsers use
> > it, and most widespread example I'm aware of is the webdav client
> > in Mac OS X.
> 
> I wanted to use it in this report-generating pipeline:
> 
> bash$ mysql -e 'generate lots of data' | perl 'do some munging' |
> csvtool 'make a proper CSV' | gzip | curl --upload-file -
> http://my.webdav.endpoint.for.customers/foo/bar.gz
> 
> The fact that the chunkin module wouldn't work properly with webdav
> due to curl choosing chunking when PUTting stdin, meant I had to break
> this into multiple parts and ensure that curl could upload a complete
> file from disk. I mentioned the issue here:
> http://mailman.nginx.org/pipermail/nginx/2012-April/033141.html
> 
> This increased the complexity of what I was doing, as I now had local
> state on disk that had to be managed/cleaned-up/etc.

JFYI, chunked request body patches are available here:

http://mailman.nginx.org/pipermail/nginx-devel/2012-November/002961.html

Review and testing appreciated.

-- 
Maxim Dounin
http://nginx.com/support.html