Nginx Upload Progress Module stops functioning once behind reverse-proxy

[Francis Daly]

On Tue, Feb 17, 2015 at 09:30:46PM -0500, Ben Johnson wrote:
> On 2/16/2015 3:52 PM, Francis Daly wrote:

Hi there,

> > ...and draw a picture of what happens when a client sends a 1MB upload
> > to nginx, and nginx sends it via proxy_pass to an upstream. That should
> > show you why it fails.

> Any further explanation that you're willing to offer as to the
> fundamental problem will not fall on deaf ears.

nginx buffers the entire request before sending anything to upstream.

So upstream sees nothing while the client is sending, and then sees
everything, fed by the (presumably) fast link between it and nginx.

So the only place a progress meter can usefully be is on the initial
nginx, which is talking directly with the client. Anything upstream of
that will show 0% until it shows 100%, which is not really a "progress"
meter.

> > Why does there need to be an upload_progress module in the first place,
> > instead of letting the upstream server take care of it?
> 
> Because the upstream server (PHP) performs miserably by comparison when
> handling massive file uploads (often in excess of 1GB),

Nope. At least, that's not the main reason. The upstream server does
not *see* the progress, and so cannot report on it.

> > Not going to work. At least until you can use an "unbuffered
> > upload". Which is not in current nginx. (But is in other things.)

> I would rather push this functionality upstream and consolidate all
> virtual-hosts, incurring whatever risk of downtime is introduced as a
> result, than replace nginx with another piece of software.

Search for "nginx unbuffered upload".

You can possibly try patching yourself, or wait until stock nginx supports
it, or offer suitable encouragement to get stock nginx to support it on
your preferred timetable.

But until it exists, only the thing talking to the client can offer any
kind of progress meter.

Cheers,

	f
-- 
Francis Daly        francis at daoine.org