proxy buffering for media files?

[bkosborne]

I'm working on a configuration for an nginx proxy that splits requests
between two upstream servers. The main reason I'm using a proxy is for SSL
termination and for redundancy between the two upstream servers. Each
upstream server is just a simple nginx server with identical media files
stored on each.

The largest media file requested is around 2.5 megabytes. There are files
larger than that, but they are requested in byte-ranges from our CDN.

I'm wondering how I should configure proxy buffering here. I noticed that
the default is set to on:
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering.
But, I'm not sure the default values for the buffer sizes and whatnot are
ideal. I've had proxy_buffering turned on for quite a while now, and have
noticed that the proxy_temp directory has over 1 GB of data in it. To my
understanding, this folder is used when the in-memory buffers cannot hold
all of the data from the upstream, so it is written to disk.

1. If I set proxy_buffering to off, does that mean that the proxy streams
the data directly to the client without buffering anything? Essentially,
that would mean that an nginx worker would be "busy" on both the upstream
and proxy server for the entire duration of the request, correct?

2. If I keep it on, does it make sense to change the buffer sizes so that
the entire response from the upstream can fit into memory? I assume that
would speed up the responses so that nothing is written to disk (slow). From
my novice perspective, it seems counter intuitive to essentially read a file
from upstream disk, write it to proxy disk, and then read it from proxy disk
again.

What is a common use case for using proxy_buffering? Since it's a default
option, I assume it's commonly used and for good reason. I'm just having a
hard time applying the thought process to my specific setup.

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,242495,242495#msg-242495