Proxying large downloads from s3

[Sudara Williams]

Hi guys!

We are proxying files from s3 through our app and had a couple questions
on ideal config.

When I arrived on the scene, the following config was in place:

location /download/ {
    internal;
    proxy_pass https://s3.amazonaws.com;
    proxy_buffering off;
    proxy_buffers 2 4m;
    proxy_buffer_size 4m;
    proxy_busy_buffers_size 4m;
}

I was called in because the server started running out of memory. (It
seemed fine for a long time, probably just didn't max out for a while)

After looking up config, it seemed like 4m was VERY large proxy_buffer
size. I was uncertain about whether proxy_buffering should be off or
not. https://coderwall.com/p/rlguog recommends it off, but most of the
mailing list replies say unless you are using Comet or something,
there's no need for it to be off.

We removed all the extra proxy config and we ended up with something
like this:

location /download/ {
    internal;
    proxy_pass https://s3.amazonaws.com;
    proxy_buffering off;
    chunked_transfer_encoding off;
}

We also tried with proxy_buffering on. In both cases it seems like we
are seeing truncated responses. This especially happens on large files
– the file will just be "done" downloading early, and the zip file will
be corrupt.

We are also seeing errors like this, but uncertain if it's related.

2014/03/20 00:02:36 [error] 15519#0: *24132 upstream timed out (110:
Connection timed out) while reading response header from upstream,
client: 207.154.10.35, server: localhost, request: "GET
/download/?uuid=bdba5a17f58c8c808b6cc7fd97cf752a7b80ce1cc6eba6f0b56e411ef7cf3135
HTTP/1.1", upstream:
"https://192.168.99.1:443/utilities/s3_manifest_for_nginx_to_zip_and_stream/?uuid=bdbaecc7fd97cf752a7b80ce1cc6eoaoef7cf3135",
host: "streaming.somesite.com"

I was suspicious of chunked_transfer_encoding  being off, so config now
looks like this:

location /pro-core.com/ {
    internal;
    proxy_pass https://s3.amazonaws.com;
    proxy_max_temp_file_size 256m;
    proxy_read_timeout 300;
}

We are now waiting to find out if our large downloads are still being
interrupted. But we would appreciate any advice about proxying large
files from s3. Should proxy_buffering be on or off?

Thanks :)

Sudara

-- 
Posted via http://www.ruby-forum.com/.