Proxy buffering

[Andrew Tynefield]

Hello all,

I've configured nginx as a load balancing proxy for my backend servers. My
backend is expecting multi-part uploads for large files in small chunks
(5-15mb). The issue I'm encountering, is that I would like for nginx to
just pass the chunked data along to the backend servers and not buffer the
requests.

Current configuration:

upstream riak-cs {
                server 192.168.1.19:8080;
                server 192.168.1.22:8080;
                #least_conn;
}

server {
        listen 80;
        server_name cs.domain.com *.cs.domain.com;
        location / {
                proxy_pass http://riak-cs;
                proxy_set_header Host $host;
                proxy_connect_timeout 59s;
                proxy_send_timeout   600;
                proxy_read_timeout   600;
                proxy_buffering off;
                proxy_pass_header Server;
                add_header Backend $proxy_host:$proxy_port;
                add_header Upstream-Response-Time $upstream_response_time;

        }

}

user              nginx;
worker_processes  4;
error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
    worker_connections  4096;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request"
'
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    tcp_nopush     on;
    tcp_nodelay     on;
    keepalive_requests 100;
    client_max_body_size 1000M;
    keepalive_timeout  3;
    reset_timedout_connection on;
    underscores_in_headers on;

    include /etc/nginx/conf.d/*.conf;

}


I have tried disabling buffers as shown above, however, when I capture the
packets on the backend servers, I see that the stream of data doesn't occur
until after the full body of the upload has completed.

[ jedi ] ~ # nginx -v
nginx version: nginx/1.4.3

If I enabled info error logging, I see:
2013/11/14 22:34:47 [warn] 2698#0: *1 a client request body is buffered to
a temporary file /var/cache/nginx/client_temp/0000000001, client:
192.168.1.1, server: cs.domain.com, request: "PUT
/huge/Windows7Ultimate.iso?partNumber=1&uploadId=1RjFvAcQTsWmpnIYD7nL7Q==
HTTP/1.1", host: "big.cs.domain.com"

How can I prevent this all together?

Thanks,
Andrew

-- 
[Andrew Tynefield]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20131114/ee0f6700/attachment.html>