Client body buffering with FastCGI

[Maxim Khitrov]

2011/2/17 Maxim Dounin <mdounin at mdounin.ru>:
> Hello!
>
> On Thu, Feb 17, 2011 at 01:47:43PM -0500, Maxim Khitrov wrote:
>
>> On Thu, Feb 17, 2011 at 12:05 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
>> > Hello!
>> >
>> > On Thu, Feb 17, 2011 at 09:48:24AM -0500, Maxim Khitrov wrote:
>> >
>> >> I'm trying to configure AjaXplorer, a PHP/Ajax file manager, to work
>> >> behind nginx 0.8.54 on FreeBSD 7.3. The problem I'm running into is
>> >> the inability to upload files more than ~64 MB in size. Ideally, I'd
>> >> like to bump that limit up to 1 GB. I realize that HTTP is not ideal
>> >> for this, but other transfer methods are not an option.
>> >>
>> >> PHP and nginx are both configured to accept 1 GB POST requests. As far
>> >> as I can tell, nginx buffers the contents of the entire upload to disk
>> >> before forwarding the request to the FastCGI process. This data is
>> >> then read from disk and written back to disk by PHP. The whole
>> >> write/read/write cycle is causing a timeout, first in nginx, and then
>> >> in the PHP process (though there may also be some other problem that I
>> >> haven't figured out yet).
>> >
>> > Setting bigger timeouts should help.  All timeouts in nginx are
>> > configurable (proxy_connect_timeout, proxy_send_timeout,
>> > proxy_read_timeout - and similar ones for other backend modules).
>> >
>> > Though it sounds strange that nginx times out while writing
>> > request to php as it should reset timer on any write operation.
>> > Timeout may happen after writing request (read timeout) - i.e. if
>> > php takes too long to process request, but you have to enlarge it
>> > anyway then.
>>
>> I think the timeouts are a side effect; the problem seems to be
>> between nginx and the FastCGI unix socket. I just ran two quick tests.
>> All possible timeouts for PHP and nginx have been set to 60 seconds,
>> memory and POST size limits are at 1 GB.
>>
>> First, I uploaded a 90 MB file. All went well - the upload finished in
>> ~3 seconds, PHP took ~8 seconds to copy it to the final destination.
>> So 11 seconds total from the time that I hit 'upload' until I got a
>> success notification.
>>
>> Next, I tried to upload a 100 MB file. The upload took ~4 seconds, but
>> then nothing... The server sat for 1 minute with CPU 100% idle. After
>> that, nginx timed out. I had these 2 messages in the error log:
>>
>> 2011/02/17 13:14:21 [warn] 68428#0: *8 a client request body is
>> buffered to a temporary file /srv/upload/tmp/0000000002
>> 2011/02/17 13:15:25 [error] 68428#0: *8 upstream timed out (60:
>> Operation timed out) while sending request to upstream
>>
>> As soon as the second message appeared, the PHP process began
>> executing, copying 20 MB of the uploaded data to the final
>> destination. The remaining 80 MB never made it. In my other tests, the
>> amount of data saved varied between 20 and 60 MB.
>>
>> In other words, it looks like nginx receives the entire request and
>> begins writing it out to the FastCGI socket. After copying a portion
>> of the data, the transfer breaks. Nginx then times out and closes the
>> socket, which causes PHP to begin executing this partially received
>> request. I did verify that AjaXplorer code is not executed until the
>> nginx timeout, so this software is not the problem. The fault is
>> either with PHP, nginx, or the operating system.
>>
>> Any ideas on what could be preventing the entire request from being
>> written out to the FastCGI socket? I have error_log set to 'debug',
>> but the two messages above is all I'm getting.
>
> You have to recompile nginx with --with-debug configure argument
> to get debugging output, see http://nginx.org/en/docs/debugging_log.html
> for details.
>
> You may also want to provide some additional info about your
> operating system and nginx config you use.
>
> Maxim Dounin

Here's the information in case you want to see why unix sockets cause
this problem. I don't think you even need to use the specific software
I was trying to configure. Write a 1-line PHP script that saves some
POSTed variable to a file. Then post more than 64 MB of data and see
if it breaks.

The OS is FreeBSD 7.3-RELEASE-p4 amd64. Nginx 0.8.54 configuration for
AjaXplorer with irrelevant parts removed:

worker_processes 2;
events { worker_connections 512; }

http
{
    include mime.types;
    sendfile on;

    server
    {
        listen 80 accept_filter=httpready;
        server_name localhost;
        root /srv/upload/ajaxplorer;
        client_max_body_size 256m;

        location = / {
            rewrite ^ /index.php last;
        }

        location ~ ^/(?:index|content)\.php$ {
            fastcgi_pass unix:/tmp/php.sock;
            fastcgi_param SCRIPT_FILENAME $request_filename;
            include fastcgi_params;
        }

        location /             { deny  all; }
        location /client/      { allow all; }
        location /client/html/ { deny  all; }
        location /plugins/     { allow all; }
        location ~* \.php$     { deny  all; }
    }
}

- Max