Client body buffering with FastCGI
igor at sysoev.ru
Thu Feb 17 22:38:41 MSK 2011
On Feb 17, 2011, at 21:47 , Maxim Khitrov wrote:
> On Thu, Feb 17, 2011 at 12:05 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
>> On Thu, Feb 17, 2011 at 09:48:24AM -0500, Maxim Khitrov wrote:
>>> I'm trying to configure AjaXplorer, a PHP/Ajax file manager, to work
>>> behind nginx 0.8.54 on FreeBSD 7.3. The problem I'm running into is
>>> the inability to upload files more than ~64 MB in size. Ideally, I'd
>>> like to bump that limit up to 1 GB. I realize that HTTP is not ideal
>>> for this, but other transfer methods are not an option.
>>> PHP and nginx are both configured to accept 1 GB POST requests. As far
>>> as I can tell, nginx buffers the contents of the entire upload to disk
>>> before forwarding the request to the FastCGI process. This data is
>>> then read from disk and written back to disk by PHP. The whole
>>> write/read/write cycle is causing a timeout, first in nginx, and then
>>> in the PHP process (though there may also be some other problem that I
>>> haven't figured out yet).
>> Setting bigger timeouts should help. All timeouts in nginx are
>> configurable (proxy_connect_timeout, proxy_send_timeout,
>> proxy_read_timeout - and similar ones for other backend modules).
>> Though it sounds strange that nginx times out while writing
>> request to php as it should reset timer on any write operation.
>> Timeout may happen after writing request (read timeout) - i.e. if
>> php takes too long to process request, but you have to enlarge it
>> anyway then.
> I think the timeouts are a side effect; the problem seems to be
> between nginx and the FastCGI unix socket. I just ran two quick tests.
> All possible timeouts for PHP and nginx have been set to 60 seconds,
> memory and POST size limits are at 1 GB.
> First, I uploaded a 90 MB file. All went well - the upload finished in
> ~3 seconds, PHP took ~8 seconds to copy it to the final destination.
> So 11 seconds total from the time that I hit 'upload' until I got a
> success notification.
> Next, I tried to upload a 100 MB file. The upload took ~4 seconds, but
> then nothing... The server sat for 1 minute with CPU 100% idle. After
> that, nginx timed out. I had these 2 messages in the error log:
> 2011/02/17 13:14:21 [warn] 68428#0: *8 a client request body is
> buffered to a temporary file /srv/upload/tmp/0000000002
> 2011/02/17 13:15:25 [error] 68428#0: *8 upstream timed out (60:
> Operation timed out) while sending request to upstream
> As soon as the second message appeared, the PHP process began
> executing, copying 20 MB of the uploaded data to the final
> destination. The remaining 80 MB never made it. In my other tests, the
> amount of data saved varied between 20 and 60 MB.
> In other words, it looks like nginx receives the entire request and
> begins writing it out to the FastCGI socket. After copying a portion
> of the data, the transfer breaks. Nginx then times out and closes the
> socket, which causes PHP to begin executing this partially received
> request. I did verify that AjaXplorer code is not executed until the
> nginx timeout, so this software is not the problem. The fault is
> either with PHP, nginx, or the operating system.
> Any ideas on what could be preventing the entire request from being
> written out to the FastCGI socket? I have error_log set to 'debug',
> but the two messages above is all I'm getting.
You need to rebuild nginx with debug log:
As to the issue, try to use FastCGI TCP socket instead of unix socket.
More information about the nginx