<div dir="ltr"><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">Thanks for your answer.<br><br>I didn't go into specifics because my problem doesn't rely at the application-level logic.<br></div>
<div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">What you describe is what my script does already.<br><br></div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">However in this particular case I have 16 files weighting each a few MB which need to be transfered back at once.<br>
<br></div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">PHP allocates 30s for each loop turn (far enough to copy the file + echo some output message about successes/failed completion).<br></div>
<div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">
Nginx cuts the execution avec fastcgi_read_timeout time even with my efforts to cut down any buffering on PHP side (thus forcing the output to be sent to Nginx to reinitialize the timeout counter).<br></div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">
That Nginx action is the center of my attention right now. How can I get read of it in a scalable fashion (ie no fastcgi_read_time = 9999999) ?<br clear="all"><div><font size="1"><span style="color:rgb(102,102,102)">---<br>
</span><b><span style="color:rgb(102,102,102)">B. R.</span></b><span style="color:rgb(102,102,102)"></span></font></div>
<font size="1"><b><span style="color:rgb(102,102,102)"><br></span></b></font></div>
<br><br><div class="gmail_quote">On Sun, May 26, 2013 at 9:46 PM, Steve Holdoway <span dir="ltr"><<a href="mailto:steve@greengecko.co.nz" target="_blank">steve@greengecko.co.nz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Write a script that lists the remote files, then checks for the<br>
existence of the file locally, and copy it if it doesn't exist? That way<br>
no internal loop is used - use a different exit code to note whether<br>
there was one copied, or there were none ready.<br>
<br>
That way you scale for a single file transfer. There's nothing to be<br>
gained from looping internally - well performance-wise that is.<br>
<br>
Steve<br>
<div><div><br>
On Sun, 2013-05-26 at 21:31 -0400, B.R. wrote:<br>
> No ideas?<br>
><br>
> ---<br>
> B. R.<br>
><br>
><br>
> On Sat, May 25, 2013 at 1:01 PM, B.R. <<a href="mailto:reallfqq-nginx@yahoo.fr" target="_blank">reallfqq-nginx@yahoo.fr</a>> wrote:<br>
> Hello,<br>
><br>
><br>
> I am trying to understand how fastcgi_read_timout works in<br>
> Nginx.<br>
><br>
><br>
> Here is what I wanna do:<br>
><br>
> I list files (few MB each) on a distant place which I copy one<br>
> by one (loop) on the local disk through PHP.<br>
><br>
> I do not know the amount of files I need to copy, thus I do<br>
> not know the total amount of time I need for the script to<br>
> finish its execution. What I know is that I can ensure is a<br>
> processing time limit per file.<br>
><br>
> I would like my script not to be forcefully interrupted by<br>
> either sides (PHP or Nginx) before completion.<br>
><br>
><br>
><br>
> What I did so far:<br>
><br>
> - PHP has a 'max_execution_time' of 30s (default?). In the<br>
> loop copying files, I use the set_time_limit() procedure to<br>
> reinitialize the limit before each file copy, hence each file<br>
> processing has 30s to go: way enough!<br>
><br>
><br>
> - The problem seems to lie on the Nginx side, with the<br>
> 'fastcgi_read_timeout' configuration entry.<br>
><br>
> I can't ensure what maximum time I need, and I would like not<br>
> to use way-off values such as 2 weeks or 1 year there. ;o)<br>
><br>
> What I understood from the documentation is that the timeout<br>
> is reinitialized after a successful read: am I right?<br>
><br>
><br>
> The challenge is now to cut any buffering occurring on the PHP<br>
> side and let Nginx manage it (since the buffering will occur<br>
> after content is being read from the backend). Here is what I<br>
> did:<br>
><br>
> * PHP's zlib.output_compression is deactivated by default in<br>
> PHP<br>
><br>
> * I deactivated PHP's output_buffering (default is 4096 bytes)<br>
><br>
> * I am using the PHP flush() procedure at the end of each<br>
> iteration of the copying loop, after a message is written to<br>
> the output<br>
><br>
><br>
><br>
> Current state:<br>
><br>
> * The script seems to still be cut after the expiration of the<br>
> 'fastcgi_read_timout' limit (confirmed by the error log entry<br>
> 'upstream timed out (110: Connection timed out) while reading<br>
> upstream')<br>
><br>
> * The PHP loop is entered several times since multiple files<br>
> have been copied<br>
><br>
> * The output sent to the browser is cut before any output from<br>
> the loop appears<br>
><br>
><br>
> It seems that there is still some unwanted buffering on the<br>
> PHP side.<br>
><br>
> I also note that the PHP's flush() procedure doesn't seem to<br>
> work since the output in the browser doesn't contain any<br>
> message written after eahc file copy.<br>
><br>
><br>
> Am I misunderstanding something about Nginx here (especially<br>
> about the 'fastcgi_read_timeout' directive)?<br>
><br>
> Have you any intel/piece of advice on hte matter?<br>
><br>
> Thanks,<br>
><br>
> ---<br>
> B. R.<br>
><br>
><br>
</div></div>> _______________________________________________<br>
> nginx mailing list<br>
> <a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
> <a href="http://mailman.nginx.org/mailman/listinfo/nginx" target="_blank">http://mailman.nginx.org/mailman/listinfo/nginx</a><br>
<span><font color="#888888"><br>
--<br>
Steve Holdoway BSc(Hons) MNZCS <<a href="mailto:steve@greengecko.co.nz" target="_blank">steve@greengecko.co.nz</a>><br>
<a href="http://www.greengecko.co.nz" target="_blank">http://www.greengecko.co.nz</a><br>
MSN: <a href="mailto:steve@greengecko.co.nz" target="_blank">steve@greengecko.co.nz</a><br>
Skype: sholdowa<br>
<br>
_______________________________________________<br>
nginx mailing list<br>
<a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
<a href="http://mailman.nginx.org/mailman/listinfo/nginx" target="_blank">http://mailman.nginx.org/mailman/listinfo/nginx</a><br>
</font></span></blockquote></div><br></div>