<div dir="ltr"><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">Thanks for programming 101.<br></div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">I'll keep your advice when my goal will be optimizing my current work, which is not currently the case.<br>
</div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">I do not simply want something to work here. I am fully capable of finding workarounds whenever I need/want them.<br>
</div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">I'll leave the 'I do not care how it works as long as it works' motto to business-related goals ;o)<br><br>I need to understand the PHP/Nginx communication. And having searched for it on the Web showed me a lot of unsatisfaying/dirty workarounds, no real solution/explanation.<br>
</div><div class="gmail_default" style="font-size:small;color:rgb(51,51,153)">
If anyone could enlighten me on those Nginx timeouts, I'd be more than glad!<br></div><div class="gmail_extra"><div><font size="1"><span style="color:rgb(102,102,102)">---<br></span><b><span style="color:rgb(102,102,102)">B. R.</span></b><span style="color:rgb(102,102,102)"></span></font></div>
<br><br><div class="gmail_quote">On Sun, May 26, 2013 at 10:44 PM, Steve Holdoway <span dir="ltr"><<a href="mailto:steve@greengecko.co.nz" target="_blank">steve@greengecko.co.nz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
OK, I leave you to it.<br>
<br>
However, asynchronously spawning subprocesses *will* allow you to<br>
parallelise the process. I'd call it design, rather than a workaround,<br>
but there you go (:<br>
<br>
Steve<br>
<div><div>On Sun, 2013-05-26 at 22:38 -0400, B.R. wrote:<br>
> One way or another, even if an external script is called, PHP will<br>
> need to wait for the scripts completion, making the parallelization<br>
> impossible or at least useless (since, to wait for a return code of an<br>
> external script is still blocking).<br>
><br>
><br>
> I am not trying to find a workaround, I need to know how the<br>
> fastcgi_reload_timeout works (if I understood it properly), if I<br>
> properly disabled PHP buffering for my example case and how eventually<br>
> to control those timeouts.<br>
><br>
> I'd like to address the central problem here, not closing my eyes on<br>
> it.<br>
><br>
><br>
> ---<br>
> B. R.<br>
><br>
><br>
> On Sun, May 26, 2013 at 10:24 PM, Steve Holdoway<br>
> <<a href="mailto:steve@greengecko.co.nz" target="_blank">steve@greengecko.co.nz</a>> wrote:<br>
> Surely, you're still serialising the transfer with a loop?<br>
><br>
> On Sun, 2013-05-26 at 22:11 -0400, B.R. wrote:<br>
> > Thanks for your answer.<br>
> ><br>
> > I didn't go into specifics because my problem doesn't rely<br>
> at the<br>
> > application-level logic.<br>
> ><br>
> > What you describe is what my script does already.<br>
> ><br>
> ><br>
> > However in this particular case I have 16 files weighting<br>
> each a few<br>
> > MB which need to be transfered back at once.<br>
> ><br>
> ><br>
> > PHP allocates 30s for each loop turn (far enough to copy the<br>
> file +<br>
> > echo some output message about successes/failed completion).<br>
> ><br>
> > Nginx cuts the execution avec fastcgi_read_timeout time even<br>
> with my<br>
> > efforts to cut down any buffering on PHP side (thus forcing<br>
> the output<br>
> > to be sent to Nginx to reinitialize the timeout counter).<br>
> ><br>
> > That Nginx action is the center of my attention right now.<br>
> How can I<br>
> > get read of it in a scalable fashion (ie no<br>
> fastcgi_read_time =<br>
> > 9999999) ?<br>
> > ---<br>
> > B. R.<br>
> ><br>
> ><br>
> ><br>
> ><br>
> > On Sun, May 26, 2013 at 9:46 PM, Steve Holdoway<br>
> > <<a href="mailto:steve@greengecko.co.nz" target="_blank">steve@greengecko.co.nz</a>> wrote:<br>
> > Write a script that lists the remote files, then<br>
> checks for<br>
> > the<br>
> > existence of the file locally, and copy it if it<br>
> doesn't<br>
> > exist? That way<br>
> > no internal loop is used - use a different exit code<br>
> to note<br>
> > whether<br>
> > there was one copied, or there were none ready.<br>
> ><br>
> > That way you scale for a single file transfer.<br>
> There's nothing<br>
> > to be<br>
> > gained from looping internally - well<br>
> performance-wise that<br>
> > is.<br>
> ><br>
> > Steve<br>
> ><br>
> > On Sun, 2013-05-26 at 21:31 -0400, B.R. wrote:<br>
> > > No ideas?<br>
> > ><br>
> > > ---<br>
> > > B. R.<br>
> > ><br>
> > ><br>
> > > On Sat, May 25, 2013 at 1:01 PM, B.R.<br>
> > <<a href="mailto:reallfqq-nginx@yahoo.fr" target="_blank">reallfqq-nginx@yahoo.fr</a>> wrote:<br>
> > > Hello,<br>
> > ><br>
> > ><br>
> > > I am trying to understand how<br>
> fastcgi_read_timout<br>
> > works in<br>
> > > Nginx.<br>
> > ><br>
> > ><br>
> > > Here is what I wanna do:<br>
> > ><br>
> > > I list files (few MB each) on a distant<br>
> place which<br>
> > I copy one<br>
> > > by one (loop) on the local disk through<br>
> PHP.<br>
> > ><br>
> > > I do not know the amount of files I need<br>
> to copy,<br>
> > thus I do<br>
> > > not know the total amount of time I need<br>
> for the<br>
> > script to<br>
> > > finish its execution. What I know is that<br>
> I can<br>
> > ensure is a<br>
> > > processing time limit per file.<br>
> > ><br>
> > > I would like my script not to be<br>
> forcefully<br>
> > interrupted by<br>
> > > either sides (PHP or Nginx) before<br>
> completion.<br>
> > ><br>
> > ><br>
> > ><br>
> > > What I did so far:<br>
> > ><br>
> > > - PHP has a 'max_execution_time' of 30s<br>
> (default?).<br>
> > In the<br>
> > > loop copying files, I use the<br>
> set_time_limit()<br>
> > procedure to<br>
> > > reinitialize the limit before each file<br>
> copy, hence<br>
> > each file<br>
> > > processing has 30s to go: way enough!<br>
> > ><br>
> > ><br>
> > > - The problem seems to lie on the Nginx<br>
> side, with<br>
> > the<br>
> > > 'fastcgi_read_timeout' configuration<br>
> entry.<br>
> > ><br>
> > > I can't ensure what maximum time I need,<br>
> and I would<br>
> > like not<br>
> > > to use way-off values such as 2 weeks or 1<br>
> year<br>
> > there. ;o)<br>
> > ><br>
> > > What I understood from the documentation<br>
> is that the<br>
> > timeout<br>
> > > is reinitialized after a successful read:<br>
> am I<br>
> > right?<br>
> > ><br>
> > ><br>
> > > The challenge is now to cut any buffering<br>
> occurring<br>
> > on the PHP<br>
> > > side and let Nginx manage it (since the<br>
> buffering<br>
> > will occur<br>
> > > after content is being read from the<br>
> backend). Here<br>
> > is what I<br>
> > > did:<br>
> > ><br>
> > > * PHP's zlib.output_compression is<br>
> deactivated by<br>
> > default in<br>
> > > PHP<br>
> > ><br>
> > > * I deactivated PHP's output_buffering<br>
> (default is<br>
> > 4096 bytes)<br>
> > ><br>
> > > * I am using the PHP flush() procedure at<br>
> the end of<br>
> > each<br>
> > > iteration of the copying loop, after a<br>
> message is<br>
> > written to<br>
> > > the output<br>
> > ><br>
> > ><br>
> > ><br>
> > > Current state:<br>
> > ><br>
> > > * The script seems to still be cut after<br>
> the<br>
> > expiration of the<br>
> > > 'fastcgi_read_timout' limit (confirmed by<br>
> the error<br>
> > log entry<br>
> > > 'upstream timed out (110: Connection timed<br>
> out)<br>
> > while reading<br>
> > > upstream')<br>
> > ><br>
> > > * The PHP loop is entered several times<br>
> since<br>
> > multiple files<br>
> > > have been copied<br>
> > ><br>
> > > * The output sent to the browser is cut<br>
> before any<br>
> > output from<br>
> > > the loop appears<br>
> > ><br>
> > ><br>
> > > It seems that there is still some unwanted<br>
> buffering<br>
> > on the<br>
> > > PHP side.<br>
> > ><br>
> > > I also note that the PHP's flush()<br>
> procedure doesn't<br>
> > seem to<br>
> > > work since the output in the browser<br>
> doesn't contain<br>
> > any<br>
> > > message written after eahc file copy.<br>
> > ><br>
> > ><br>
> > > Am I misunderstanding something about<br>
> Nginx here<br>
> > (especially<br>
> > > about the 'fastcgi_read_timeout'<br>
> directive)?<br>
> > ><br>
> > > Have you any intel/piece of advice on hte<br>
> matter?<br>
> > ><br>
> > > Thanks,<br>
> > ><br>
> > > ---<br>
> > > B. R.<br>
> > ><br>
> > ><br>
> ><br>
> > > _______________________________________________<br>
> > > nginx mailing list<br>
> > > <a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
> > > <a href="http://mailman.nginx.org/mailman/listinfo/nginx" target="_blank">http://mailman.nginx.org/mailman/listinfo/nginx</a><br>
> ><br>
> > --<br>
> > Steve Holdoway BSc(Hons) MNZCS<br>
> <<a href="mailto:steve@greengecko.co.nz" target="_blank">steve@greengecko.co.nz</a>><br>
> > <a href="http://www.greengecko.co.nz" target="_blank">http://www.greengecko.co.nz</a><br>
> > MSN: <a href="mailto:steve@greengecko.co.nz" target="_blank">steve@greengecko.co.nz</a><br>
> > Skype: sholdowa<br>
> ><br>
> > _______________________________________________<br>
> > nginx mailing list<br>
> > <a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
> > <a href="http://mailman.nginx.org/mailman/listinfo/nginx" target="_blank">http://mailman.nginx.org/mailman/listinfo/nginx</a><br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > nginx mailing list<br>
> > <a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
> > <a href="http://mailman.nginx.org/mailman/listinfo/nginx" target="_blank">http://mailman.nginx.org/mailman/listinfo/nginx</a><br>
><br>
> --<br>
> Steve Holdoway BSc(Hons) MNZCS <<a href="mailto:steve@greengecko.co.nz" target="_blank">steve@greengecko.co.nz</a>><br>
> <a href="http://www.greengecko.co.nz" target="_blank">http://www.greengecko.co.nz</a><br>
> MSN: <a href="mailto:steve@greengecko.co.nz" target="_blank">steve@greengecko.co.nz</a><br>
> Skype: sholdowa<br>
><br>
> _______________________________________________<br>
> nginx mailing list<br>
> <a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
> <a href="http://mailman.nginx.org/mailman/listinfo/nginx" target="_blank">http://mailman.nginx.org/mailman/listinfo/nginx</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> nginx mailing list<br>
> <a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
> <a href="http://mailman.nginx.org/mailman/listinfo/nginx" target="_blank">http://mailman.nginx.org/mailman/listinfo/nginx</a><br>
<br>
--<br>
Steve Holdoway BSc(Hons) MNZCS <<a href="mailto:steve@greengecko.co.nz" target="_blank">steve@greengecko.co.nz</a>><br>
<a href="http://www.greengecko.co.nz" target="_blank">http://www.greengecko.co.nz</a><br>
MSN: <a href="mailto:steve@greengecko.co.nz" target="_blank">steve@greengecko.co.nz</a><br>
Skype: sholdowa<br>
<br>
_______________________________________________<br>
nginx mailing list<br>
<a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
<a href="http://mailman.nginx.org/mailman/listinfo/nginx" target="_blank">http://mailman.nginx.org/mailman/listinfo/nginx</a><br>
</div></div></blockquote></div><br></div></div>