Using proxy_store under heavy load.

[zepolen]

On Wed, May 27, 2009 at 4:23 AM, Dave Cheney <dave at cheney.net> wrote:
> I suspect that the worker is spending a lot of time in the write() to the disk of the cached file. How many workers are your running ?
>
>> Response times are slow, iotop reports ~10-15MB/s being written to disk by nginx.
>> I'm thinking nginx is probably getting a request for a photo it doesn't already have, and while it is retrieving the file from s3, more requests come in for the same file, meaning more round trips and more temp files being created.

It was a typo in the path where nginx was supposed to find the files
that had been stored. As a result every single file was being
retrieved from the backend, worse, it was being written to disk, only
to be discarded.

It made perfect sense in retrospect as outgoing eth traffic was also
stuck at about ~15MB/s, and incoming had jumped to the same level,
unfortunately had to wait for the graphs before I could realise the
problem.