question about traffic statistics on each vhost,is it po

[mike]

This is what I have right now (set on a global http {} level)

log_format traffic '$http_host|$bytes_sent|$time_local|$remote_addr';
access_log /var/log/nginx/traffic traffic;

Problem is, I don't seem to be getting large requests - or am I, but
the requests are split up into buffers and logged as such? So one 1MB
file might show up 8x in the log if each 128k was logged separately?


On Wed, Oct 1, 2008 at 1:43 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> Hello!
>
> On Wed, Oct 01, 2008 at 12:06:11PM -0700, mike wrote:
>
>> On Wed, Oct 1, 2008 at 9:43 AM, Maxim Dounin <mdounin at mdounin.ru> wrote:
>>>
>>> Hello!
>>> Why not just log needed numbers ($bytes_sent) to access_log and
>>> post-process
>>> it with anything you want?
>>
>> I might be forced to do this right now. That means millions of lines
>> per day on 3 servers to combine, merge and calculate...
>>
>> I'm sure it would be simple for someone to write a module that dumps
>> out Host: header stats to a file every so often (one per day) ...
>> *fingers crossed*
>
> It's not simple "dump out Host: header", it's shared memory, locking mutex
> for each request, loosing stats on binary upgrade, etc, etc.
>
> I see no reason why this can't be done from access_logs by simple perl
> script - either running side-by-side with nginx but perfectly restarable
> whenever you want, or periodically run at low-load periods.
>
> We used to run similar script for calculation of average
> $upstream_response_time for monitoring purposes and as far as I recall it
> took less than 1% cpu on not-really-fast machine with more than 10 mln log
> lines per day.
>
> Maxim Dounin
>
>