question about traffic statistics on each vhost,is it po

[mike]

largest $bytes_sent i seem to be getting is 83,802 bytes...

also, i want to log both inbound and outbound http traffic - file
uploads, etc... is there a combination of $bytes_sent i need + another
variable etc? to be able to track all the bytes in and out of nginx
and relate it to a host: header (if one exists)

p.s. i also have a LOT of 0 byte $bytes_sent too



On Wed, Oct 8, 2008 at 12:07 AM, mike <mike503 at gmail.com> wrote:
> This is what I have right now (set on a global http {} level)
>
> log_format traffic '$http_host|$bytes_sent|$time_local|$remote_addr';
> access_log /var/log/nginx/traffic traffic;
>
> Problem is, I don't seem to be getting large requests - or am I, but
> the requests are split up into buffers and logged as such? So one 1MB
> file might show up 8x in the log if each 128k was logged separately?
>
>
> On Wed, Oct 1, 2008 at 1:43 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
>> Hello!
>>
>> On Wed, Oct 01, 2008 at 12:06:11PM -0700, mike wrote:
>>
>>> On Wed, Oct 1, 2008 at 9:43 AM, Maxim Dounin <mdounin at mdounin.ru> wrote:
>>>>
>>>> Hello!
>>>> Why not just log needed numbers ($bytes_sent) to access_log and
>>>> post-process
>>>> it with anything you want?
>>>
>>> I might be forced to do this right now. That means millions of lines
>>> per day on 3 servers to combine, merge and calculate...
>>>
>>> I'm sure it would be simple for someone to write a module that dumps
>>> out Host: header stats to a file every so often (one per day) ...
>>> *fingers crossed*
>>
>> It's not simple "dump out Host: header", it's shared memory, locking mutex
>> for each request, loosing stats on binary upgrade, etc, etc.
>>
>> I see no reason why this can't be done from access_logs by simple perl
>> script - either running side-by-side with nginx but perfectly restarable
>> whenever you want, or periodically run at low-load periods.
>>
>> We used to run similar script for calculation of average
>> $upstream_response_time for monitoring purposes and as far as I recall it
>> took less than 1% cpu on not-really-fast machine with more than 10 mln log
>> lines per day.
>>
>> Maxim Dounin
>>
>>
>