Add support for buffering is scripted logs

[Alexey Ivanov]

You are indeed right about the open_file_cache, but you still left with a sync write(2) that can block, and blocking of eventloop is a major source of latency variance (unless you use Flash-based drives of course).

> On Aug 14, 2017, at 2:24 PM, Eran Kornblau <eran.kornblau at kaltura.com> wrote:
> 
>> -----Original Message-----
>> From: nginx-devel [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Alexey Ivanov
>> Sent: Monday, August 14, 2017 9:25 PM
>> To: nginx-devel at nginx.org
>> Subject: Re: Add support for buffering is scripted logs
>> 
>> using syslog for that particular usecase seem way more elegant, customizable, and simple. As a side bonus you won't block event loop on vfs operations (open/write/close).
>> 
> Thanks Alexey.
> 
> Regarding the last point about performance, I ran a quick test, compared -
> 1. Nginx writing to gzip access log -
> access_log /var/log/nginx/access_log.gz main gzip flush=5m;
> 2. Nginx writing to local rsyslog over UDP, which in turn writes to local file -
> access_log syslog:server=127.0.0.1 main;
> 3. Nginx writing to remote rsyslog on the same LAN over UDP -
> access_log syslog:server=192.168.11.94 main;
> 
> Ran this command (execute 10 apache bench tests, take the median value):
> (for i in `seq 1 10`; do ab -c 1000 -n 100000 127.0.0.1/alive.html 2>/dev/null | grep '^  50%' ; done) | awk '{print $2}' | xargs
> 
> The nginx location being tested simply does 'return 200 "hello"'.
> 
> Results are:
> 1. 4 6 5 6 3 5 3 5 7 5
> 2. 6 7 5 7 6 7 7 7 7 7
> 3. 5 6 6 6 6 6 6 6 4 6
> 
> The numbers fluctuated a bit from one run to another (did it a few times), but the overall trend was the same -
> syslog is slower than having nginx write with gzip. The difference is not dramatic, but it's visible.
> This makes sense since with buffered write, nginx only has to perform a syscall once every X messages,
> most messages are written without any syscalls.
> Regarding what you wrote about open/write/close, when using variables (even before this patch),
> it is possible to enable open file cache on log files, so you only have 'write'.
> 
> Eran
> 
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20170814/42033faa/attachment.bin>