Log Parsing - Near Real Time
neil.mckee.ca at gmail.com
Mon Aug 1 16:24:38 UTC 2011
You could use nginx-sflow-module (binary logging over UDP). The "sflowtool" program can convert this back to a continuous feed of common log file format at the collector.
This way you can receive from multiple servers all sending to the same UDP port. You can also apply random 1-in-N sampling at source as an efficient data-reduction measure if required.
On Aug 1, 2011, at 6:53 AM, John Macleod <jcdmacleod at gmail.com> wrote:
> I'm looking for a near real-time script to parse log files and insert interesting data into a db.
> Does anyone know of an existing script to do this?
> nginx mailing list
> nginx at nginx.org
More information about the nginx