Log Parsing - Near Real Time

[Neil McKee]

You could use nginx-sflow-module (binary logging over UDP).  The "sflowtool" program can convert this back to a continuous feed of common log file format at the collector.

  This way you can receive from multiple servers all sending to the same UDP port.  You can also apply random 1-in-N sampling at source as an efficient data-reduction measure if required.

Neil Mckee



On Aug 1, 2011, at 6:53 AM, John Macleod <jcdmacleod at gmail.com> wrote:

> I'm looking for a near real-time script to parse log files and insert interesting data into a db.
> 
> Does anyone know of an existing script to do this?
> 
> John
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx