Log Parsing - Near Real Time

[Dennis Jacobfeuerborn]

An alternative is to tail -F (aka. "--follow=name --retry") the log file 
and pipe the output into a script. This allows you to parse the entries as 
they come in and rotate the log file as often as you want independently of 
the parsing script.

Regards,
   Dennis

On 08/01/2011 04:57 PM, Randy Parker wrote:
> My app has a request that opens the log file, fseeks to the end, backs up
> as many bytes as it takes to get to the size the log file was on the last
> similar request by that user, and runs a regex over the novel part to get
> interesting metrics before closing the file.  Since this happens less than
> once per minute, I have not done anything fancy to optimize.
>
> - Randy
>
> On Mon, Aug 1, 2011 at 10:39 AM, Reinis Rozitis <r at roze.lv
> <mailto:r at roze.lv>> wrote:
>
>         I'm looking for a near real-time script to parse log files and
>         insert interesting data into a db.
>         Does anyone know of an existing script to do this?
>
>
>     You can check/try http://www.splunk.com
>
>     rr
>
>
>     _________________________________________________
>     nginx mailing list
>     nginx at nginx.org <mailto:nginx at nginx.org>
>     http://mailman.nginx.org/__mailman/listinfo/nginx
>     <http://mailman.nginx.org/mailman/listinfo/nginx>
>
>
>
>
> --
> http://mobiledyne.com
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx