Status of Nginx centralised logging

[Jonathan Matthews]

On 7 June 2012 00:03, Matthieu Tourne <matthieu.tourne at gmail.com> wrote:
> I think this thread from a few days ago might answer some of your questions :
> http://forum.nginx.org/read.php?2,227234

Yes, it was glancing over this thread that made me think I needed to
raise this issue kinda sorta formally, and ask for some official
recognition of this relatively important issue.

> We use log_by_lua to aggregate data, and then fetch the aggregated
> data from another location.
> In turns this feeds into a bigger centralized system (OpenTSDB).
> This way we don't have to output one log line, or hit one logging
> endpoint for each request that goes through.
>
> One interesting thing we've noticed while developing this is that even
> an error in our logging code really doesn't affect serving pages, so
> this might be the safest use case for 3rd party modules you could
> imagine.

Interesting, thank you. [ I've so far avoided enabling lua in my
production Nginx instances as it's almost /too/ powerful. If our
systems team (or devs!) get the idea we can do /anything/ we like in
the reverse proxy layer in front of our app servers, I'm concerned
we'll start to add too much logic in there that should actually live
in the apps :-) ]

I'll definitely take a look at log_by_lua, however I still think we
should get the centralised logging issue recognised officially as a
area where Nginx is deficient, along with suggestions for bringing it
up to spec. Nginx is a vital part of many people's infrastructures
now: it's a shame it's behind the (devops'y!) times in this area.

> Currently your only other official alternative to get data, besides
> the access_log is the stub_status module :
> http://wiki.nginx.org/HttpStubStatusModule

Appreciated but, not being per-request, it isn't really in the same
space as the other things we're discussing.

Cheers,
Jonathan
-- 
Jonathan Matthews
Oxford, London, UK
http://www.jpluscplusm.com/contact.html