Feature request: Run a script when upstream detected down/up

[Mansoor Peerbhoy]

hello,

i have a small suggestion to make (slightly off topic):

i think this problem would be best solvable by having nginx capable of logging to a file-type other than a regular file.
for instance, if the nginx error_log directive (http://wiki.codemongers.com/NginxMainModule#error_log) could support a TCP/IP or a unix domain socket or a named pipe, then interesting programs can be written around it which may be an easy approach:

a) for instance, having an external program monitor nginx logs for a particular log message (or "event"), is not so easy when the file in question is a regular file. the reason for that is, regular files are *always* ready to read on linux (even if the FD is at EOF), so you can't use an open file descriptor to a regular file in a select()/poll() system call and expect it to work the way you want. (i.e. ready-to-read)
it could be done (tail -f does that), but it's rather messy to have a file-change notification, plus seeking to the appropriate offset. -- with sockets/fifos, these problems can go away. you can set up a socket server to listen on the given socket for connections, and when nginx starts up, the error log initialization code can connect to the tcp or unix domain socket or fifo in question, and then all calls to nginx_log_* will naturally be written to the socket, instead of written to a regular file

in fact, if one adopts the syntax of socat, we might have an nginx conf that looks like:

error_log TCP4:localhost:12345
-or-
error_log UNIX:/path/to/socket

nginx_log_ functions (and the output log format) is structured enough that you can parse the output of that (possibly designate an entire "event." namespace to indicate interesting events, such as upstream went down, memcache server went down, imap connection timed out, etc)

b) with this approach, the problem of counting the number of times an event was fired in a particular time, is solved (because the socket server will receive data in real time, and you can always aggregate similar events, and fork appropriate scripts if you want to).

c) also, if this event approach is taken further, this can also be used as a non-intrusive way of profiling nginx (measuring response times, measuring # of simultaneous connections, etc)

d) and it is also purely in the spirit of a webserver -- apache, if you recollect, can pipe its access log to a program -- that's messy for nginx, but i think, IMHO, that the sockets approach would be better -- this way, nginx need not even be blocked on disk I/O in extreme cases -- although the health of the TCP connection is also relevant here, and the socket server needs to be able to handle large volumes of nginx logs in a short period of time.

so if herr igor thinks it's ok, i think without too much change, this can be coded

regards,
mansoor peerbhoy

ps: having nginx execute hooks directly, is, imho not a very good idea. sure it can be done, but it's not the spirit of nginx




----- Original Message -----
From: "François Battail" <fb at francois.battail.name>
To: nginx at sysoev.ru
Sent: Wednesday, April 30, 2008 11:50:42 AM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi
Subject: Re: Feature request: Run a script when upstream detected down/up

Aleksandar Lazic <al-nginx at ...> writes:

> I think for this the embedded script language
> (perl/neko/lua/python/your_preferd_lang will be very helpfull 

I'm not too sure. If for writing *some bytes* on memory we need to instantiate
an interpretor or a VM and after writing these bytes running a garbage
collector, there's something wrong. 

> Brainstorming:
> 
> 1.) add hook into upstream module, maybe some other modules also 
> 2.) use this hook in $EMBEDDED_SCRIPT (current only perl) to write into
>      $OUTPUT_THING (file/shm/...) 
> 3.) use external program which monitor the $OUTPUT_THING
> 
> The benefit to use a embedded script language is that the user can write
> some infos into the output string and some predefined variables from
> nginx.

Yes, but it is not the idea. It's about monitoring indicators like:

counters on connections state (same as sub_status module)
upstream servers statuses
error counters (like out of fd...)
min max avg on resource allocation
...

That way it will be possible to have useful information for tuning or monitoring
servers running Nginx in an efficient way. It's not designed to 
help debugging an application.

Best regards.