Is it possible to monitor the fair proxy balancer?

[Grzegorz Nosek]

Hi,

On pon, cze 30, 2008 at 09:23:30 +0200, Brice Figureau wrote:
> Many thanks for doing the hard work. It looks like good from my quick
> glance. I'll add this to the upload progress module (if there are some
> monitoring needs there, but at least showing debug information could be
> great).

Thanks :) Not really hard, just tedious copy&paste. The real meat of the
patch is only a few lines.

As for monitoring the upload progress module, off the top of my head:
 - number of concurrent uploads
 - speed + total size of every upload

:)

> > The patchset is rather ugly because it requires adding a field to every
> > HTTP
> > module definition, thus breaking 3rd party modules. However, the
> > required changes are trivial (as you can see in the patch).
> 
> Yes, I'm fine with this, although I envisioned something reverse ie
> modules declaring fields/counters (kind of a mib if you see what I mean)
> to the "monitoring engine", and the status module displaying this

I know what you wanted to do. Maybe it is a good idea but it would have
to allow registering/unregistering variables on the fly. E.g.
upstream_fair keeps a tree of upstreams, where every element lives
almost but not exactly as long as an nginx cycle (they get created and
destroyed with the first request of the new cycle).

> information. But your scheme has the big advantage to be simple, and the
> displaying of the information is up to the module.

Your approach would lead to more rigid status reports, which would be
easier to parse (and could eventually enable selective status reports
for ultra low-overhead monitoring, e.g.
/status/upstream_fair/mongrel_cluster1/10.0.0.1/nreq). If the free-form
approach gets into nginx, I think it would be nice to create a standard
format of the reports, and maybe even some Perl/Ruby/whatever glue to
parse it in a uniform way.

> > I considered adding support for all modules (not only HTTP ones) and
> > changing the ngx_module_t structure instead (which has some room to
> > spare and would only require a few macro changes) but I think it's up to
> > Igor to decide.
> >
> > Please have a look and let me know what you think.
> 
> Yes, that's a good work. I hope Igor could accept such kind of patch to be
> part of nginx.

So, Igor, what's your opinion? :) The patch can be made less intrusive
by hooking into ngx_module_t and using e.g. spare_hook0 plus new macro
(e.g. NGX_MODULE_V2_PADDING with one fewer zero). Then, only modules
which do implement the ->report_status hook would have to be changed.

However, then I'd probably also change the signature not to take an
ngx_http_request_t*, but only an ngx_pool_t* for allocation (to get rid
of HTTP-specific stuff).

Best regards,
 Grzegorz Nosek