2013/12/10 MAGNIEN, Thierry email@example.com:
I don't know much about Zeromq performance, but we pushed redis to about 300k commands per second on our production servers (8 cores), and benchmarks on better hardware (24 cores) show it could handle close to 500k/s.
ØMQ should come pretty close or even exceed that. You will require a free core for its process, though. Nevertheless implementing it looks rather straightforward (I've done it in a different product) and with an event consumer (Nginx being the producer) you will get one part of a fantastic local analytics tool, similar to Piwik.
Here's an introduction to authentication and encryption:  http://hintjens.com/blog:49#toc6
Here's how you can prototype the plugin using Lua:
[7a] http://wiki.nginx.org/HttpLuaModule#init_by_lua — initialize context [7b] http://wiki.nginx.org/HttpLuaModule#Data_Sharing_within_an_Nginx_Worker — the place for PUB sockets [7c] https://github.com/mtourne/nginx_log_by_lua/blob/master/conf/nginx.conf — collect data for logging, and send a multipart message here… [7d] https://github.com/Neopallium/lua-zmq/blob/master/examples/client_multipart.... — … that's how the call looks like
Everything else on the consumer will be counters of some sort. You could still use Redis there for small sites, though.
Other noteworthy data structures for the consumer:  http://blog.aggregateknowledge.com/2012/10/25/sketch-of-the-day-hyperloglog-...  http://billmill.org/bloomfilter-tutorial/  http://www.igvita.com/2010/01/06/flow-analysis-time-based-bloom-filters/
I've written this in the hope that other developers of Nginx modules have an idea on where to start (their sockets, for example ;-) ). We should move discussion of further details to a sparkngin group.