<div dir="ltr">Piotr,<div>  You've worked on something similar in the past what's your opinion on the performance of a zeromq module vs redis?</div><div>-Steve</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On Tue, Dec 10, 2013 at 7:00 AM, W-Mark Kubacki <span dir="ltr"><<a href="mailto:wmark+nginx@hurrikane.de" target="_blank">wmark+nginx@hurrikane.de</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
2013/12/10 MAGNIEN, Thierry <<a href="mailto:thierry.magnien@sfr.com">thierry.magnien@sfr.com</a>>:<br>
<div class="im">><br>
> I don't know much about Zeromq performance, but we pushed redis to about 300k commands per second on our production servers (8 cores), and benchmarks on better hardware (24 cores) show it could handle close to 500k/s.<br>

<br>
</div>ØMQ should come pretty close or even exceed that. You will require a<br>
free core for its process, though.<br>
      Nevertheless implementing it looks rather straightforward (I've<br>
done it in a different product) and with an event consumer (Nginx<br>
being the producer) you will get one part of a fantastic local<br>
analytics tool, similar to Piwik.<br>
<br>
Here's an introduction to authentication and encryption:<br>
[6] <a href="http://hintjens.com/blog:49#toc6" target="_blank">http://hintjens.com/blog:49#toc6</a><br>
<br>
Here's how you can prototype the plugin using Lua:<br>
<br>
[7a] <a href="http://wiki.nginx.org/HttpLuaModule#init_by_lua" target="_blank">http://wiki.nginx.org/HttpLuaModule#init_by_lua</a> — initialize context<br>
[7b] <a href="http://wiki.nginx.org/HttpLuaModule#Data_Sharing_within_an_Nginx_Worker" target="_blank">http://wiki.nginx.org/HttpLuaModule#Data_Sharing_within_an_Nginx_Worker</a><br>
— the place for PUB sockets<br>
[7c] <a href="https://github.com/mtourne/nginx_log_by_lua/blob/master/conf/nginx.conf" target="_blank">https://github.com/mtourne/nginx_log_by_lua/blob/master/conf/nginx.conf</a><br>
— collect data for logging, and send a multipart message here…<br>
[7d] <a href="https://github.com/Neopallium/lua-zmq/blob/master/examples/client_multipart.lua" target="_blank">https://github.com/Neopallium/lua-zmq/blob/master/examples/client_multipart.lua</a><br>
— … that's how the call looks like<br>
<br>
Everything else on the consumer will be counters of some sort. You<br>
could still use Redis there for small sites, though.<br>
<br>
Other noteworthy data structures for the consumer:<br>
[8] <a href="http://blog.aggregateknowledge.com/2012/10/25/sketch-of-the-day-hyperloglog-cornerstone-of-a-big-data-infrastructure/" target="_blank">http://blog.aggregateknowledge.com/2012/10/25/sketch-of-the-day-hyperloglog-cornerstone-of-a-big-data-infrastructure/</a><br>

[9] <a href="http://billmill.org/bloomfilter-tutorial/" target="_blank">http://billmill.org/bloomfilter-tutorial/</a><br>
[10] <a href="http://www.igvita.com/2010/01/06/flow-analysis-time-based-bloom-filters/" target="_blank">http://www.igvita.com/2010/01/06/flow-analysis-time-based-bloom-filters/</a><br>
<br>
I've written this in the hope that other developers of Nginx modules<br>
have an idea on where to start (their sockets, for example ;-) ). We<br>
should move discussion of further details to a sparkngin group.<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Mark<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
_______________________________________________<br>
nginx-devel mailing list<br>
<a href="mailto:nginx-devel@nginx.org">nginx-devel@nginx.org</a><br>
<a href="http://mailman.nginx.org/mailman/listinfo/nginx-devel" target="_blank">http://mailman.nginx.org/mailman/listinfo/nginx-devel</a></div></div></blockquote></div><br></div>