G-WAN assumptions: Part of truth/lies?

[B.R.]

Hello,

Documentating myself on proper benchmarking, I ran into the following page:
http://gwan.com/en_apachebench_httperf.html

Their conclusion is that their product is the best of all. Well, 'of
course' one might say... ;o)

What surprised me most that they claim to use less resources AND perform
better. That particularly strikes me because usually ot favor one side, you
take blows on the other one.

To me, the problem of such tests is that they are a mix of
realistic/unrealistic behaviors, the first being invoked to justify useful
conclusions, the latter to make a specific environment so that features
from the Web server (as opposed to other components of the infrastructure)
are tested.

They are arrogant enough to claim theirs is bigger and paranoid enough to
call almost every other benchmark biased or coming from haste/FUD
campaigns. That is only OK if they are as pure as the driven snow...

I need expert eyes of yours to determine to which end those claims are
grounded.

Particular points:
- Is their nginx configuration <http://gwan.com/source/nginx.conf> suitable
for valid benchmark results?
- Why is your wrk test tool built in such way in pre-establishes TCP?
- Why is nginx pre-allocating resources so its memory footprint is large
when connections are pre-established? I thought nginx event-based system
was allocating resources on-the-fly, as G-WAN seems to be doing it. (cf.
'The (long) story of Nginx's "wrk"' section)
- Why is wrk (in G-WAN's opinion) 'too slow under 10,000 simultaneous
connections'? (cf. 'The (long) story of Nginx's "wrk"' section)
---
*B. R.*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20150208/8510893c/attachment.html>