Trying to understand benchmarking results

[Sergey Brester]

Hi,

Dynamic content? You can measure static also (exapmle URL to any image),
to see difference to your another test. 

So if it is fast enough, your post does not apply to nginx (so check
your backend / upstream and please contact manufacturer of it). 

In addition to database, hdd, cpu etc, it can be "ab" self: 

- "ab" does not support http 1.1 (so it can be for example not
keep-alive - too many connections);
- sometimes "ab" can not correctly measure non 200 requests (I don't
remember what it was) - and all your requests are non 200 (not OK); 

Can you test it again with something else, for example wrk [5] or siege
benchmark [6]? 

Regards, 

sebres. 

Am 29.04.2015 18:30, schrieb Bill Appleton: 

> Hi All,
> 
> I ran some benchmarks on our DreamFactory product running on NGINX using Apache Benchmark
> 
> DreamFactory is a REST API platform, but from a server point of view it should look more or less like a simple WordPress website
> 
> see the graphical results here: www.DREAMFACTORY.COM/PUBLICATIONS/COMPARISON.PNG [2] the red lines show the average and maximum response time for 500 requests at different levels of concurrency running on a "medium" aws server with 1 CPU
> 
> the blue lines show the average and maximum response time for 500 requests at different levels of concurrency running on a "large" aws server with 2 CPUs
> 
> Does anyone know why at about 100 concurrent requests we start to see some requests taking much longer to finish?
> 
> Is this too much work for the processor? The database is overloaded? There is some server setting that limits concurrency?
> 
> I have also included one of the benchmark reports below, as you can see most of the requests are handled promptly but about 5% are deferred and take a much longer time....
> 
> Any help would be appreciated!
> 
> Thanks,
> 
> Bill Appleton 
> 
> www.dreamfactory.com [3]
> 
> Server Software: nginx
> Server Hostname: next.cloud.dreamfactory.com [4]
> Server Port: 80
> Document Length: 126 bytes
> 
> Concurrency Level: 200
> Time taken for tests: 16.147 seconds
> Complete requests: 500
> Failed requests: 0
> Non-2xx responses: 500
> Total transferred: 298000 bytes
> HTML transferred: 63000 bytes
> Requests per second: 30.97 [#/sec] (mean)
> Time per request: 6458.755 [ms] (mean)
> Time per request: 32.294 [ms] (mean, across all concurrent requests)
> Transfer rate: 18.02 [Kbytes/sec] received
> 
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 85 209 208.7 91 725
> Processing: 146 3605 2629.2 3096 15633
> Waiting: 145 3604 2629.2 3095 15633
> Total: 243 3814 2725.4 3189 16143
> 
> Percentage of the requests served within a certain time (ms)
> 50% 3189
> 66% 3264
> 75% 3831
> 80% 4175
> 90% 6766
> 95% 10568
> 98% 11114
> 99% 16069
> 100% 16143 (longest request)
> 
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel [1]
 

Links:
------
[1] http://mailman.nginx.org/mailman/listinfo/nginx-devel
[2] http://dreamfactory.com/publications/comparison.png
[3] http://dreamfactory.com
[4] http://next.cloud.dreamfactory.com
[5] https://github.com/wg/wrk
[6] https://www.joedog.org/siege-home/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20150429/1628a082/attachment.html>