[PATCH] Use BPF to distribute packet to different work thread.

[Liu, Qiao]

Hi, Maxim Dounin:
Thanks for your reply, this server is random selected, we just do BPF and no-BPF test, I think the latency based on server configuration, not related with BPF patch, also the NIC
of the server is Mellanox, not ADQ capable hardware , we will do more test
Thanks
LQ

-----Original Message-----
From: nginx-devel <nginx-devel-bounces at nginx.org> On Behalf Of Maxim Dounin
Sent: Monday, September 14, 2020 7:40 AM
To: nginx-devel at nginx.org
Subject: Re: [PATCH] Use BPF to distribute packet to different work thread.

Hello!

On Fri, Sep 11, 2020 at 05:41:47AM +0000, Liu, Qiao wrote:

> Hi, Vladimir Homutov:
> The below is our WRK test result output with BPF enable
> 
>   112 threads and 10000 connections
>   Thread Stats   Avg      Stdev     Max   +/- Stdev
>     Latency   608.23ms  820.71ms  10.00s    87.48%
>     Connect    16.52ms   54.53ms   1.99s    94.73%
>     Delay     153.13ms  182.17ms   2.00s    90.74%
>     Req/Sec   244.79    142.32     1.99k    68.40%
>   Latency Distribution
>   50.00%  293.50ms
>   75.00%  778.33ms
>   90.00%    1.61s
>   99.00%    3.71s
>   99.90%    7.03s
>   99.99%    8.94s
>   Connect Distribution
>   50.00%    1.93ms
>   75.00%    2.85ms
>   90.00%   55.76ms
>   99.00%  229.19ms
>   99.90%  656.79ms
>   99.99%    1.43s
>   Delay Distribution
>   50.00%  110.96ms
>   75.00%  193.67ms
>   90.00%  321.77ms
>   99.00%  959.27ms
>   99.90%    1.57s
>   99.99%    1.91s
> Compared with no BPF but enable reuseport as below
> 
> 112 threads and 10000 connections
>   Thread Stats   Avg      Stdev     Max   +/- Stdev
>     Latency   680.50ms  943.69ms  10.00s    87.18%
>     Connect    58.44ms  238.08ms   2.00s    94.58%
>     Delay     158.84ms  256.28ms   2.00s    90.92%
>     Req/Sec   244.51    151.00     1.41k    69.67%
>   Latency Distribution
>   50.00%  317.61ms
>   75.00%  913.52ms
>   90.00%    1.90s
>   99.00%    4.30s
>   99.90%    6.52s
>   99.99%    8.80s
>   Connect Distribution
>   50.00%    1.88ms
>   75.00%    2.21ms
>   90.00%   55.94ms
>   99.00%    1.45s
>   99.90%    1.95s
>   99.99%    2.00s
>   Delay Distribution
>   50.00%   73.01ms
>   75.00%  190.40ms
>   90.00%  387.01ms
>   99.00%    1.34s
>   99.90%    1.86s
>   99.99%    1.99s
> 
> 
> From the above results, there shows almost 20% percent latency 
> reduction. P99 latency of BPF is 3.71s , but without BPF is 4.3s.

Thank you for the results.

Given that latency stdev is way higher than the average latency, I don't think the "20% percent latency reduction" observed is statistically significant.  Please try running several tests and use ministat(1) to check the results.

Also, the latency values look very high, and request rate very low.  What's on the server side?

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx-devel mailing list
nginx-devel at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel