[PATCH] Use BPF to distribute packet to different work thread.

[Maxim Dounin]

Hello!

On Fri, Sep 11, 2020 at 05:41:47AM +0000, Liu, Qiao wrote:

> Hi, Vladimir Homutov:
> The below is our WRK test result output with BPF enable 
> 
>   112 threads and 10000 connections
>   Thread Stats   Avg      Stdev     Max   +/- Stdev
>     Latency   608.23ms  820.71ms  10.00s    87.48%
>     Connect    16.52ms   54.53ms   1.99s    94.73%
>     Delay     153.13ms  182.17ms   2.00s    90.74%
>     Req/Sec   244.79    142.32     1.99k    68.40%
>   Latency Distribution
>   50.00%  293.50ms
>   75.00%  778.33ms
>   90.00%    1.61s
>   99.00%    3.71s
>   99.90%    7.03s
>   99.99%    8.94s
>   Connect Distribution
>   50.00%    1.93ms
>   75.00%    2.85ms
>   90.00%   55.76ms
>   99.00%  229.19ms
>   99.90%  656.79ms
>   99.99%    1.43s
>   Delay Distribution
>   50.00%  110.96ms
>   75.00%  193.67ms
>   90.00%  321.77ms
>   99.00%  959.27ms
>   99.90%    1.57s
>   99.99%    1.91s
> Compared with no BPF but enable reuseport as below
> 
> 112 threads and 10000 connections
>   Thread Stats   Avg      Stdev     Max   +/- Stdev
>     Latency   680.50ms  943.69ms  10.00s    87.18%
>     Connect    58.44ms  238.08ms   2.00s    94.58%
>     Delay     158.84ms  256.28ms   2.00s    90.92%
>     Req/Sec   244.51    151.00     1.41k    69.67%
>   Latency Distribution
>   50.00%  317.61ms
>   75.00%  913.52ms
>   90.00%    1.90s
>   99.00%    4.30s
>   99.90%    6.52s
>   99.99%    8.80s
>   Connect Distribution
>   50.00%    1.88ms
>   75.00%    2.21ms
>   90.00%   55.94ms
>   99.00%    1.45s
>   99.90%    1.95s
>   99.99%    2.00s
>   Delay Distribution
>   50.00%   73.01ms
>   75.00%  190.40ms
>   90.00%  387.01ms
>   99.00%    1.34s
>   99.90%    1.86s
>   99.99%    1.99s
> 
> 
> From the above results, there shows almost 20% percent latency 
> reduction. P99 latency of BPF is 3.71s , but without BPF is 
> 4.3s.

Thank you for the results.

Given that latency stdev is way higher than the average latency, 
I don't think the "20% percent latency reduction" observed is 
statistically significant.  Please try running several tests and 
use ministat(1) to check the results.

Also, the latency values look very high, and request rate very 
low.  What's on the server side?

-- 
Maxim Dounin
http://mdounin.ru/