nginx KTLS and HTTP/2 performance degradation

[Noam Cvikel]

Hello!

Late to the party. We've seen the same results over here when using sendfile
with HTTP/2. You can increase it from 8k to 16 frames using http2_chunk_size
but that still won't be good performance when dealing with files that aren't
tiny.

Really glad I found this thread, and I appreciate the clarification Maxim.
One thing I ponder though, shouldn't it be beneficial to have a directive to
toggle Sendfile on/off specifically for HTTP/2?
It would be nice to allow a location to Sendfile over HTTP/1.1 by still
respond to HTTP/2 requests.

Best,

Noam Cvikel
Qwilt | Delivery Software Engineer | noamc at qwilt.com


-----Original Message-----
From: nginx-devel [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Maxim
Dounin
Sent: Thursday, December 2, 2021 15:07
To: nginx-devel at nginx.org
Subject: Re: nginx KTLS and HTTP/2 performance degradation

Hello!

On Thu, Dec 02, 2021 at 02:05:52PM +0200, Lyuben Stoev wrote:

> Hello,
>      I have tested the nginx with the patch
> https://hg.nginx.org/nginx/rev/65946a191197 (SSL: SSL_sendfile()
> support with kernel TLS.) following the nginx blog article
> https://www.nginx.com/blog/improving-nginx-performance-with-kernel-tls
> / And it sort of works, but I have bad performance when making HTTP/2
> requests. If I made a HTTP/1.1 request there is 30-35% increase in
> performance as the Nginx blog article stated, but when I changed the
> request to use HTTP/2 the request was 40% slower than an ordinary
> nginx without KTLS enabled. Does anyone have such perfomance
> degradation with nginx KTLS and HTTP/2? I am using generic setup -
> Ubuntu 20.04.3 LTS and kernels 5.8.0-63-generic (the same results are with
> 5.4.0-91-generic).
> The nginx vritual host is the same as in the Nginx blog article with
> exception of adding http2 to the listen! OpenSSL 3.0.0 and nginx
> 1.21.4 are used.
> The KTLS seems to work, because the strace and debug logs show it.
> Just the sstrange thing is when using HTTP2, the sendfile syscalls look:
>      write(39, "\0 \0\0\0\0\0\0\1", 9)       = 9
>      sendfile(39, 131, [1418218] => [1426410], 8192) = 8192
>      write(39, "\0 \0\0\0\0\0\0\1", 9)       = 9
>      sendfile(39, 131, [1426410] => [1434602], 8192) = 8192
>      write(39, "\0 \0\0\0\0\0\0\1", 9)       = 9
>      sendfile(39, 131, [1434602] => [1442794], 8192) = 8192
>      write(39, "\0 \0\0\0\0\0\0\1", 9)       = 9
>      sendfile(39, 131, [1442794] => [1450986], 8192) = 8192
>      write(39, "\0 \0\0\0\0\0\0\1", 9)       = 9
>      sendfile(39, 131, [1450986] => [1459178], 8192) = 8192
>      write(39, "\0 \0\0\0\0\0\0\1", 9)       = 9
>      sendfile(39, 131, [1459178] => [1467370], 8192) = 8192
>
> It is always 8K and there are thousands of sendfile syscalls....

That's expected, because of HTTP/2 framing.  Unfortunately, HTTP/2 isn't
designed to work with sendfile(), and sending large files over HTTP/2
require a lot of sendfile() syscalls.  In general, for
HTTP/2 it is better to keep sendfile() disabled.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx-devel mailing list
nginx-devel at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel