nginx KTLS and HTTP/2 performance degradation

[Vadim Fedorenko]

Hello!
I would say that current implementation of Kernel TLS in OpenSSL will give
huge overhead because of additional syscall for every frame and it's header,
it doesn't matter if it's sendfile or not. Without sendfile it's actually
5% worse
in my tests. That's why it's better to disable Kernel TLS for HTTP/2
requests
in Nginx + OpenSSL.
The only solution for this would be implementation of sendmsg()/sendmmsg()
in OpenSSL and support for such implementation in Nginx together with mmap()
for files. This solution would have the same performance as sendfile() from
kernel perspective.

чт, 2 дек. 2021 г. в 13:08, Maxim Dounin <mdounin at mdounin.ru>:

> Hello!
>
> On Thu, Dec 02, 2021 at 02:05:52PM +0200, Lyuben Stoev wrote:
>
> > Hello,
> >      I have tested the nginx with the patch
> > https://hg.nginx.org/nginx/rev/65946a191197 (SSL: SSL_sendfile()
> support
> > with kernel TLS.) following the nginx blog article
> > https://www.nginx.com/blog/improving-nginx-performance-with-kernel-tls/
> > And it sort of works, but I have bad performance when making HTTP/2
> > requests. If I made a HTTP/1.1 request there is 30-35% increase in
> > performance as the Nginx blog article stated, but when I changed the
> > request to use HTTP/2 the request was 40% slower than an ordinary nginx
> > without KTLS enabled. Does anyone have such perfomance degradation with
> > nginx KTLS and HTTP/2? I am using generic setup - Ubuntu 20.04.3 LTS and
> > kernels 5.8.0-63-generic (the same results are with 5.4.0-91-generic).
> > The nginx vritual host is the same as in the Nginx blog article with
> > exception of adding http2 to the listen! OpenSSL 3.0.0 and nginx 1.21.4
> > are used.
> > The KTLS seems to work, because the strace and debug logs show it. Just
> > the sstrange thing is when using HTTP2, the sendfile syscalls look:
> >      write(39, "\0 \0\0\0\0\0\0\1", 9)       = 9
> >      sendfile(39, 131, [1418218] => [1426410], 8192) = 8192
> >      write(39, "\0 \0\0\0\0\0\0\1", 9)       = 9
> >      sendfile(39, 131, [1426410] => [1434602], 8192) = 8192
> >      write(39, "\0 \0\0\0\0\0\0\1", 9)       = 9
> >      sendfile(39, 131, [1434602] => [1442794], 8192) = 8192
> >      write(39, "\0 \0\0\0\0\0\0\1", 9)       = 9
> >      sendfile(39, 131, [1442794] => [1450986], 8192) = 8192
> >      write(39, "\0 \0\0\0\0\0\0\1", 9)       = 9
> >      sendfile(39, 131, [1450986] => [1459178], 8192) = 8192
> >      write(39, "\0 \0\0\0\0\0\0\1", 9)       = 9
> >      sendfile(39, 131, [1459178] => [1467370], 8192) = 8192
> >
> > It is always 8K and there are thousands of sendfile syscalls....
>
> That's expected, because of HTTP/2 framing.  Unfortunately, HTTP/2
> isn't designed to work with sendfile(), and sending large files
> over HTTP/2 require a lot of sendfile() syscalls.  In general, for
> HTTP/2 it is better to keep sendfile() disabled.
>
> --
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20211203/dd07be5f/attachment.htm>