nginx KTLS and HTTP/2 performance degradation

[Maxim Dounin]

Hello!

On Fri, Dec 03, 2021 at 12:31:50AM +0000, Vadim Fedorenko wrote:

> I would say that current implementation of Kernel TLS in OpenSSL will give
> huge overhead because of additional syscall for every frame and it's header,
> it doesn't matter if it's sendfile or not. Without sendfile it's actually
> 5% worse
> in my tests. That's why it's better to disable Kernel TLS for HTTP/2
> requests
> in Nginx + OpenSSL.

Without SSL_sendfile(), kernel TLS might make sense if TLS 
offloading is supported by a NIC, freeing some CPU power.

With SSL_sendfile(), it is beneficial even without any specialized 
hardware.  Just not with HTTP/2.

> The only solution for this would be implementation of sendmsg()/sendmmsg()
> in OpenSSL and support for such implementation in Nginx together with mmap()
> for files. This solution would have the same performance as sendfile() from
> kernel perspective.

On FreeBSD sendfile() is much more than mmap() and send[m]msg().

Also I tend to think that mmap() is a very risky mechanism for 
sending files, and shouldn't be used by a general-purpose server 
such as nginx, as it kills the server on disk errors.

Rather, solution for HTTP/2 would be to implement in-kernel HTTP/2 
framing along with in-kernel TLS.  Or a more general sendfile() 
implementation, such as sendfilev() on Solaris.  Not sure it worth 
the effort though.

-- 
Maxim Dounin
http://mdounin.ru/