<div dir="ltr"><div>Hello!</div><div>I would say that current implementation of Kernel TLS in OpenSSL will give</div><div>huge overhead because of additional syscall for every frame and it's header,</div><div>it doesn't matter if it's sendfile or not. Without sendfile it's actually 5% worse</div><div>in my tests. That's why it's better to disable Kernel TLS for HTTP/2 requests</div><div>in Nginx + OpenSSL.</div><div>The only solution for this would be implementation of sendmsg()/sendmmsg()</div><div>in OpenSSL and support for such implementation in Nginx together with mmap()</div><div>for files. This solution would have the same performance as sendfile() from</div><div>kernel perspective.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">чт, 2 дек. 2021 г. в 13:08, Maxim Dounin <<a href="mailto:mdounin@mdounin.ru">mdounin@mdounin.ru</a>>:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello!<br>
<br>
On Thu, Dec 02, 2021 at 02:05:52PM +0200, Lyuben Stoev wrote:<br>
<br>
> Hello,<br>
> I have tested the nginx with the patch <br>
> <a href="https://hg.nginx.org/nginx/rev/65946a191197" rel="noreferrer" target="_blank">https://hg.nginx.org/nginx/rev/65946a191197</a> (SSL: SSL_sendfile() support <br>
> with kernel TLS.) following the nginx blog article <br>
> <a href="https://www.nginx.com/blog/improving-nginx-performance-with-kernel-tls/" rel="noreferrer" target="_blank">https://www.nginx.com/blog/improving-nginx-performance-with-kernel-tls/</a> <br>
> And it sort of works, but I have bad performance when making HTTP/2 <br>
> requests. If I made a HTTP/1.1 request there is 30-35% increase in <br>
> performance as the Nginx blog article stated, but when I changed the <br>
> request to use HTTP/2 the request was 40% slower than an ordinary nginx <br>
> without KTLS enabled. Does anyone have such perfomance degradation with <br>
> nginx KTLS and HTTP/2? I am using generic setup - Ubuntu 20.04.3 LTS and <br>
> kernels 5.8.0-63-generic (the same results are with 5.4.0-91-generic). <br>
> The nginx vritual host is the same as in the Nginx blog article with <br>
> exception of adding http2 to the listen! OpenSSL 3.0.0 and nginx 1.21.4 <br>
> are used.<br>
> The KTLS seems to work, because the strace and debug logs show it. Just <br>
> the sstrange thing is when using HTTP2, the sendfile syscalls look:<br>
> write(39, "\0 \0\0\0\0\0\0\1", 9) = 9<br>
> sendfile(39, 131, [1418218] => [1426410], 8192) = 8192<br>
> write(39, "\0 \0\0\0\0\0\0\1", 9) = 9<br>
> sendfile(39, 131, [1426410] => [1434602], 8192) = 8192<br>
> write(39, "\0 \0\0\0\0\0\0\1", 9) = 9<br>
> sendfile(39, 131, [1434602] => [1442794], 8192) = 8192<br>
> write(39, "\0 \0\0\0\0\0\0\1", 9) = 9<br>
> sendfile(39, 131, [1442794] => [1450986], 8192) = 8192<br>
> write(39, "\0 \0\0\0\0\0\0\1", 9) = 9<br>
> sendfile(39, 131, [1450986] => [1459178], 8192) = 8192<br>
> write(39, "\0 \0\0\0\0\0\0\1", 9) = 9<br>
> sendfile(39, 131, [1459178] => [1467370], 8192) = 8192<br>
> <br>
> It is always 8K and there are thousands of sendfile syscalls....<br>
<br>
That's expected, because of HTTP/2 framing. Unfortunately, HTTP/2 <br>
isn't designed to work with sendfile(), and sending large files <br>
over HTTP/2 require a lot of sendfile() syscalls. In general, for <br>
HTTP/2 it is better to keep sendfile() disabled.<br>
<br>
-- <br>
Maxim Dounin<br>
<a href="http://mdounin.ru/" rel="noreferrer" target="_blank">http://mdounin.ru/</a><br>
_______________________________________________<br>
nginx-devel mailing list<br>
<a href="mailto:nginx-devel@nginx.org" target="_blank">nginx-devel@nginx.org</a><br>
<a href="http://mailman.nginx.org/mailman/listinfo/nginx-devel" rel="noreferrer" target="_blank">http://mailman.nginx.org/mailman/listinfo/nginx-devel</a></blockquote></div>