ktls nginx not working
Anoop Alias
anoopalias01 at gmail.com
Thu Jan 27 13:48:13 UTC 2022
Hi,
I am trying to implement/test ktls as per the blog article
https://www.nginx.com/blog/improving-nginx-performance-with-kernel-tls/#tls-protocol
###########################
This is done on CentOS8 VM
# uname -r
4.18.0-348.7.1.el8_5.x86_64
###########################
# openssl-3.0.1/.openssl/bin/openssl ciphers
TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES256-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:RSA-PSK-AES256-GCM-SHA384:DHE-PSK-AES256-GCM-SHA384:RSA-PSK-CHACHA20-POLY1305:DHE-PSK-CHACHA20-POLY1305:ECDHE-PSK-CHACHA20-POLY1305:AES256-GCM-SHA384:PSK-AES256-GCM-SHA384:PSK-CHACHA20-POLY1305:RSA-PSK-AES128-GCM-SHA256:DHE-PSK-AES128-GCM-SHA256:AES128-GCM-SHA256:PSK-AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:ECDHE-PSK-AES256-CBC-SHA384:ECDHE-PSK-AES256-CBC-SHA:SRP-RSA-AES-256-CBC-SHA:SRP-AES-256-CBC-SHA:RSA-PSK-AES256-CBC-SHA384:DHE-PSK-AES256-CBC-SHA384:RSA-PSK-AES256-CBC-SHA:DHE-PSK-AES256-CBC-SHA:AES256-SHA:PSK-AES256-CBC-SHA384:PSK-AES256-CBC-SHA:ECDHE-PSK-AES128-CBC-SHA256:ECDHE-PSK-AES128-CBC-SHA:SRP-RSA-AES-128-CBC-SHA:SRP-AES-128-CBC-SHA:RSA-PSK-AES128-CBC-SHA256:DHE-PSK-AES128-CBC-SHA256:RSA-PSK-AES128-CBC-SHA:DHE-PSK-AES128-CBC-SHA:AES128-SHA:PSK-AES128-CBC-SHA256:PSK-AES128-CBC-SHA
###########################
# /usr/sbin/nginx-debug -V
nginx version: nginx/1.21.6
built by gcc 8.5.0 20210514 (Red Hat 8.5.0-4) (GCC)
built with OpenSSL 3.0.1 14 Dec 2021
TLS SNI support enabled
configure arguments: --with-debug --prefix=/etc/nginx
--sbin-path=/usr/sbin/nginx --modules-path=/etc/nginx/modules
--with-pcre=./pcre2-10.39 --with-pcre-jit --with-zlib=./zlib-1.2.11
--with-openssl=./openssl-3.0.1 --with-openssl-opt=enable-ktls
--with-openssl-opt=enable-tls1_3 --conf-path=/etc/nginx/nginx.conf
--error-log-path=/var/log/nginx/error_log
############################
The debug log does not show any signs of ktls in use
(snippet from the log provided below on download of a 1G file)
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002077A08 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002077A08 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002077D30 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002077D30 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002075E58 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002075E58 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002075F60 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002075F60 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002077BA8 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002077BA8 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002077AA0 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002077AA0 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002077890 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002077890 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002075BC8 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002075BC8 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http write filter
0000000000000000
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 read: 15,
0000000002791FC0, 32768, 21168128
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 read: 15,
0000000002791FC0, 32768, 21168128
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 read: 15,
0000000002799FD0, 32768, 21200896
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http postpone filter
"/1G?" 0000000002075DD8
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 write new buf t:1 f:1
0000000002791FC0, pos 0000000002791FC0, size: 32768 file: 21168128, size:
32768
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 write new buf t:1 f:1
0000000002799FD0, pos 0000000002799FD0, size: 32768 file: 21200896, size:
32768
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http write filter: l:0
f:1 s:65536
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http write filter limit
2097152
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 send chain:
0000000002075DF8
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 windows:
conn:10297344 stream:868352
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002075BC8: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002077890: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002077AA0: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002077BA8: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002075F60: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002075E58: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002077D30: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002077A08: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002077A08 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002077D30 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002075E58 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002075F60 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002077BA8 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002077AA0 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002077890 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002075BC8 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8174
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL to write: 16384
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL_write: 16384
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 18
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8156
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL to write: 16384
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL_write: 16384
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 36
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8138
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL to write: 16384
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL_write: 16384
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 54
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8120
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL to write: 16384
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL_write: 16384
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 72
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL to write: 72
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL_write: 72
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002075BC8 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002075BC8 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002077890 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002075BC8 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002077890 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002077890 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002077AA0 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002077AA0 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002077BA8 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002077BA8 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002075F60 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002075F60 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002075E58 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002075E58 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002077D30 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002077D30 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame
0000000002077A08 was sent
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent:
0000000002077A08 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http write filter
0000000000000000
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 read: 15,
0000000002799FD0, 32768, 21233664
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 read: 15,
0000000002791FC0, 32768, 21266432
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http postpone filter
"/1G?" 0000000002075DE8
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 write new buf t:1 f:1
0000000002799FD0, pos 0000000002799FD0, size: 32768 file: 21233664, size:
32768
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 write new buf t:1 f:1
0000000002791FC0, pos 0000000002791FC0, size: 32768 file: 21266432, size:
32768
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http write filter: l:0
f:1 s:65536
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 send chain:
0000000002077C50
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 windows:
conn:10231808 stream:802816
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002077A08: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002077D30: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002075E58: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002075F60: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002077BA8: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002077AA0: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002077890: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame
0000000002075BC8: len:8192 flags:0
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002075BC8 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002077890 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002077AA0 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002077BA8 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002075F60 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002075E58 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002077D30 sid:1 bl:0 len:8192
2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out:
0000000002077A08 sid:1 bl:0 len:8192
#############################################
[root at 65-108-156-104 nginx-1.21.6]# grep SSL_sendfile
/var/log/nginx/error_log
[root at 65-108-156-104 nginx-1.21.6]# grep BIO /var/log/nginx/error_log
[root at 65-108-156-104 nginx-1.21.6]#
There is no SSL_sendfile in the log
##############################################
# TLS Settings
ssl_protocols TLSv1.3;
ssl_session_cache shared:SSL:32m;
ssl_dhparam /etc/nginx/ssl/dhparam.pem;
ssl_session_timeout 1d;
ssl_session_tickets off;
ssl_ocsp_cache shared:ocspcache:10m;
server{
...
ssl_conf_command Options KTLS;
..
}
#################################################
What am I doing wrong?
Thanks in advance,
--
*Anoop P Alias*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20220127/ad257647/attachment.htm>
More information about the nginx
mailing list