Nginx KTLS hardware offloading not working

[Sergey A. Osokin]

Hi there,

hope you're doing well.

On Mon, Jun 13, 2022 at 07:57:26PM -0400, liwuliu wrote:
> Hi Team,

[...]

> Here are further system information:
> 
> root at r57-8814:/boot# nginx -V
> nginx version: nginx/1.21.4

This is a bit unclear: nginx version here is 1.21.4, but earlier you've
reported about 1.21.1.  Could you confirm what version is in use.
I'd recommend to use the recent stable version 1.22.0, so please
upgrade.

> built by gcc 11.2.0 (Ubuntu 11.2.0-19ubuntu1)
> built with OpenSSL 3.0.0 7 sep 2021
> TLS SNI support enabled
> configure arguments: --with-debug --prefix=/usr/local
> --conf-path=/usr/local/etc/nginx/nginx.conf
> --error-log-path=/var/log/nginx/error.log
> --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid
> --lock-path=/var/run/nginx.lock
> --http-client-body-temp-path=/var/cache/nginx/client_temp
> --http-proxy-temp-path=/var/cache/nginx/proxy_temp
> --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp
> --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp
> --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx
> --with-compat --with-file-aio --with-threads --with-http_addition_module
> --with-http_auth_request_module --with-http_dav_module
> --with-http_flv_module --with-http_gunzip_module
> --with-http_gzip_static_module --with-http_mp4_module
> --with-http_random_index_module --with-http_realip_module
> --with-http_secure_link_module --with-http_slice_module
> --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module
> --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream
> --with-stream_realip_module --with-stream_ssl_module
> --with-stream_ssl_preread_module --with-openssl=../openssl-3.0.0
> --with-openssl-opt=enable-ktls --with-cc-opt='-g -O2
> -fstack-protector-strong -Wformat -Werror=format-security
> -Wp,-D_FORTIFY_SOURCE=2 -fPIC'
> 
> 
> root at r57-8814:/usr/local/etc/nginx# cat nginx.conf

[...]

>     server {
>         listen 443 ssl;
>         ssl_certificate /usr/local/etc/nginx/cert.crt;
>         ssl_certificate_key /usr/local/etc/nginx/cert.key;
>         ssl_session_cache builtin:1000 shared:SSL:10m;
>         ssl_conf_command Options KTLS;
>         ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
>         ssl_ciphers
> HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;

Could you provide the output of the following command:

% openssl-3.0.0/.openssl/bin/openssl ciphers

to verify which TLS ciphers are supported by OpenSSL.

>         ssl_prefer_server_ciphers on;
>         access_log /var/log/nginx/access.log;
>         error_log  /var/log/nginx/error.log;
>         location / {
>                 proxy_set_header Host $host;
>                 proxy_set_header X-Real-IP $remote_addr;
>                 proxy_set_header X-Forwarded-For
> $proxy_add_x_forwarded_for;
>                 proxy_set_header X-Forwarded-Proto $scheme;
>                 # Fix the “It appears that your reverse proxy set up is
> broken" error.
>                 proxy_pass https://backend;

In the blog post [1], the root location in NGINX configuraion looks like
the following:

        location / {
            root /data;
    	}

So, that works for static content.
Could you try and confirm that works for you.

Thank you.

References:
1. https://www.nginx.com/blog/improving-nginx-performance-with-kernel-tls/

-- 
Sergey A. Osokin