nginx-tests SSL tests failing out of the box?

[Maxim Dounin]

Hello!

On Fri, Jan 26, 2024 at 09:29:58PM +0400, Sergey Kandaurov wrote:

> On Thu, Jan 25, 2024 at 11:38:57PM +0300, Maxim Dounin wrote:
> > Hello!
> > 
> > On Thu, Jan 25, 2024 at 06:59:36PM +0000, Mayerhofer, Austin via nginx-devel wrote:
> > 
> > > Hi all,
> > > 
> > > I have not made any changes to NGINX. Vanilla NGINX (./configure with no flags) passes all tests that run, but when compiling with SSL, not all SSL tests are passing. Is this expected, or do I need to configure nginx further aside from adding the --with-http_ssl_module flag? Do each of the failing tests below require separate fixes, or is there a one-size-fits-all solution for all of them?
> > > 
> > > OS: MacOS 12.6.3
> > > Chip: Apple M1 Max
> > > NGINX: 1.24.0 built from source code with ./configure --with-debug --with-http_ssl_module
> > > Nginx-tests: https://github.com/nginx/nginx-tests/tree/4c2ad8093952706f327d04887c5546bad91b75a6
> > > OpenSSL: 3.2.0 (/opt/homebrew/bin/openssl)
> > > Perl: 5.30.3 (/usr/bin/perl)
> > > 
> > > When I run
> > > 
> > > ```
> > > TEST_NGINX_BINARY=/usr/local/nginx/sbin/nginx prove -v ssl.t
> > > ```
> > > 
> > > I see
> > > 
> > > ```
> > > not ok 2 - session reused
> > > 
> > > #   Failed test 'session reused'
> > > #   at ssl.t line 187.
> > > #                   'HTTP/1.1 200 OK
> > > # Server: nginx/1.24.0
> > > # Date: Thu, 25 Jan 2024 18:50:10 GMT
> > > # Content-Type: text/plain
> > > # Content-Length: 6
> > > # Connection: close
> > > #
> > > # body .'
> > > #     doesn't match '(?^m:^body r$)'
> > > ```
> > 
> > [...]
> > 
> > It looks like SSL session reuse is broken in Perl you are 
> > using.  This might be the case if, for example, Net::SSLeay in 
> > your installation was compiled with system LibreSSL as an SSL 
> > library - at least on the server side LibreSSL simply does not 
> > support session reuse with TLSv1.3.
> > 
> > Test suite checks if nginx was compiled with LibreSSL and marks 
> > appropriate tests as TODO, but if the Perl module is broken 
> > instead, the test will fail.
> > 
> 
> Well, technically, we could test this and skip appropriately:
> 
> diff --git a/ssl_session_reuse.t b/ssl_session_reuse.t
> --- a/ssl_session_reuse.t
> +++ b/ssl_session_reuse.t
> @@ -166,7 +166,9 @@ local $TODO = 'no TLSv1.3 sessions, old 
>  local $TODO = 'no TLSv1.3 sessions, old IO::Socket::SSL'
>  	if $IO::Socket::SSL::VERSION < 2.061 && test_tls13();
>  local $TODO = 'no TLSv1.3 sessions in LibreSSL'
> -	if $t->has_module('LibreSSL') && test_tls13();
> +	if ($t->has_module('LibreSSL')
> +		|| Net::SSLeay::constant("LIBRESSL_VERSION_NUMBER"))
> +		&& test_tls13();
>  
>  is(test_reuse(8443), 1, 'tickets reused');
>  is(test_reuse(8444), 1, 'tickets and cache reused');
> 
> But I see little to no purpose: if the testing tool is broken
> in various unexpected ways (another example is X509_V_ERR_INVALID_PURPOSE
> in peer certificate verification as reported in the adjacent thread),
> I think we barely can handle this in general.

I generally agree.

Still, the X509_V_ERR_INVALID_PURPOSE seems to be an OpenSSL 
3.2.0-related issue: for tests using CA root certificates without 
CA:TRUE it now generates X509_V_ERR_INVALID_CA on the root 
certificate, which then changed to X509_V_ERR_INVALID_PURPOSE.

Given the list of incompatible changes from NEWS.md, and the fact 
that the same tests work fine with OpenSSL 3.2.0 but with 
"openssl" binary from older versions, it seems to be this:

  * The `x509`, `ca`, and `req` apps now always produce X.509v3 certificates.

This needs to be addressed.

-- 
Maxim Dounin
http://mdounin.ru/