nginx-tests SSL tests failing out of the box?

[Sergey Kandaurov]

On Thu, Jan 25, 2024 at 11:38:57PM +0300, Maxim Dounin wrote:
> Hello!
> 
> On Thu, Jan 25, 2024 at 06:59:36PM +0000, Mayerhofer, Austin via nginx-devel wrote:
> 
> > Hi all,
> > 
> > I have not made any changes to NGINX. Vanilla NGINX (./configure with no flags) passes all tests that run, but when compiling with SSL, not all SSL tests are passing. Is this expected, or do I need to configure nginx further aside from adding the --with-http_ssl_module flag? Do each of the failing tests below require separate fixes, or is there a one-size-fits-all solution for all of them?
> > 
> > OS: MacOS 12.6.3
> > Chip: Apple M1 Max
> > NGINX: 1.24.0 built from source code with ./configure --with-debug --with-http_ssl_module
> > Nginx-tests: https://github.com/nginx/nginx-tests/tree/4c2ad8093952706f327d04887c5546bad91b75a6
> > OpenSSL: 3.2.0 (/opt/homebrew/bin/openssl)
> > Perl: 5.30.3 (/usr/bin/perl)
> > 
> > When I run
> > 
> > ```
> > TEST_NGINX_BINARY=/usr/local/nginx/sbin/nginx prove -v ssl.t
> > ```
> > 
> > I see
> > 
> > ```
> > not ok 2 - session reused
> > 
> > #   Failed test 'session reused'
> > #   at ssl.t line 187.
> > #                   'HTTP/1.1 200 OK
> > # Server: nginx/1.24.0
> > # Date: Thu, 25 Jan 2024 18:50:10 GMT
> > # Content-Type: text/plain
> > # Content-Length: 6
> > # Connection: close
> > #
> > # body .'
> > #     doesn't match '(?^m:^body r$)'
> > ```
> 
> [...]
> 
> It looks like SSL session reuse is broken in Perl you are 
> using.  This might be the case if, for example, Net::SSLeay in 
> your installation was compiled with system LibreSSL as an SSL 
> library - at least on the server side LibreSSL simply does not 
> support session reuse with TLSv1.3.
> 
> Test suite checks if nginx was compiled with LibreSSL and marks 
> appropriate tests as TODO, but if the Perl module is broken 
> instead, the test will fail.
> 

Well, technically, we could test this and skip appropriately:

diff --git a/ssl_session_reuse.t b/ssl_session_reuse.t
--- a/ssl_session_reuse.t
+++ b/ssl_session_reuse.t
@@ -166,7 +166,9 @@ local $TODO = 'no TLSv1.3 sessions, old 
 local $TODO = 'no TLSv1.3 sessions, old IO::Socket::SSL'
 	if $IO::Socket::SSL::VERSION < 2.061 && test_tls13();
 local $TODO = 'no TLSv1.3 sessions in LibreSSL'
-	if $t->has_module('LibreSSL') && test_tls13();
+	if ($t->has_module('LibreSSL')
+		|| Net::SSLeay::constant("LIBRESSL_VERSION_NUMBER"))
+		&& test_tls13();
 
 is(test_reuse(8443), 1, 'tickets reused');
 is(test_reuse(8444), 1, 'tickets and cache reused');

But I see little to no purpose: if the testing tool is broken
in various unexpected ways (another example is X509_V_ERR_INVALID_PURPOSE
in peer certificate verification as reported in the adjacent thread),
I think we barely can handle this in general.