nginx-tests SSL tests failing out of the box?

[Sergey Kandaurov]

> On 29 Jan 2024, at 07:24, Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> Hello!
> 
> On Sat, Jan 27, 2024 at 07:19:45AM +0300, Maxim Dounin wrote:
> 
>> Hello!
>> 
>> On Fri, Jan 26, 2024 at 09:29:58PM +0400, Sergey Kandaurov wrote:
>> 
>>> On Thu, Jan 25, 2024 at 11:38:57PM +0300, Maxim Dounin wrote:
>>>> Hello!
>>>> 
>>>> On Thu, Jan 25, 2024 at 06:59:36PM +0000, Mayerhofer, Austin via nginx-devel wrote:
>>>> 
>>>>> Hi all,
>>>>> 
>>>>> I have not made any changes to NGINX. Vanilla NGINX (./configure with no flags) passes all tests that run, but when compiling with SSL, not all SSL tests are passing. Is this expected, or do I need to configure nginx further aside from adding the --with-http_ssl_module flag? Do each of the failing tests below require separate fixes, or is there a one-size-fits-all solution for all of them?
>>>>> 
>>>>> OS: MacOS 12.6.3
>>>>> Chip: Apple M1 Max
>>>>> NGINX: 1.24.0 built from source code with ./configure --with-debug --with-http_ssl_module
>>>>> Nginx-tests: https://github.com/nginx/nginx-tests/tree/4c2ad8093952706f327d04887c5546bad91b75a6
>>>>> OpenSSL: 3.2.0 (/opt/homebrew/bin/openssl)
>>>>> Perl: 5.30.3 (/usr/bin/perl)
>>>>> 
>>>>> When I run
>>>>> 
>>>>> ```
>>>>> TEST_NGINX_BINARY=/usr/local/nginx/sbin/nginx prove -v ssl.t
>>>>> ```
>>>>> 
>>>>> I see
>>>>> 
>>>>> ```
>>>>> not ok 2 - session reused
>>>>> 
>>>>> #   Failed test 'session reused'
>>>>> #   at ssl.t line 187.
>>>>> #                   'HTTP/1.1 200 OK
>>>>> # Server: nginx/1.24.0
>>>>> # Date: Thu, 25 Jan 2024 18:50:10 GMT
>>>>> # Content-Type: text/plain
>>>>> # Content-Length: 6
>>>>> # Connection: close
>>>>> #
>>>>> # body .'
>>>>> #     doesn't match '(?^m:^body r$)'
>>>>> ```
>>>> 
>>>> [...]
>>>> 
>>>> It looks like SSL session reuse is broken in Perl you are 
>>>> using.  This might be the case if, for example, Net::SSLeay in 
>>>> your installation was compiled with system LibreSSL as an SSL 
>>>> library - at least on the server side LibreSSL simply does not 
>>>> support session reuse with TLSv1.3.
>>>> 
>>>> Test suite checks if nginx was compiled with LibreSSL and marks 
>>>> appropriate tests as TODO, but if the Perl module is broken 
>>>> instead, the test will fail.
>>>> 
>>> 
>>> Well, technically, we could test this and skip appropriately:
>>> 
>>> diff --git a/ssl_session_reuse.t b/ssl_session_reuse.t
>>> --- a/ssl_session_reuse.t
>>> +++ b/ssl_session_reuse.t
>>> @@ -166,7 +166,9 @@ local $TODO = 'no TLSv1.3 sessions, old 
>>> local $TODO = 'no TLSv1.3 sessions, old IO::Socket::SSL'
>>> if $IO::Socket::SSL::VERSION < 2.061 && test_tls13();
>>> local $TODO = 'no TLSv1.3 sessions in LibreSSL'
>>> - if $t->has_module('LibreSSL') && test_tls13();
>>> + if ($t->has_module('LibreSSL')
>>> + || Net::SSLeay::constant("LIBRESSL_VERSION_NUMBER"))
>>> + && test_tls13();
>>> 
>>> is(test_reuse(8443), 1, 'tickets reused');
>>> is(test_reuse(8444), 1, 'tickets and cache reused');
>>> 
>>> But I see little to no purpose: if the testing tool is broken
>>> in various unexpected ways (another example is X509_V_ERR_INVALID_PURPOSE
>>> in peer certificate verification as reported in the adjacent thread),
>>> I think we barely can handle this in general.
>> 
>> I generally agree.
>> 
>> Still, the X509_V_ERR_INVALID_PURPOSE seems to be an OpenSSL 
>> 3.2.0-related issue: for tests using CA root certificates without 
>> CA:TRUE it now generates X509_V_ERR_INVALID_CA on the root 
>> certificate, which then changed to X509_V_ERR_INVALID_PURPOSE.
>> 
>> Given the list of incompatible changes from NEWS.md, and the fact 
>> that the same tests work fine with OpenSSL 3.2.0 but with 
>> "openssl" binary from older versions, it seems to be this:
>> 
>>  * The `x509`, `ca`, and `req` apps now always produce X.509v3 certificates.
>> 
>> This needs to be addressed.
> 
> Patch:
> 
> # HG changeset patch
> # User Maxim Dounin <mdounin at mdounin.ru>
> # Date 1706477656 -10800
> #      Mon Jan 29 00:34:16 2024 +0300
> # Node ID 156665421f83a054cf331e8f9a27dd4d2f86114d
> # Parent  27a79d3a8658794d7c0f8c246bcd92a9861da468
> Tests: compatibility with "openssl" app from OpenSSL 3.2.0.
> 
> OpenSSL 3.2.0's "openssl" app generates X.509v3 certificates unless explicitly
> asked not to.  Such certificates, even self-signed ones, cannot be used to sign
> other certificates without CA:TRUE explicitly set in the basicConstraints
> extension.  As a result, tests doing so are now failing.
> 
> Fix is to provide basicConstraints with CA:TRUE for self-signed root
> certificates used in "openssl ca" calls.
> 

Looks good.

[..]

-- 
Sergey Kandaurov