[PATCH 4 of 6] Tests: avoid specifying PSS in sigalgs unless in TLSv1.3

[Maxim Dounin]

Hello!

On Mon, May 22, 2023 at 11:52:16PM +0400, Sergey Kandaurov wrote:

> # HG changeset patch
> # User Sergey Kandaurov <pluknet at nginx.com>
> # Date 1684774417 -14400
> #      Mon May 22 20:53:37 2023 +0400
> # Node ID 42066e126d2ca0f6d5095d818910559adf5d4bdc
> # Parent  e60c76cbf2a5b0d9e1d235770d68f260cf1a4e3e
> Tests: avoid specifying PSS in sigalgs unless in TLSv1.3.
> 
> It might happen that TLSv1.3 is disabled and PSS isn't supported as seen
> on Amazon Linux (LTS).  The change restores old logic before 0e1865aa9b33.
> 
> diff --git a/ssl_certificates.t b/ssl_certificates.t
> --- a/ssl_certificates.t
> +++ b/ssl_certificates.t
> @@ -120,8 +120,8 @@ sub get_socket {
>  		return unless defined $type;
>  		my $ssleay = Net::SSLeay::SSLeay();
>  		return if ($ssleay < 0x1000200f || $ssleay == 0x20000000);
> -		my $sigalgs = 'RSA+SHA256:PSS+SHA256';
> -		$sigalgs = $type . '+SHA256' unless $type eq 'RSA';
> +		my $sigalgs = $type eq 'RSA' && test_tls13()
> +			? 'RSA+SHA256:PSS+SHA256' : $type . '+SHA256';
>  		# SSL_CTRL_SET_SIGALGS_LIST
>  		Net::SSLeay::CTX_ctrl($ctx, 98, 0, $sigalgs)
>  			or die("Failed to set sigalgs");
> diff --git a/ssl_stapling.t b/ssl_stapling.t
> --- a/ssl_stapling.t
> +++ b/ssl_stapling.t
> @@ -321,8 +321,8 @@ sub staple {
>  		return unless defined $ciphers;
>  		my $ssleay = Net::SSLeay::SSLeay();
>  		return if ($ssleay < 0x1000200f || $ssleay == 0x20000000);
> -		my $sigalgs = 'RSA+SHA256:PSS+SHA256';
> -		$sigalgs = $ciphers . '+SHA256' unless $ciphers eq 'RSA';
> +		my $sigalgs = $ciphers eq 'RSA' && test_tls13()
> +			? 'RSA+SHA256:PSS+SHA256' : $ciphers . '+SHA256';
>  		# SSL_CTRL_SET_SIGALGS_LIST
>  		Net::SSLeay::CTX_ctrl($ctx, 98, 0, $sigalgs)
>  			or die("Failed to set sigalgs");

I would rather refrain from SSL connections as in test_tls13() 
when creating an SSL context, hence the change.

But it looks like I was wrong assuming OpenSSL handles sigalgs 
similarly to ciphers, and ignores unknown ones.  Looking through 
the code suggests it instead returns an error if it sees an 
unknown signature algorithm, so trying to set 
'RSA+SHA256:PSS+SHA256' fails if OpenSSL does not support TLSv1.3.

Something like this should be enough to address this without 
introducing additional TLSv1.3 tests:

diff -r a797d7428fa5 ssl_certificates.t
--- a/ssl_certificates.t	Thu May 18 18:07:19 2023 +0300
+++ b/ssl_certificates.t	Tue May 23 01:03:42 2023 +0000
@@ -120,10 +120,11 @@
 		return unless defined $type;
 		my $ssleay = Net::SSLeay::SSLeay();
 		return if ($ssleay < 0x1000200f || $ssleay == 0x20000000);
-		my $sigalgs = 'RSA+SHA256:PSS+SHA256';
-		$sigalgs = $type . '+SHA256' unless $type eq 'RSA';
+		my @sigalgs = ('RSA+SHA256:PSS+SHA256', 'RSA+SHA256');
+		@sigalgs = ($type . '+SHA256') unless $type eq 'RSA';
 		# SSL_CTRL_SET_SIGALGS_LIST
-		Net::SSLeay::CTX_ctrl($ctx, 98, 0, $sigalgs)
+		Net::SSLeay::CTX_ctrl($ctx, 98, 0, $sigalgs[0])
+			or Net::SSLeay::CTX_ctrl($ctx, 98, 0, $sigalgs[1])
 			or die("Failed to set sigalgs");
 	};
 
diff -r a797d7428fa5 ssl_stapling.t
--- a/ssl_stapling.t	Thu May 18 18:07:19 2023 +0300
+++ b/ssl_stapling.t	Tue May 23 01:03:42 2023 +0000
@@ -319,10 +319,11 @@
 		return unless defined $ciphers;
 		my $ssleay = Net::SSLeay::SSLeay();
 		return if ($ssleay < 0x1000200f || $ssleay == 0x20000000);
-		my $sigalgs = 'RSA+SHA256:PSS+SHA256';
-		$sigalgs = $ciphers . '+SHA256' unless $ciphers eq 'RSA';
+		my @sigalgs = ('RSA+SHA256:PSS+SHA256', 'RSA+SHA256');
+		@sigalgs = ($ciphers . '+SHA256') unless $ciphers eq 'RSA';
 		# SSL_CTRL_SET_SIGALGS_LIST
-		Net::SSLeay::CTX_ctrl($ctx, 98, 0, $sigalgs)
+		Net::SSLeay::CTX_ctrl($ctx, 98, 0, $sigalgs[0])
+			or Net::SSLeay::CTX_ctrl($ctx, 98, 0, $sigalgs[1])
 			or die("Failed to set sigalgs");
 	};
 

(The code basically retries with 'RSA+SHA256' if setting sigalgs 
to 'RSA+SHA256:PSS+SHA256'.  If an error happens with ECDSA, it 
also retries with undefined, and then reports the error.)

-- 
Maxim Dounin
http://mdounin.ru/