[nginx] SSL: logging level of "no suitable key share".

[Maxim Dounin]

details:   http://hg.nginx.org/nginx/rev/8f25a44d9add
branches:  
changeset: 7360:8f25a44d9add
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Sep 25 13:59:53 2018 +0300
description:
SSL: logging level of "no suitable key share".

The "no suitable key share" errors are reported by OpenSSL 1.1.1 when
using TLSv1.3 if there are no shared groups (that is, elliptic curves).
In particular, it is easy enough to trigger by using only a single
curve in ssl_ecdh_curve:

    ssl_ecdh_curve secp384r1;

and using a different curve in the client:

    openssl s_client -connect 127.0.0.1:443 -curves prime256v1

On the client side it is seen as "sslv3 alert handshake failure",
"SSL alert number 40":

0:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:ssl/record/rec_layer_s3.c:1528:SSL alert number 40

It can be also triggered with default ssl_ecdh_curve by using a curve
which is not in the default list (X25519, prime256v1, X448, secp521r1,
secp384r1):

    openssl s_client -connect 127.0.0.1:8443 -curves brainpoolP512r1

Given that many clients hardcode prime256v1, these errors might become
a common problem with TLSv1.3 if ssl_ecdh_curve is redefined.  Previously
this resulted in not using ECDH with such clients, but with TLSv1.3 it
is no longer possible and will result in a handshake failure.

The SSL_R_NO_SHARED_GROUP error is what BoringSSL returns in the same
situation.

Seen at:

https://serverfault.com/questions/932102/nginx-ssl-handshake-error-no-suitable-key-share

diffstat:

 src/event/ngx_event_openssl.c |  6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diffs (23 lines):

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -2585,6 +2585,9 @@ ngx_ssl_connection_error(ngx_connection_
 
             /* handshake failures */
         if (n == SSL_R_BAD_CHANGE_CIPHER_SPEC                        /*  103 */
+#ifdef SSL_R_NO_SUITABLE_KEY_SHARE
+            || n == SSL_R_NO_SUITABLE_KEY_SHARE                      /*  101 */
+#endif
             || n == SSL_R_BLOCK_CIPHER_PAD_IS_WRONG                  /*  129 */
             || n == SSL_R_DIGEST_CHECK_FAILED                        /*  149 */
             || n == SSL_R_ERROR_IN_RECEIVED_CIPHER_LIST              /*  151 */
@@ -2607,6 +2610,9 @@ ngx_ssl_connection_error(ngx_connection_
             || n == SSL_R_UNKNOWN_ALERT_TYPE                         /*  246 */
             || n == SSL_R_UNKNOWN_PROTOCOL                           /*  252 */
             || n == SSL_R_UNSUPPORTED_PROTOCOL                       /*  258 */
+#ifdef SSL_R_NO_SHARED_GROUP
+            || n == SSL_R_NO_SHARED_GROUP                            /*  266 */
+#endif
             || n == SSL_R_WRONG_VERSION_NUMBER                       /*  267 */
             || n == SSL_R_DECRYPTION_FAILED_OR_BAD_RECORD_MAC        /*  281 */
 #ifdef SSL_R_RENEGOTIATE_EXT_TOO_LONG