[njs] WebCrypto: improved njs_bn_bn2binpad() for OpenSSL < 1.1.0.

[Dmitry Volyntsev]

details:   https://hg.nginx.org/njs/rev/2a412a132cf2
branches:  
changeset: 2022:2a412a132cf2
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed Jan 04 20:39:21 2023 -0800
description:
WebCrypto: improved njs_bn_bn2binpad() for OpenSSL < 1.1.0.

The patch makes njs_bn_bn2binpad() more similar to BN_bn2binpad().

diffstat:

 external/njs_openssl.h          |  13 ++++++++++++-
 external/njs_webcrypto_module.c |   4 ----
 2 files changed, 12 insertions(+), 5 deletions(-)

diffs (37 lines):

diff -r 2e3bbe8743af -r 2a412a132cf2 external/njs_openssl.h
--- a/external/njs_openssl.h	Wed Jan 04 18:07:30 2023 -0800
+++ b/external/njs_openssl.h	Wed Jan 04 20:39:21 2023 -0800
@@ -62,7 +62,18 @@ njs_bn_bn2binpad(const BIGNUM *bn, unsig
 #if (OPENSSL_VERSION_NUMBER >= 0x10100000L)
     return BN_bn2binpad(bn, to, tolen);
 #else
-    return BN_bn2bin(bn, &to[tolen - BN_num_bytes(bn)]);
+    int  len;
+
+    len = BN_num_bytes(bn);
+
+    if (tolen > len) {
+        memset(to, 0, tolen - len);
+
+    } else if (tolen < len) {
+        return -1;
+    }
+
+    return BN_bn2bin(bn, &to[tolen - len]);
 #endif
 }
 
diff -r 2e3bbe8743af -r 2a412a132cf2 external/njs_webcrypto_module.c
--- a/external/njs_webcrypto_module.c	Wed Jan 04 18:07:30 2023 -0800
+++ b/external/njs_webcrypto_module.c	Wed Jan 04 20:39:21 2023 -0800
@@ -3646,10 +3646,6 @@ njs_convert_der_to_p1363(njs_vm_t *vm, E
         goto fail;
     }
 
-#if (OPENSSL_VERSION_NUMBER < 0x10100000L)
-    memset(data, 0, 2 * n);
-#endif
-
     if (njs_bn_bn2binpad(ECDSA_SIG_get0_r(ec_sig), data, n) <= 0) {
         goto fail;
     }