[Rpm-maint] [rpm-software-management/rpm] Improve the OpenSSL crypto backend (#1610)

[Demi Marie Obenour]

@DemiMarie commented on this pull request.



>  
-    ret = EVP_PKEY_verify_init(pkey_ctx);
-    if (ret < 0) {
-        rc = 1;
+    if (EVP_PKEY_verify_init(pkey_ctx) <= 0)

`man 3ssl EVP_PKEY_verify` states that 0 and negative numbers are error conditions.  That said, `!= 1` might be clearer.

>  
-    ret = EVP_PKEY_CTX_set_rsa_padding(pkey_ctx, RSA_PKCS1_PADDING);
-    if (ret < 0) {
-        rc = 1;
+    if (EVP_PKEY_CTX_set_rsa_padding(pkey_ctx, RSA_PKCS1_PADDING) <= 0)

According to `man 3ssl EVP_PKEY_CTX_set_rsa_padding` 0 or a negative value indicates an error, and any positive 
value indicates success.  So this is the _only_ correct way to check for an error.

>  
     int pkey_len = EVP_PKEY_size(key->evp_pkey);
     padded_sig = xcalloc(1, pkey_len);
-    if (!BN_bn2binpad(sig->bn, padded_sig, pkey_len)) {
-        rc = 1;
+    if (BN_bn2binpad(sig->bn, padded_sig, pkey_len) <= 0)

According to `man 3ssl BN_bin2binpad` a return value of -1 indicates an error, so the original code is wrong.

>      EVP_PKEY *evp_pkey; /* Fully constructed key */
+    unsigned int poison: 1; /* if set, this key cannot be mutated */

1 is the length of a bit-field, not a default value.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/1610#discussion_r806010195
You are receiving this because you are subscribed to this thread.

Message ID: <rpm-software-management/rpm/pull/1610/review/881866209 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rpm.org/pipermail/rpm-maint/attachments/20220214/514b7a95/attachment.html>