CVE-2023-6237: Addressing the Risk of DoS Attacks due to Excessively Long Invalid RSA Public Keys

CVE-2023-6237 is a security vulnerability discovered within the crypto library OpenSSL, impacting applications that use the function EVP_PKEY_public_check() to validate RSA public keys. When provided with excessively long RSA keys from untrusted sources, the application may experience severe delays, potentially leading to Denial of Service (DoS) attacks.

In this post, we will dive deeper into the details of this vulnerability, provide a code snippet illustrating the issue, explore the potential attack scenarios, and discuss the possible solutions to mitigate this vulnerability.

Issue Summary

CVE-2023-6237 is centered around the EVP_PKEY_public_check() function in OpenSSL, which is utilized to verify RSA public keys. When the validation process encounters an RSA key with a modulus ('*n*') that is excessively large and prime, the computation time for validating the RSA key increases significantly.

Here's a code snippet from OpenSSL that demonstrates how the validation process performs

// OpenSSL snippet
int EVP_PKEY_public_check(const EVP_PKEY_CTX *ctx)
{
    int ret;
    if (!ctx || !ctx->pmeth || !ctx->pmeth->public_check)
        return -1;
    if (ctx->operation != EVP_PKEY_OP_UNDEFINED &&
        ctx->operation != EVP_PKEY_OP_PUBLIC)
        goto not_supported;
    
    ret = ctx->pmeth->public_check(ctx);
    return ret;
    
    not_supported:
    RSAerr(RSA_F_EVP_PKEY_PUBLIC_CHECK, RSA_R_OPERATION_NOT_SUPPORTED_FOR_THIS_KEY);
    return -2;
}

Impact Summary

If an application using the OpenSSL library calls the EVP_PKEY_public_check() function when validating an RSA key obtained from an untrusted source, it may become vulnerable to DoS attacks due to the aforementioned delays. This vulnerability is found in OpenSSL 3. and 3.1 FIPS providers.

Notably, the OpenSSL pkey command line application is also affected by this vulnerability when it is executed with the -pubin and -check options on untrusted data. However, OpenSSL's SSL/TLS implementation is not impacted by this issue.

Exploit Details

An attacker could exploit the CVE-2023-6237 vulnerability by crafting an RSA key with an exceedingly large prime 'n' value to cause a lengthy computation when processed by an application using the EVP_PKEY_public_check() function. As a result, an attacker could potentially cause a DoS attack by sending multiple such requests to the application, causing a significant system slowdown or crash.

To mitigate this vulnerability, users should

1. Update their OpenSSL library to the latest version that includes a fix for CVE-2023-6237. OpenSSL has addressed this issue in their security advisory. Updating the library will help in ensuring that the applications using the library are no longer susceptible to this vulnerability.

2. Verify and sanitize input data when receiving RSA public keys from untrusted sources. Applications should implement proper checks for the RSA key sizes and validate the input before calling the EVP_PKEY_public_check() function.

3. Limit the time spent on key validation before considering it a potential attack. Applications should set a reasonable timeout on key validation functions to avoid indefinite delays that might signal a DoS attack.

By taking these precautions, developers can reduce the risk of falling victim to CVE-2023-6237-related DoS attacks and ensure that their applications are secure from this vulnerability.

Timeline

Published on: 04/25/2024 07:15:45 UTC
Last modified on: 11/01/2024 15:35:06 UTC