[prev in list] [next in list] [prev in thread] [next in thread] 

List:       oss-security
Subject:    [oss-security] CVE-2016-9015: Python urllib3 1.17 and 1.18 certificate verification failure
From:       Cory Benfield <cory () lukasa ! co ! uk>
Date:       2016-10-27 11:26:20
Message-ID: 7FA392D7-7065-4BE4-80BD-E3118A003F79 () lukasa ! co ! uk
[Download RAW message or body]

Versions 1.17 and 1.18 of the Python urllib3 library suffer from a vulnerability that can cause \
them, in certain configurations, to not correctly validate TLS certificates. This places users \
of the library with those configurations at risk of man-in-the-middle and information leakage \
attacks. This vulnerability affects users using versions 1.17 and 1.18 of the urllib3 library, \
who are using the optional PyOpenSSL support for TLS instead of the regular standard library \
TLS backend, and who are using OpenSSL 1.1.0 via PyOpenSSL. This is an extremely uncommon \
configuration, so the security impact of this vulnerability is low.

Affected users should upgrade to urllib3 1.18.1, which has been published today and contains \
only the mitigation for this vulnerability on top of the changes in 1.18. If unable to upgrade, \
users should downgrade their OpenSSL version or temporarily stop injecting PyOpenSSL into \
urllib3 until they are able to upgrade. A more lengthy description of the vulnerability \
follows.

—

This vulnerability was introduced in a substantial refactor of the PyOpenSSL contrib module. \
During this refactor, a branch of code that mapped the Python standard library certificate \
verification constants (ssl.CERT_NONE, ssl.CERT_OPTIONAL, ssl.CERT_REQUIRED) to OpenSSL \
verification mode flags (SSL_VERIFY_NONE, SSL_VERIFY_PEER, etc.) was accidentally lost. This \
meant that Python standard library constants would be passed directly to OpenSSL via the \
SSL_CTX_set_verify function's mode argument.

Unfortunately, these Python standard library constants are Python wrappers around the values of \
a C enumerated type, declared like this:

    enum py_ssl_cert_requirements {
        PY_SSL_CERT_NONE,
        PY_SSL_CERT_OPTIONAL,
        PY_SSL_CERT_REQUIRED
    };

Per the standard C enumerated type rules, these constants have the values 0, 1, and 2 \
respectively. These integers do not all map to their OpenSSL verification mode flag \
equivalents. While PY_SSL_CERT_NONE and SSL_VERIFY_NONE have the same value (0) and \
PY_SSL_CERT_OPTIONAL and SSL_VERIFY_PEER have the same value (1), PY_SSL_CERT_REQUIRED has the \
value 2, which maps to SSL_VERIFY_FAIL_IF_NO_PEER_CERT. This flag is defined by the OpenSSL \
manual page as being meaningless on its own, requiring SSL_VERIFY_PEER to also be set in order \
to have any effect. Additionally, the manual page declares that SSL_VERIFY_FAIL_IF_NO_PEER_CERT \
has no effect in client mode.

In OpenSSL versions prior to 1.1.0, an implementation detail in the OpenSSL codebase would mean \
that if any nonzero value was passed to the mode argument of SSL_CTX_set_verify this would \
implicitly have the same effect as setting SSL_VERIFY_PEER. Essentially, the only value of mode \
in OpenSSL versions prior to 1.1.0 that would cause certificate validation to be disabled was 0 \
(SSL_VERIFY_NONE).

In the work done for OpenSSL 1.1.0, this implementation detail was changed to check for the \
SSL_VERIFY_PEER bit directly. As the OpenSSL flags are a bit mask, passing PY_SSL_CERT_REQUIRED \
(2) would *not* have the SSL_VERIFY_PEER bit (1) set, which means that OpenSSL would act act as \
though SSL_VERIFY_PEER was not set. Thus, if PY_SSL_CERT_REQUIRED is passed to OpenSSL directly \
in client mode, OpenSSL 1.0.2 and earlier treat it as equivalent to SSL_VERIFY_PEER, whereas \
OpenSSL 1.1.0 and later treat it as equivalent to SSL_VERIFY_NONE.

This is unquestionably an application error. The OpenSSL documentation is clear that the \
SSL_VERIFY_FAIL_IF_NO_PEER_CERT flag is both meaningless by itself and in client mode. However, \
OpenSSL's unexpected change in behaviour, combined with the fact that it has no way to report \
an invalid flag combination to SSL_CTX_set_verify, means that this application error leads to a \
catastrophic silent security failure when used with OpenSSL 1.1.0.

The fix for urllib3 is to reintroduce a mapping between the Python standard library enumerated \
type and the OpenSSL flags. Other applications should audit and confirm that they always \
successfully pass SSL_VERIFY_PEER to SSL_CTX_set_verify. The OpenSSL team have been notified of \
this behaviour and have concluded that it is not a security vulnerability in OpenSSL. However, \
they are considering reverting this change regardless, on the principle that it is better to \
fail closed than to fail open: https://github.com/openssl/openssl/pull/1793

Fortunately, urllib3's test suite caught this failure, which led to this investigation. \
Unfortunately, due to the relative scarcity of OpenSSL 1.1.0, two released versions of urllib3 \
had passed before anyone attempted to run urllib3 with OpenSSL 1.1.0.

The urllib3 team have contacted downstream redistributors for Red Hat and Debian: both \
distributions are not using versions of urllib3 later than 1.16, and so are unaffected. \
Additionally, the Python Requests library is using urllib3 version 1.16 and is also not \
affected.


[prev in list] [next in list] [prev in thread] [next in thread]