The running time of the load_verify_locations functions of the SSL module using the cafile and cadata parameters is quite different.

[aaronhua123]

Bug report

Bug description:

The running time of the load_verify_locations functions of the SSL module using the cafile and cadata parameters is quite different.
The cafile parameter takes much more time to run than the cadata.

import ssl
import time
import certifi

from pathlib import Path

DEFAULT_CA_BUNDLE_PATH = Path(certifi.where())
print(DEFAULT_CA_BUNDLE_PATH)
context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)

start = time.time()
context.load_verify_locations(cafile=DEFAULT_CA_BUNDLE_PATH)
end = time.time()
print(end - start)

start = time.time()
cdata = open(DEFAULT_CA_BUNDLE_PATH, 'r', encoding='utf-8').read()
context.load_verify_locations(cadata=cdata)
end = time.time()
print(end - start)

out :

C:\Program Files\Python311\Lib\site-packages\certifi\cacert.pem
0.5879108905792236
0.06105494499206543

CPython versions tested on:

3.11

Operating systems tested on:

Windows

[serhiy-storchaka]

If cafile or capath arguments are provided, this method is a thin wrapper around SSL_CTX_load_verify_locations(). This is either the purposed behavior, or a bug in OpenSSL.

There may be a bug on our side if load_verify_locations() skips an expected time-consuming verification for cafile. I don't know if it is.

[serhiy-storchaka]

I can reproduce this on Linux and without using third-party modules:

$ ./python -m timeit -s 'import ssl; fn = "Lib/test/certdata/allsans.pem"; context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)' 'context.load_verify_locations(cafile=fn)'
200 loops, best of 5: 1 msec per loop
$ ./python -m timeit -s 'import ssl; fn = "Lib/test/certdata/allsans.pem"; context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)' 'with open(fn) as f: d = f.read()' 'context.load_verify_locations(cadata=d)'
1000 loops, best of 5: 388 usec per loop

@tiran, can you comment this? The support of cadata was added in bpo-18138.

[keepworking]

I just read the Module/_ssl.c code and openssl.

The cafile aims to inquire both the cert and crl in the pem file when reading the file and store it in the store.

The cadata collects only the cert, reads the data, and stores it in the store.

There seems to be a difference in movement in this area, so the performance seems to change.

If you read the certificate directly and turn it over to the cadata, it doesn't seem like a good way because you can't collect crl information.

I think cadata is a bit of a special purpose.

I think it's for special cases such as dynamically generated certificates.