docker python requests results in dh key too small error

  • Last Update :
  • Techknowledgy :

I’m scraping some internal pages using Python and requests. I’ve turned off SSL verifications and warnings.

requests.packages.urllib3.disable_warnings()
page = requests.get(url, verify = False)

On certain servers I receive an SSL error I can’t get past.

Traceback (most recent call last):
File "scraper.py", line 6, in <module>
   page = requests.get(url, verify=False)
   File "/cygdrive/c/Users/jfeocco/VirtualEnv/scraping/lib/python3.4/site-packages/requests/api.py", line 71, in get
   return request('get', url, params=params, **kwargs)
   File "/cygdrive/c/Users/jfeocco/VirtualEnv/scraping/lib/python3.4/site-packages/requests/api.py", line 57, in request
   return session.request(method=method, url=url, **kwargs)
   File "/cygdrive/c/Users/jfeocco/VirtualEnv/scraping/lib/python3.4/site-packages/requests/sessions.py", line 475, in request
   resp = self.send(prep, **send_kwargs)
   File "/cygdrive/c/Users/jfeocco/VirtualEnv/scraping/lib/python3.4/site-packages/requests/sessions.py", line 585, in send
   r = adapter.send(request, **kwargs)
   File "/cygdrive/c/Users/jfeocco/VirtualEnv/scraping/lib/python3.4/site-packages/requests/adapters.py", line 477, in send
   raise SSLError(e, request=request)
   requests.exceptions.SSLError: [SSL: SSL_NEGATIVE_LENGTH] dh key too small (_ssl.c:600)

Edit:
I was able to resolve this by using a cipher set

import requests

requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS += 'HIGH:!DH:!aNULL'
try:
requests.packages.urllib3.contrib.pyopenssl.DEFAULT_SSL_CIPHER_LIST += 'HIGH:!DH:!aNULL'
except AttributeError:
   # no pyopenssl support used / needed / available
pass

page = requests.get(url, verify = False)

Besides the hacked string for 'HIGH:!DH:!aNULL', urllib3 module can be imported to disable the warning if it has

import requests
import urllib3

requests.packages.urllib3.disable_warnings()
requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS += ':HIGH:!DH:!aNULL'
try:
requests.packages.urllib3.contrib.pyopenssl.util.ssl_.DEFAULT_CIPHERS += ':HIGH:!DH:!aNULL'
except AttributeError:
   # no pyopenssl support used / needed / available
pass

page = requests.get(url, verify = False)

And it was fixed by commenting

CipherString = DEFAULT @SECLEVEL = 2

This also worked for me:

import requests
import urllib3
requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS = 'ALL:@SECLEVEL=1'

Error:

Traceback(most recent call last): [...]
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 581, in post
return self.request('POST', url, data = data, json = json, ** kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, ** send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, ** kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 514, in send
raise SSLError(e, request = request)
requests.exceptions.SSLError: HTTPSConnectionPool(host = 'somehost.com', port = 443): Max retries exceeded with url: myurl(Caused by SSLError(SSLError(1, '[SSL: WRONG_SIGNATURE_TYPE] wrong signature type (_ssl.c:1108)')))

check_supported_ciphers.sh

#!/usr/bin/env bash

# OpenSSL requires the port number.
SERVER = $1
DELAY = 1
ciphers = $(openssl ciphers 'ALL:eNULL' | sed - e 's/:/ /g')

echo Obtaining cipher list from $(openssl version).

for cipher in $ {
   ciphers[@]
}
do
   echo - n Testing $cipher...
   result = $(echo - n | openssl s_client - cipher "$cipher" - connect $SERVER 2 > & 1)
if [
   ["$result" = ~":error:"]
];
then
error = $(echo - n $result | cut - d ':' - f6)
echo NO($error)
else
if [
   ["$result" = ~"Cipher is ${cipher}" || "$result" = ~"Cipher    :"]
];
then
echo YES
else
   echo UNKNOWN RESPONSE
echo $result
fi
fi
sleep $DELAY
done

Give it permissions:

chmod + x check_supported_ciphers.sh

And execute it:

. / check_supported_ciphers.sh myremoteserver.example.com | grep OK

And replace:

_DEFAULT_CIPHERS = (
   'ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:DH+CHACHA20:ECDH+AES256:DH+AES256:'
   'ECDH+AES128:DH+AES:ECDH+HIGH:DH+HIGH:RSA+AESGCM:RSA+AES:RSA+HIGH:'
   '!aNULL:!eNULL:!MD5:!3DES'
)

By:

_DEFAULT_CIPHERS = (
   'AES128-SHA'
)

On CentOS 7, search for the following content in /etc/pki/tls/openssl.cnf:

[crypto_policy]
.include / etc / crypto - policies / back - ends / opensslcnf.config[new_oids]

Suggestion : 2

It is raised by a python script calling a rest API to oanda.com.,Connecting to the service with Postman or OANDA's java app both work without fault. Also, have the python script running on an RPi OK, after one change, see below!,Connect and share knowledge within a single location that is structured and easy to search., By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy.

I was able to workaround it by adjusting the ciphers used for the connection by creating an SSLContext. Here is an example of the code (for SMTPS connection - sending an e-mail):

connection = smtplib.SMTP(
   config['Email']['host'] + ':' + config['Email']['port'])
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
# Either of the following context settings worked
for me - choose one
# context.set_ciphers('HIGH:!DH:!aNULL')
context.set_ciphers('DEFAULT@SECLEVEL=1')
connection.starttls(context = context)
connection.login(config['Email']['user'], config['Email']['pw'])
connection.sendmail(config['Email']['sender'], recipients, msg.as_string())
connection.close()

The important lines are these - chose one of these CipherString adjustments:

context.set_ciphers('HIGH:!DH:!aNULL')
context.set_ciphers('DEFAULT@SECLEVEL=1')

Suggestion : 3

This issue tracker has been migrated to GitHub, and is currently read-only. For more information, see the GitHub FAQs in the Python's Developer Guide.

I 'm testing 3.5.10rc1 on a freshly installed Linux (Pop!_OS 20.04), and I'
m getting a lot of these test failures:

   ssl.SSLError: [SSL: EE_KEY_TOO_SMALL] ee key too small(_ssl.c: 2951)

Apparently the 2048 keys used in the tests are considered "too small"
with brand - new builds of the SSL library.

Christian: you upgraded the test suite keys to 3072 bits back in 2018(issue #34542), but didn't backport this as far as 3.5 because it was in security-fixes-only mode.  I experimented with taking your patch to 3.6 and applying it to 3.5, but 80% of the patches didn't apply cleanly.  Could you either backport this upgrade to 3.5 (I'll happily accept the PR), or advise me on how to otherwise mitigate the problem?  I don't really want to turn off all those tests.  Thanks!
I 'll look into it.
New changeset d565be84993a3d618add139cf21038e12c60a13e by Christian Heimes in branch '3.5':
   bpo - 41183: Update test certs and keys(#21258)
https://github.com/python/cpython/commit/d565be84993a3d618add139cf21038e12c60a13e
Thanks
for the backport!
Christian: Help!Again!

   I merged your PR, pulled a fresh copy, built it, and ran the test suite.I get seven failures in I think the same modules.

Most of the failures are either "ssl.SSLError: [SSL] internal error (_ssl.c:728)", or some flavor of "OSError: [Errno 0] Error".Sadly not helpful.

But!Occasionally the test suite prints a very telling error:

   ssl.SSLError: [SSL: DH_KEY_TOO_SMALL] dh key too small(_ssl.c: 3233)

Attached is the output of running just those seven tests.(One test is now working, not sure why.)

Obviously these tests pass on the buildbots, I assume that 's because their OpenSSL is slightly older.  But I don'
t think I can ship 3.5 .10 rc1
if it won 't build with current OpenSSL.

You should be able to simply pull the current 3.5 head(d565be84993a3d618add139cf21038e12c60a13e) to reproduce the error.
Upgrading to release blocker.

Suggestion : 4

It is not only a DH Key issues in server side, but also lots of different libraries are mismatched in python modules.,Code segment below is used to ignore those securitry issues because it may be not able be solved in server side. For example if it is internal legacy server, no one wants to update it.,Disabling warnings or certificate validation will not help. The underlying problem is a weak DH key used by the server which can be misused in the Logjam Attack. ,this is not an extra answer just try to combine the solution code from question with extra information So others can copy it directly without extra try

Besides the hacked string for 'HIGH:!DH:!aNULL', urllib3 module can be imported to disable the warning if it has

import requests
import urllib3

requests.packages.urllib3.disable_warnings()
requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS += ':HIGH:!DH:!aNULL'
try:
requests.packages.urllib3.contrib.pyopenssl.util.ssl_.DEFAULT_CIPHERS += ':HIGH:!DH:!aNULL'
except AttributeError:
   # no pyopenssl support used / needed / available
pass

page = requests.get(url, verify = False)

And it was fixed by commenting

CipherString = [email protected] = 2