tahoe-lafs/trac

OpenSSL.crypto.Error (unknown message digest algorithm) when starting a node, using OpenSSL 1.0.1k-fips #2400

New issue

Closed

opened 2015-04-10 23:59:39 +00:00 by daira · 10 comments

daira commented

2015-04-10 23:59:39 +00:00

Gabe reported:

was able to build tahoe successfully, and can 'create-client' as well, however `tahoe start` fails due to pyOpenSSL error:

    Traceback (most recent call last):
    File "/usr/lib64/python2.7/site-packages/twisted/application/app.py", line 645, in run
    runApp(config)
    File "/usr/lib64/python2.7/site-packages/twisted/scripts/twistd.py", line 23, in runApp
    _SomeApplicationRunner(config).run()
    File "/usr/lib64/python2.7/site-packages/twisted/application/app.py", line 379, in run
    self.application = self.createOrGetApplication()
    File "/usr/lib64/python2.7/site-packages/twisted/application/app.py", line 444, in createOrGetApplication
    application = getApplication(self.config, passphrase)
    --- <exception caught here> ---
    File "/usr/lib64/python2.7/site-packages/twisted/application/app.py", line 455, in getApplication
    application = service.loadApplication(filename, style, passphrase)
    File "/usr/lib64/python2.7/site-packages/twisted/application/service.py", line 403, in loadApplication
    application = sob.loadValueFromFile(filename, 'application', passphrase)
    File "/usr/lib64/python2.7/site-packages/twisted/persisted/sob.py", line 210, in loadValueFromFile
    exec fileObj in d, d
    File "tahoe-client.tac", line 10, in <module>
    c = client.Client()
    File "/home/gabeos/tahoe/src/allmydata/client.py", line 130, in __init__
    node.Node.__init__(self, basedir)
    File "/home/gabeos/tahoe/src/allmydata/node.py", line 82, in __init__
    self.create_tub()
    File "/home/gabeos/tahoe/src/allmydata/node.py", line 174, in create_tub
    self.tub = Tub(certFile=certfile)
    File "/usr/lib/python2.7/site-packages/foolscap/pb.py", line 240, in __init__
    self.setupEncryptionFile(certFile)
    File "/usr/lib/python2.7/site-packages/foolscap/pb.py", line 252, in setupEncryptionFile
    self.setupEncryption(certData)
    File "/usr/lib/python2.7/site-packages/foolscap/pb.py", line 267, in setupEncryption
    cert = self.createCertificate()
    File "/usr/lib/python2.7/site-packages/foolscap/pb.py", line 476, in createCertificate
    132)
    File "/usr/lib64/python2.7/site-packages/twisted/internet/_sslverify.py", line 853, in signCertificateRequest
    hlreq = CertificateRequest.load(requestData, requestFormat)
    File "/usr/lib64/python2.7/site-packages/twisted/internet/_sslverify.py", line 571, in load
    if not req.verify(req.get_pubkey()):
    OpenSSL.crypto.Error: [('asn1 encoding routines', 'ASN1_item_verify', 'unknown message digest algorithm')]

    Failed to load application: [('asn1 encoding routines', 'ASN1_item_verify', 'unknown message digest algorithm')]

There are reports of this issue popping up online, but seems that consensus is that it is due to openssl versions < [0.98], at which point SHA-256 algorithms were added. However, I'm on Fedora 21, running openssl version 1.0.1k-fips, and definitely have sha* available.

    python -c 'import ssl; print ssl.OPENSSL_VERSION'
    OpenSSL 1.0.1k-fips 8 Jan 2015

anyway, if you have any thoughts or suggestions, it'd be much appreciated.
Thanks,
Gabe

Gabe reported: was able to build tahoe successfully, and can 'create-client' as well, however `tahoe start` fails due to pyOpenSSL error: ``` Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/twisted/application/app.py", line 645, in run runApp(config) File "/usr/lib64/python2.7/site-packages/twisted/scripts/twistd.py", line 23, in runApp _SomeApplicationRunner(config).run() File "/usr/lib64/python2.7/site-packages/twisted/application/app.py", line 379, in run self.application = self.createOrGetApplication() File "/usr/lib64/python2.7/site-packages/twisted/application/app.py", line 444, in createOrGetApplication application = getApplication(self.config, passphrase) --- <exception caught here> --- File "/usr/lib64/python2.7/site-packages/twisted/application/app.py", line 455, in getApplication application = service.loadApplication(filename, style, passphrase) File "/usr/lib64/python2.7/site-packages/twisted/application/service.py", line 403, in loadApplication application = sob.loadValueFromFile(filename, 'application', passphrase) File "/usr/lib64/python2.7/site-packages/twisted/persisted/sob.py", line 210, in loadValueFromFile exec fileObj in d, d File "tahoe-client.tac", line 10, in <module> c = client.Client() File "/home/gabeos/tahoe/src/allmydata/client.py", line 130, in __init__ node.Node.__init__(self, basedir) File "/home/gabeos/tahoe/src/allmydata/node.py", line 82, in __init__ self.create_tub() File "/home/gabeos/tahoe/src/allmydata/node.py", line 174, in create_tub self.tub = Tub(certFile=certfile) File "/usr/lib/python2.7/site-packages/foolscap/pb.py", line 240, in __init__ self.setupEncryptionFile(certFile) File "/usr/lib/python2.7/site-packages/foolscap/pb.py", line 252, in setupEncryptionFile self.setupEncryption(certData) File "/usr/lib/python2.7/site-packages/foolscap/pb.py", line 267, in setupEncryption cert = self.createCertificate() File "/usr/lib/python2.7/site-packages/foolscap/pb.py", line 476, in createCertificate 132) File "/usr/lib64/python2.7/site-packages/twisted/internet/_sslverify.py", line 853, in signCertificateRequest hlreq = CertificateRequest.load(requestData, requestFormat) File "/usr/lib64/python2.7/site-packages/twisted/internet/_sslverify.py", line 571, in load if not req.verify(req.get_pubkey()): OpenSSL.crypto.Error: [('asn1 encoding routines', 'ASN1_item_verify', 'unknown message digest algorithm')] Failed to load application: [('asn1 encoding routines', 'ASN1_item_verify', 'unknown message digest algorithm')] ``` There are reports of this issue popping up online, but seems that consensus is that it is due to openssl versions < [0.98], at which point SHA-256 algorithms were added. However, I'm on Fedora 21, running openssl version 1.0.1k-fips, and definitely have sha* available. ``` python -c 'import ssl; print ssl.OPENSSL_VERSION' OpenSSL 1.0.1k-fips 8 Jan 2015 ``` anyway, if you have any thoughts or suggestions, it'd be much appreciated. Thanks, Gabe

daira added the

labels 2015-04-10 23:59:39 +00:00

daira added this to the undecided milestone 2015-04-10 23:59:39 +00:00

daira self-assigned this 2015-04-10 23:59:39 +00:00

daira commented

2015-04-11 00:05:47 +00:00

Author

It's quite unfortunate that the error message doesn't specify which message digest algorithm is unknown. I thought that the FIPS build might not include SHA-1, but Gabe posted (on the LeastAuthority Zendesk ticket) an algorithm list that does include that:

Here is the list of message digest and cipher algorithms available, looks like SHA1 is supported:

 gabeos  ~  openssl list-message-digest-algorithms
DSA
DSA-SHA
DSA-SHA1 => DSA
DSA-SHA1-old => DSA-SHA1
DSS1 => DSA-SHA1
MD4
MD5
RIPEMD160
RSA-MD4 => MD4
RSA-MD5 => MD5
RSA-RIPEMD160 => RIPEMD160
RSA-SHA => SHA
RSA-SHA1 => SHA1
RSA-SHA1-2 => RSA-SHA1
RSA-SHA224 => SHA224
RSA-SHA256 => SHA256
RSA-SHA384 => SHA384
RSA-SHA512 => SHA512
SHA
SHA1
SHA224
SHA256
SHA384
SHA512
DSA
DSA-SHA
dsaWithSHA1 => DSA
dss1 => DSA-SHA1
ecdsa-with-SHA1
MD4
md4WithRSAEncryption => MD4
MD5
md5WithRSAEncryption => MD5
ripemd => RIPEMD160
RIPEMD160
ripemd160WithRSA => RIPEMD160
rmd160 => RIPEMD160
SHA
SHA1
sha1WithRSAEncryption => SHA1
SHA224
sha224WithRSAEncryption => SHA224
SHA256
sha256WithRSAEncryption => SHA256
SHA384
sha384WithRSAEncryption => SHA384
SHA512
sha512WithRSAEncryption => SHA512
shaWithRSAEncryption => SHA
ssl2-md5 => MD5
ssl3-md5 => MD5
ssl3-sha1 => SHA1
whirlpool
 gabeos  ~  openssl list-cipher-algorithms
AES-128-CBC
AES-128-CBC-HMAC-SHA1
AES-128-CFB
AES-128-CFB1
AES-128-CFB8
AES-128-CTR
AES-128-ECB
AES-128-OFB
AES-128-XTS
AES-192-CBC
AES-192-CFB
AES-192-CFB1
AES-192-CFB8
AES-192-CTR
AES-192-ECB
AES-192-OFB
AES-256-CBC
AES-256-CBC-HMAC-SHA1
AES-256-CFB
AES-256-CFB1
AES-256-CFB8
AES-256-CTR
AES-256-ECB
AES-256-OFB
AES-256-XTS
AES128 => AES-128-CBC
AES192 => AES-192-CBC
AES256 => AES-256-CBC
BF => BF-CBC
BF-CBC
BF-CFB
BF-ECB
BF-OFB
CAMELLIA-128-CBC
CAMELLIA-128-CFB
CAMELLIA-128-CFB1
CAMELLIA-128-CFB8
CAMELLIA-128-ECB
CAMELLIA-128-OFB
CAMELLIA-192-CBC
CAMELLIA-192-CFB
CAMELLIA-192-CFB1
CAMELLIA-192-CFB8
CAMELLIA-192-ECB
CAMELLIA-192-OFB
CAMELLIA-256-CBC
CAMELLIA-256-CFB
CAMELLIA-256-CFB1
CAMELLIA-256-CFB8
CAMELLIA-256-ECB
CAMELLIA-256-OFB
CAMELLIA128 => CAMELLIA-128-CBC
CAMELLIA192 => CAMELLIA-192-CBC
CAMELLIA256 => CAMELLIA-256-CBC
CAST => CAST5-CBC
CAST-cbc => CAST5-CBC
CAST5-CBC
CAST5-CFB
CAST5-ECB
CAST5-OFB
DES => DES-CBC
DES-CBC
DES-CFB
DES-CFB1
DES-CFB8
DES-ECB
DES-EDE
DES-EDE-CBC
DES-EDE-CFB
DES-EDE-OFB
DES-EDE3
DES-EDE3-CBC
DES-EDE3-CFB
DES-EDE3-CFB1
DES-EDE3-CFB8
DES-EDE3-OFB
DES-OFB
DES3 => DES-EDE3-CBC
DESX => DESX-CBC
DESX-CBC
IDEA => IDEA-CBC
IDEA-CBC
IDEA-CFB
IDEA-ECB
IDEA-OFB
RC2 => RC2-CBC
RC2-40-CBC
RC2-64-CBC
RC2-CBC
RC2-CFB
RC2-ECB
RC2-OFB
RC4
RC4-40
RC4-HMAC-MD5
SEED => SEED-CBC
SEED-CBC
SEED-CFB
SEED-ECB
SEED-OFB
AES-128-CBC
AES-128-CBC-HMAC-SHA1
AES-128-CFB
AES-128-CFB1
AES-128-CFB8
AES-128-CTR
AES-128-ECB
id-aes128-GCM
AES-128-OFB
AES-128-XTS
AES-192-CBC
AES-192-CFB
AES-192-CFB1
AES-192-CFB8
AES-192-CTR
AES-192-ECB
id-aes192-GCM
AES-192-OFB
AES-256-CBC
AES-256-CBC-HMAC-SHA1
AES-256-CFB
AES-256-CFB1
AES-256-CFB8
AES-256-CTR
AES-256-ECB
id-aes256-GCM
AES-256-OFB
AES-256-XTS
aes128 => AES-128-CBC
aes192 => AES-192-CBC
aes256 => AES-256-CBC
bf => BF-CBC
BF-CBC
BF-CFB
BF-ECB
BF-OFB
blowfish => BF-CBC
CAMELLIA-128-CBC
CAMELLIA-128-CFB
CAMELLIA-128-CFB1
CAMELLIA-128-CFB8
CAMELLIA-128-ECB
CAMELLIA-128-OFB
CAMELLIA-192-CBC
CAMELLIA-192-CFB
CAMELLIA-192-CFB1
CAMELLIA-192-CFB8
CAMELLIA-192-ECB
CAMELLIA-192-OFB
CAMELLIA-256-CBC
CAMELLIA-256-CFB
CAMELLIA-256-CFB1
CAMELLIA-256-CFB8
CAMELLIA-256-ECB
CAMELLIA-256-OFB
camellia128 => CAMELLIA-128-CBC
camellia192 => CAMELLIA-192-CBC
camellia256 => CAMELLIA-256-CBC
cast => CAST5-CBC
cast-cbc => CAST5-CBC
CAST5-CBC
CAST5-CFB
CAST5-ECB
CAST5-OFB
des => DES-CBC
DES-CBC
DES-CFB
DES-CFB1
DES-CFB8
DES-ECB
DES-EDE
DES-EDE-CBC
DES-EDE-CFB
DES-EDE-OFB
DES-EDE3
DES-EDE3-CBC
DES-EDE3-CFB
DES-EDE3-CFB1
DES-EDE3-CFB8
DES-EDE3-OFB
DES-OFB
des3 => DES-EDE3-CBC
desx => DESX-CBC
DESX-CBC
id-aes128-GCM
id-aes128-wrap
id-aes128-wrap-pad
id-aes192-GCM
id-aes192-wrap
id-aes192-wrap-pad
id-aes256-GCM
id-aes256-wrap
id-aes256-wrap-pad
id-smime-alg-CMS3DESwrap
idea => IDEA-CBC
IDEA-CBC
IDEA-CFB
IDEA-ECB
IDEA-OFB
rc2 => RC2-CBC
RC2-40-CBC
RC2-64-CBC
RC2-CBC
RC2-CFB
RC2-ECB
RC2-OFB
RC4
RC4-40
RC4-HMAC-MD5
seed => SEED-CBC
SEED-CBC
SEED-CFB
SEED-ECB
SEED-OFB

It's quite unfortunate that the error message doesn't specify *which* message digest algorithm is unknown. I thought that the FIPS build might not include SHA-1, but Gabe posted (on the LeastAuthority Zendesk ticket) an algorithm list that does include that: > Here is the list of message digest and cipher algorithms available, looks like SHA1 is supported: ``` gabeos  ~  openssl list-message-digest-algorithms DSA DSA-SHA DSA-SHA1 => DSA DSA-SHA1-old => DSA-SHA1 DSS1 => DSA-SHA1 MD4 MD5 RIPEMD160 RSA-MD4 => MD4 RSA-MD5 => MD5 RSA-RIPEMD160 => RIPEMD160 RSA-SHA => SHA RSA-SHA1 => SHA1 RSA-SHA1-2 => RSA-SHA1 RSA-SHA224 => SHA224 RSA-SHA256 => SHA256 RSA-SHA384 => SHA384 RSA-SHA512 => SHA512 SHA SHA1 SHA224 SHA256 SHA384 SHA512 DSA DSA-SHA dsaWithSHA1 => DSA dss1 => DSA-SHA1 ecdsa-with-SHA1 MD4 md4WithRSAEncryption => MD4 MD5 md5WithRSAEncryption => MD5 ripemd => RIPEMD160 RIPEMD160 ripemd160WithRSA => RIPEMD160 rmd160 => RIPEMD160 SHA SHA1 sha1WithRSAEncryption => SHA1 SHA224 sha224WithRSAEncryption => SHA224 SHA256 sha256WithRSAEncryption => SHA256 SHA384 sha384WithRSAEncryption => SHA384 SHA512 sha512WithRSAEncryption => SHA512 shaWithRSAEncryption => SHA ssl2-md5 => MD5 ssl3-md5 => MD5 ssl3-sha1 => SHA1 whirlpool gabeos  ~  openssl list-cipher-algorithms AES-128-CBC AES-128-CBC-HMAC-SHA1 AES-128-CFB AES-128-CFB1 AES-128-CFB8 AES-128-CTR AES-128-ECB AES-128-OFB AES-128-XTS AES-192-CBC AES-192-CFB AES-192-CFB1 AES-192-CFB8 AES-192-CTR AES-192-ECB AES-192-OFB AES-256-CBC AES-256-CBC-HMAC-SHA1 AES-256-CFB AES-256-CFB1 AES-256-CFB8 AES-256-CTR AES-256-ECB AES-256-OFB AES-256-XTS AES128 => AES-128-CBC AES192 => AES-192-CBC AES256 => AES-256-CBC BF => BF-CBC BF-CBC BF-CFB BF-ECB BF-OFB CAMELLIA-128-CBC CAMELLIA-128-CFB CAMELLIA-128-CFB1 CAMELLIA-128-CFB8 CAMELLIA-128-ECB CAMELLIA-128-OFB CAMELLIA-192-CBC CAMELLIA-192-CFB CAMELLIA-192-CFB1 CAMELLIA-192-CFB8 CAMELLIA-192-ECB CAMELLIA-192-OFB CAMELLIA-256-CBC CAMELLIA-256-CFB CAMELLIA-256-CFB1 CAMELLIA-256-CFB8 CAMELLIA-256-ECB CAMELLIA-256-OFB CAMELLIA128 => CAMELLIA-128-CBC CAMELLIA192 => CAMELLIA-192-CBC CAMELLIA256 => CAMELLIA-256-CBC CAST => CAST5-CBC CAST-cbc => CAST5-CBC CAST5-CBC CAST5-CFB CAST5-ECB CAST5-OFB DES => DES-CBC DES-CBC DES-CFB DES-CFB1 DES-CFB8 DES-ECB DES-EDE DES-EDE-CBC DES-EDE-CFB DES-EDE-OFB DES-EDE3 DES-EDE3-CBC DES-EDE3-CFB DES-EDE3-CFB1 DES-EDE3-CFB8 DES-EDE3-OFB DES-OFB DES3 => DES-EDE3-CBC DESX => DESX-CBC DESX-CBC IDEA => IDEA-CBC IDEA-CBC IDEA-CFB IDEA-ECB IDEA-OFB RC2 => RC2-CBC RC2-40-CBC RC2-64-CBC RC2-CBC RC2-CFB RC2-ECB RC2-OFB RC4 RC4-40 RC4-HMAC-MD5 SEED => SEED-CBC SEED-CBC SEED-CFB SEED-ECB SEED-OFB AES-128-CBC AES-128-CBC-HMAC-SHA1 AES-128-CFB AES-128-CFB1 AES-128-CFB8 AES-128-CTR AES-128-ECB id-aes128-GCM AES-128-OFB AES-128-XTS AES-192-CBC AES-192-CFB AES-192-CFB1 AES-192-CFB8 AES-192-CTR AES-192-ECB id-aes192-GCM AES-192-OFB AES-256-CBC AES-256-CBC-HMAC-SHA1 AES-256-CFB AES-256-CFB1 AES-256-CFB8 AES-256-CTR AES-256-ECB id-aes256-GCM AES-256-OFB AES-256-XTS aes128 => AES-128-CBC aes192 => AES-192-CBC aes256 => AES-256-CBC bf => BF-CBC BF-CBC BF-CFB BF-ECB BF-OFB blowfish => BF-CBC CAMELLIA-128-CBC CAMELLIA-128-CFB CAMELLIA-128-CFB1 CAMELLIA-128-CFB8 CAMELLIA-128-ECB CAMELLIA-128-OFB CAMELLIA-192-CBC CAMELLIA-192-CFB CAMELLIA-192-CFB1 CAMELLIA-192-CFB8 CAMELLIA-192-ECB CAMELLIA-192-OFB CAMELLIA-256-CBC CAMELLIA-256-CFB CAMELLIA-256-CFB1 CAMELLIA-256-CFB8 CAMELLIA-256-ECB CAMELLIA-256-OFB camellia128 => CAMELLIA-128-CBC camellia192 => CAMELLIA-192-CBC camellia256 => CAMELLIA-256-CBC cast => CAST5-CBC cast-cbc => CAST5-CBC CAST5-CBC CAST5-CFB CAST5-ECB CAST5-OFB des => DES-CBC DES-CBC DES-CFB DES-CFB1 DES-CFB8 DES-ECB DES-EDE DES-EDE-CBC DES-EDE-CFB DES-EDE-OFB DES-EDE3 DES-EDE3-CBC DES-EDE3-CFB DES-EDE3-CFB1 DES-EDE3-CFB8 DES-EDE3-OFB DES-OFB des3 => DES-EDE3-CBC desx => DESX-CBC DESX-CBC id-aes128-GCM id-aes128-wrap id-aes128-wrap-pad id-aes192-GCM id-aes192-wrap id-aes192-wrap-pad id-aes256-GCM id-aes256-wrap id-aes256-wrap-pad id-smime-alg-CMS3DESwrap idea => IDEA-CBC IDEA-CBC IDEA-CFB IDEA-ECB IDEA-OFB rc2 => RC2-CBC RC2-40-CBC RC2-64-CBC RC2-CBC RC2-CFB RC2-ECB RC2-OFB RC4 RC4-40 RC4-HMAC-MD5 seed => SEED-CBC SEED-CBC SEED-CFB SEED-ECB SEED-OFB ```

daira commented

2015-04-11 00:34:26 +00:00

Author

It appears that foolscap, before the fix to http://foolscap.lothar.com/trac/ticket/141, generates certificates that use RSA with MD5 as the signature algorithm. Although RSA-MD5 is listed in the algorithms in comment:398921, it may be that OpenSSL in FIPS mode is (not entirely unreasonably) refusing to use it to sign certificates.

It appears that foolscap, before the fix to <http://foolscap.lothar.com/trac/ticket/141>, generates certificates that use RSA with MD5 as the signature algorithm. Although RSA-MD5 is listed in the algorithms in [comment:398921](/tahoe-lafs/trac/issues/2400#issuecomment-398921), it may be that OpenSSL in FIPS mode is (not entirely unreasonably) refusing to use it to sign certificates.

daira commented

2015-04-13 19:41:00 +00:00

Author

This was indeed the cause. There's another change needed to foolscap needed, as described at (@@http://foolscap.lothar.com/trac/ticket/141#comment:-1@@).

Here's what Gabe wrote about the debugging of this problem:

FWIW, looks like the error in twisted is coming from a method that doesn't directly take digestAlgorithm as a parameter--the "sha256" argument change in the patch goes into /usr/lib64/python2.7/site-packages/twisted/internet/_sslverify.py in the signCertificateRequest method just fine (verified via printing statements), but the error is coming from the load method, and signCertificateRequest doesn't pass digestAlgorithm on to load as a parameter. Don't really know if that's useful, since I'm not familiar with these packages at all, but it makes sense to me that the patch wouldn't affect this particular error, since the argument doesn't influence the call site of the error.

I wrote the requestData parameter to a file, however, and toyed around with it using the system openssl, to avoid any pythonic errors and came up with the following output (openssl.cnf attached):

 ~  openssl req -noout -text -sha256 -inform der -verify -verbose -modulus -in tahoeReqData
Using configuration from /etc/pki/tls/openssl.cnf
140135384266608:error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm:a_verify.c:191:

 ~  openssl req -noout -text -md5 -inform der -verify -verbose -modulus -in tahoeReqData
Using configuration from /etc/pki/tls/openssl.cnf
140343873476464:error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm:a_verify.c:191:

 ~  openssl req -noout -text -md5 -inform der -verify -verbose -in tahoeReqData
Using configuration from /etc/pki/tls/openssl.cnf
140596394153840:error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm:a_verify.c:191:

 ~  openssl req -noout -text -sha256 -inform der -verify -verbose -in tahoeReqData
Using configuration from /etc/pki/tls/openssl.cnf
140197933606768:error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm:a_verify.c:191:
gabeos  ~  openssl req -noout -text -inform der -verify -verbose -in tahoeReqData
Using configuration from /etc/pki/tls/openssl.cnf
140231916164976:error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm:a_verify.c:191:

 ~  openssl req -noout -text -inform der -verbose -in tahoeReqData
Using configuration from /etc/pki/tls/openssl.cnf
Certificate Request:
Data:
Version: 0 (0x0)
Subject: CN=newpb_thingy
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:99:ff:6c:b8:ad:68:bc:42:c7:e9:9d:c7:5b:b3:
c2:50:84:b3:ad:0e:cc:fa:01:e6:27:8a:87:24:1a:
20:e2:31:54:86:e0:8a:18:46:dd:5b:7d:92:28:5c:
05:14:c8:39:cc:15:33:72:65:f0:c2:cf:27:62:68:
a4:ef:0a:b5:63:f5:91:fe:32:06:69:ad:76:67:1e:
bb:5c:a8:b0:63:87:e2:eb:73:d7:18:15:9b:f3:75:
0a:7a:c4:f8:6d:f5:4a:a8:a8:d7:c1:3f:1b:45:f6:
d1:f7:4a:a5:5f:3a:91:e4:4b:4d:cb:ce:25:22:75:
ce:24:18:31:df:e5:7e:7d:c4:28:a5:13:bd:de:fe:
7c:1d:ee:13:d6:ae:87:d0:9a:56:3d:f8:64:e1:46:
69:de:db:96:26:28:e2:ad:83:db:02:8c:50:39:71:
e3:d9:4c:c3:1d:f1:ef:6c:d8:38:a1:46:c0:52:48:
db:7c:75:7a:5e:04:17:08:76:d3:3d:a7:c0:2a:2b:
06:d6:60:fd:9b:18:74:b8:b1:3e:fb:52:68:3c:c3:
6b:68:e9:c4:20:a8:15:69:27:eb:32:3d:65:4b:c4:
1a:27:4c:6d:b8:cc:ce:4f:7c:32:9d:c7:5d:b9:ad:
03:7f:11:36:55:f7:2a:97:d6:23:5c:67:c7:15:cf:
74:57
Exponent: 65537 (0x10001)
Attributes:
a0:00
Signature Algorithm: md5WithRSAEncryption
48:a3:3c:fe:fa:0a:26:b7:d6:2c:de:48:a9:d3:8b:67:79:d1:
dc:cc:e4:ab:8b:59:af:17:5a:d1:6c:40:43:27:34:2f:bb:f1:
b8:50:81:9a:92:d1:6c:8a:ee:0c:fd:b1:06:c7:12:fe:ee:d0:
42:8a:84:70:ce:69:0c:a2:a7:41:4c:71:ee:26:df:e5:37:a7:
a2:93:8f:b4:6c:74:f8:5d:b2:5f:a1:83:45:c3:f0:7b:31:a9:
7f:5c:9e:8c:eb:a5:d7:dd:ed:4b:39:3c:6f:8b:e3:5c:13:b5:
e0:23:26:47:0a:e1:4b:00:fc:91:cd:6d:de:d3:2b:d7:b5:17:
e7:7d:f1:a4:da:3f:af:78:22:dc:4f:26:92:f3:1c:53:a5:3f:
c4:4c:ad:11:21:49:64:b8:9f:d4:ef:1d:0c:cb:14:17:63:b7:
84:81:2f:d8:d1:00:c6:44:b1:f9:24:a6:80:92:88:17:b3:58:
4c:30:29:80:96:54:e1:de:ee:88:44:cb:16:3d:04:6d:5b:04:
09:b9:52:88:12:c5:4d:5b:b4:87:f3:aa:a2:51:d7:fa:a5:29:
9d:63:fd:90:b1:f5:b7:28:48:cc:61:a0:64:da:c4:ee:68:f9:
fc:f6:e6:24:c8:3a:33:ac:54:c4:4a:33:81:f5:d9:62:1f:9b:
49:5b:99:14

Seeing that it's still md5, I changed the foolscap/pb.py code to include digestAlgorithm="sha256" in the keypair.certificateRequest(..) method [...] which appears to have solved the problem.

I have the openssl output:

openssl req -noout -text -inform der -verbose -verify -in tahoeReqDataSHA256
Using configuration from /etc/pki/tls/openssl.cnf
verify OK
Certificate Request:
Data:
Version: 0 (0x0)
Subject: CN=newpb_thingy
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:98:17:5d:bc:ef:7d:d3:b1:fd:f1:ea:02:5d:9d:
48:06:c4:4c:75:74:e5:2e:0d:09:c0:a1:58:d6:fe:
d7:db:4d:2d:85:93:45:a2:3c:e7:55:08:b2:fb:9f:
7c:c9:d0:47:13:9f:60:33:78:42:5c:d9:5f:8e:a6:
0f:9b:90:38:ea:af:50:0b:51:16:33:79:58:16:e5:
28:b2:a1:6a:64:df:b1:f4:91:c3:0d:5c:25:49:6b:
44:6b:c1:88:4e:96:c9:81:fe:08:56:7d:0e:3c:40:
60:a4:51:6d:93:21:79:90:7d:ad:f9:de:fc:36:35:
51:82:bf:be:43:3e:0d:6e:26:c8:18:a8:44:44:3a:
72:7a:e6:0d:1c:93:e4:5c:45:5c:04:e5:7d:ef:2c:
0b:0c:76:4b:d3:85:24:c2:0d:d6:0c:51:2f:08:29:
5b:c0:98:5f:30:1d:a0:2e:ae:e9:e5:3d:b5:5d:79:
58:92:8f:0c:a8:10:61:1f:5a:62:81:85:fc:0c:c9:
09:9a:a3:84:13:52:74:37:ea:a1:87:93:70:86:0a:
52:02:c8:91:28:0e:05:13:18:81:3f:d2:d7:a1:7c:
54:20:17:fd:af:f3:59:82:29:73:0f:66:41:40:55:
79:f9:a3:78:17:34:33:61:b8:76:f4:ec:c6:14:f2:
10:25
Exponent: 65537 (0x10001)
Attributes:
a0:00
Signature Algorithm: sha256WithRSAEncryption
6e:2d:a2:51:3a:29:b6:3a:8a:56:43:1f:85:da:17:12:82:4f:
b3:4f:0e:3d:fc:e1:83:f5:f0:0f:a1:42:af:33:43:78:66:d0:
f3:b3:9f:14:7e:5a:bd:e6:c8:3a:a8:2b:54:e8:b8:f4:06:09:
cf:a4:87:74:df:27:d3:18:61:f1:eb:7f:1a:48:35:92:70:09:
99:f7:85:4f:fb:0f:b1:6e:8e:2b:f1:f3:d5:9d:a2:8b:3f:bf:
5f:7f:82:36:93:26:94:f4:a4:ae:48:db:a0:b7:49:44:c3:a9:
6f:16:13:25:aa:34:4f:b9:26:15:59:96:2e:f7:ea:9f:eb:a9:
1a:e8:78:0e:2f:b5:69:65:20:3c:7a:30:e2:9b:09:f3:26:17:
36:2d:a8:2d:55:22:94:49:f7:84:da:e9:7a:54:a8:bb:7e:ce:
98:94:2a:e1:0a:14:45:db:fd:89:b1:ab:10:49:78:69:2b:36:
21:fa:8b:9b:1b:f2:55:ab:4c:65:07:92:ed:92:03:89:89:f6:
4e:da:2f:eb:6d:a5:7a:73:02:21:cd:4c:f2:41:62:47:0b:57:
b8:43:6f:93:0a:9a:2c:c7:79:75:51:d7:68:41:62:52:7e:ad:
10:10:97:cd:b2:db:7c:22:90:82:c9:c8:f0:08:1d:4d:ff:03:
d6:8e:ff:89

and

> ./allmydata-tahoe-1.10.0/bin/tahoe start
STARTING '/home/gabeos/.tahoe'

 > ps aux | grep tahoe
gabeos 14821 0.0 0.5 401200 65632 ? Sl 11:25 0:00 /usr/bin/python /home/gabeos/allmydata-tahoe-1.10.0/support/bin/tahoe start

So it was the correct site to be patching, the patch just didn't cover everything since twisted has md5 as the default digest algorithm param.

New foolscap/pb.py method should look like so:

    def createCertificate(self):
        # this is copied from test_sslverify.py
        dn = crypto.DistinguishedName(commonName="newpb_thingy")
        keypair = crypto.KeyPair.generate(size=2048)
        # Following line is the change that wasn't in the patch.
        # Otherwise req has signature algorithm md5WithRSAEncryption.
        # Should be sha256WithRSAEncryption for OpenSSL 1.0.1k-fips
        req = keypair.certificateRequest(dn, digestAlgorithm="sha256")
        certData = keypair.signCertificateRequest(dn, req,
                                                  lambda dn: True,
                                                  1, # serial number
                                                  digestAlgorithm="sha256",
                                                  )
        cert = keypair.newCertificate(certData)
        #opts = cert.options()
        # 'opts' can be given to reactor.listenSSL, or to transport.startTLS

        return cert

This was indeed the cause. There's another change needed to foolscap needed, as described at (@@http://foolscap.lothar.com/trac/ticket/141#[comment:-1](/tahoe-lafs/trac/issues/2400#issuecomment--1)@@). Here's what Gabe wrote about the debugging of this problem: > FWIW, looks like the error in twisted is coming from a method that doesn't directly take `digestAlgorithm` as a parameter--the `"sha256"` argument change in the patch goes into `/usr/lib64/python2.7/site-packages/twisted/internet/_sslverify.py` in the `signCertificateRequest` method just fine (verified via printing statements), but the error is coming from the `load` method, and `signCertificateRequest` doesn't pass `digestAlgorithm` on to `load` as a parameter. Don't really know if that's useful, since I'm not familiar with these packages at all, but it makes sense to me that the patch wouldn't affect this particular error, since the argument doesn't influence the call site of the error. > > I wrote the `requestData` parameter to a file, however, and toyed around with it using the system openssl, to avoid any pythonic errors and came up with the following output (`openssl.cnf` [attached](/tahoe-lafs/trac/attachments/000078ac-3f17-1a37-11d4-b434d448f1b1)): ```  ~  openssl req -noout -text -sha256 -inform der -verify -verbose -modulus -in tahoeReqData Using configuration from /etc/pki/tls/openssl.cnf 140135384266608:error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm:a_verify.c:191:  ~  openssl req -noout -text -md5 -inform der -verify -verbose -modulus -in tahoeReqData Using configuration from /etc/pki/tls/openssl.cnf 140343873476464:error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm:a_verify.c:191:  ~  openssl req -noout -text -md5 -inform der -verify -verbose -in tahoeReqData Using configuration from /etc/pki/tls/openssl.cnf 140596394153840:error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm:a_verify.c:191:  ~  openssl req -noout -text -sha256 -inform der -verify -verbose -in tahoeReqData Using configuration from /etc/pki/tls/openssl.cnf 140197933606768:error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm:a_verify.c:191: gabeos  ~  openssl req -noout -text -inform der -verify -verbose -in tahoeReqData Using configuration from /etc/pki/tls/openssl.cnf 140231916164976:error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm:a_verify.c:191:  ~  openssl req -noout -text -inform der -verbose -in tahoeReqData Using configuration from /etc/pki/tls/openssl.cnf Certificate Request: Data: Version: 0 (0x0) Subject: CN=newpb_thingy Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:99:ff:6c:b8:ad:68:bc:42:c7:e9:9d:c7:5b:b3: c2:50:84:b3:ad:0e:cc:fa:01:e6:27:8a:87:24:1a: 20:e2:31:54:86:e0:8a:18:46:dd:5b:7d:92:28:5c: 05:14:c8:39:cc:15:33:72:65:f0:c2:cf:27:62:68: a4:ef:0a:b5:63:f5:91:fe:32:06:69:ad:76:67:1e: bb:5c:a8:b0:63:87:e2:eb:73:d7:18:15:9b:f3:75: 0a:7a:c4:f8:6d:f5:4a:a8:a8:d7:c1:3f:1b:45:f6: d1:f7:4a:a5:5f:3a:91:e4:4b:4d:cb:ce:25:22:75: ce:24:18:31:df:e5:7e:7d:c4:28:a5:13:bd:de:fe: 7c:1d:ee:13:d6:ae:87:d0:9a:56:3d:f8:64:e1:46: 69:de:db:96:26:28:e2:ad:83:db:02:8c:50:39:71: e3:d9:4c:c3:1d:f1:ef:6c:d8:38:a1:46:c0:52:48: db:7c:75:7a:5e:04:17:08:76:d3:3d:a7:c0:2a:2b: 06:d6:60:fd:9b:18:74:b8:b1:3e:fb:52:68:3c:c3: 6b:68:e9:c4:20:a8:15:69:27:eb:32:3d:65:4b:c4: 1a:27:4c:6d:b8:cc:ce:4f:7c:32:9d:c7:5d:b9:ad: 03:7f:11:36:55:f7:2a:97:d6:23:5c:67:c7:15:cf: 74:57 Exponent: 65537 (0x10001) Attributes: a0:00 Signature Algorithm: md5WithRSAEncryption 48:a3:3c:fe:fa:0a:26:b7:d6:2c:de:48:a9:d3:8b:67:79:d1: dc:cc:e4:ab:8b:59:af:17:5a:d1:6c:40:43:27:34:2f:bb:f1: b8:50:81:9a:92:d1:6c:8a:ee:0c:fd:b1:06:c7:12:fe:ee:d0: 42:8a:84:70:ce:69:0c:a2:a7:41:4c:71:ee:26:df:e5:37:a7: a2:93:8f:b4:6c:74:f8:5d:b2:5f:a1:83:45:c3:f0:7b:31:a9: 7f:5c:9e:8c:eb:a5:d7:dd:ed:4b:39:3c:6f:8b:e3:5c:13:b5: e0:23:26:47:0a:e1:4b:00:fc:91:cd:6d:de:d3:2b:d7:b5:17: e7:7d:f1:a4:da:3f:af:78:22:dc:4f:26:92:f3:1c:53:a5:3f: c4:4c:ad:11:21:49:64:b8:9f:d4:ef:1d:0c:cb:14:17:63:b7: 84:81:2f:d8:d1:00:c6:44:b1:f9:24:a6:80:92:88:17:b3:58: 4c:30:29:80:96:54:e1:de:ee:88:44:cb:16:3d:04:6d:5b:04: 09:b9:52:88:12:c5:4d:5b:b4:87:f3:aa:a2:51:d7:fa:a5:29: 9d:63:fd:90:b1:f5:b7:28:48:cc:61:a0:64:da:c4:ee:68:f9: fc:f6:e6:24:c8:3a:33:ac:54:c4:4a:33:81:f5:d9:62:1f:9b: 49:5b:99:14 ``` > Seeing that it's still md5, I changed the `foolscap/pb.py` code to include `digestAlgorithm="sha256"` in the `keypair.certificateRequest(..)` method [...] which appears to have solved the problem. > > I have the openssl output: ``` openssl req -noout -text -inform der -verbose -verify -in tahoeReqDataSHA256 Using configuration from /etc/pki/tls/openssl.cnf verify OK Certificate Request: Data: Version: 0 (0x0) Subject: CN=newpb_thingy Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:98:17:5d:bc:ef:7d:d3:b1:fd:f1:ea:02:5d:9d: 48:06:c4:4c:75:74:e5:2e:0d:09:c0:a1:58:d6:fe: d7:db:4d:2d:85:93:45:a2:3c:e7:55:08:b2:fb:9f: 7c:c9:d0:47:13:9f:60:33:78:42:5c:d9:5f:8e:a6: 0f:9b:90:38:ea:af:50:0b:51:16:33:79:58:16:e5: 28:b2:a1:6a:64:df:b1:f4:91:c3:0d:5c:25:49:6b: 44:6b:c1:88:4e:96:c9:81:fe:08:56:7d:0e:3c:40: 60:a4:51:6d:93:21:79:90:7d:ad:f9:de:fc:36:35: 51:82:bf:be:43:3e:0d:6e:26:c8:18:a8:44:44:3a: 72:7a:e6:0d:1c:93:e4:5c:45:5c:04:e5:7d:ef:2c: 0b:0c:76:4b:d3:85:24:c2:0d:d6:0c:51:2f:08:29: 5b:c0:98:5f:30:1d:a0:2e:ae:e9:e5:3d:b5:5d:79: 58:92:8f:0c:a8:10:61:1f:5a:62:81:85:fc:0c:c9: 09:9a:a3:84:13:52:74:37:ea:a1:87:93:70:86:0a: 52:02:c8:91:28:0e:05:13:18:81:3f:d2:d7:a1:7c: 54:20:17:fd:af:f3:59:82:29:73:0f:66:41:40:55: 79:f9:a3:78:17:34:33:61:b8:76:f4:ec:c6:14:f2: 10:25 Exponent: 65537 (0x10001) Attributes: a0:00 Signature Algorithm: sha256WithRSAEncryption 6e:2d:a2:51:3a:29:b6:3a:8a:56:43:1f:85:da:17:12:82:4f: b3:4f:0e:3d:fc:e1:83:f5:f0:0f:a1:42:af:33:43:78:66:d0: f3:b3:9f:14:7e:5a:bd:e6:c8:3a:a8:2b:54:e8:b8:f4:06:09: cf:a4:87:74:df:27:d3:18:61:f1:eb:7f:1a:48:35:92:70:09: 99:f7:85:4f:fb:0f:b1:6e:8e:2b:f1:f3:d5:9d:a2:8b:3f:bf: 5f:7f:82:36:93:26:94:f4:a4:ae:48:db:a0:b7:49:44:c3:a9: 6f:16:13:25:aa:34:4f:b9:26:15:59:96:2e:f7:ea:9f:eb:a9: 1a:e8:78:0e:2f:b5:69:65:20:3c:7a:30:e2:9b:09:f3:26:17: 36:2d:a8:2d:55:22:94:49:f7:84:da:e9:7a:54:a8:bb:7e:ce: 98:94:2a:e1:0a:14:45:db:fd:89:b1:ab:10:49:78:69:2b:36: 21:fa:8b:9b:1b:f2:55:ab:4c:65:07:92:ed:92:03:89:89:f6: 4e:da:2f:eb:6d:a5:7a:73:02:21:cd:4c:f2:41:62:47:0b:57: b8:43:6f:93:0a:9a:2c:c7:79:75:51:d7:68:41:62:52:7e:ad: 10:10:97:cd:b2:db:7c:22:90:82:c9:c8:f0:08:1d:4d:ff:03: d6:8e:ff:89 ``` > and ``` > ./allmydata-tahoe-1.10.0/bin/tahoe start STARTING '/home/gabeos/.tahoe' > ps aux | grep tahoe gabeos 14821 0.0 0.5 401200 65632 ? Sl 11:25 0:00 /usr/bin/python /home/gabeos/allmydata-tahoe-1.10.0/support/bin/tahoe start ``` > So it was the correct site to be patching, the patch just didn't cover everything since twisted has md5 as the default digest algorithm param. > > New foolscap/pb.py method should look like so: ``` def createCertificate(self): # this is copied from test_sslverify.py dn = crypto.DistinguishedName(commonName="newpb_thingy") keypair = crypto.KeyPair.generate(size=2048) # Following line is the change that wasn't in the patch. # Otherwise req has signature algorithm md5WithRSAEncryption. # Should be sha256WithRSAEncryption for OpenSSL 1.0.1k-fips req = keypair.certificateRequest(dn, digestAlgorithm="sha256") certData = keypair.signCertificateRequest(dn, req, lambda dn: True, 1, # serial number digestAlgorithm="sha256", ) cert = keypair.newCertificate(certData) #opts = cert.options() # 'opts' can be given to reactor.listenSSL, or to transport.startTLS return cert ```

daira commented

2015-04-13 19:42:05 +00:00

Author

Attachment openssl.cnf (11068 bytes) added

**Attachment** openssl.cnf (11068 bytes) added

openssl.cnf

11 KiB

daira commented

2015-04-13 19:46:52 +00:00

Author

When this fix is released in foolscap 0.8, we should depend on that version, and write a unit test verifying that the foolscap cert generated for a Tahoe node has a 2048-bit key and sha256WithRSAEncryption as the algorithm.

When this fix is released in foolscap 0.8, we should depend on that version, and write a unit test verifying that the foolscap cert generated for a Tahoe node has a 2048-bit key and `sha256WithRSAEncryption` as the algorithm.

daira added

c/packaging

and removed

c/code-network

labels 2015-04-13 19:49:14 +00:00

daira modified the milestone from undecided to 1.10.1

2015-04-13 19:49:14 +00:00

daira commented

2015-04-13 23:32:41 +00:00

Author

Foolscap (master) fixed in http://foolscap.lothar.com/trac/changeset/2a76437af31c97bdbfdc65ffda4d0cb60a97684a/.

Foolscap (master) fixed in <http://foolscap.lothar.com/trac/changeset/2a76437af31c97bdbfdc65ffda4d0cb60a97684a/>.

warner commented

2015-04-17 03:04:10 +00:00

I released foolscap-0.8.0 with this fix a few days ago, please re-test with a dependency on that version.

daira commented

2015-04-21 17:37:16 +00:00

Author

I will ask Gabe to re-test this.

daira commented

2015-04-22 08:36:53 +00:00

Author

Gabe confirmed that it is fixed in foolscap 0.8.0.

We still need to decide whether Tahoe-LAFS 1.10.1 will require foolscap 0.8.0. If it does, we can then include the test described in comment:398924 .

Gabe confirmed that it is fixed in foolscap 0.8.0. We still need to decide whether Tahoe-LAFS 1.10.1 will require foolscap 0.8.0. If it does, we can then include the test described in [comment:398924](/tahoe-lafs/trac/issues/2400#issuecomment-398924) .