Mercurial > cpython-lru_cache-weakref

annotate Lib/hashlib.py @ 87690:66c1c9f32567 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add use_weakref option to functools.lru_cache The use_weakref option makes lru_cache create weak references to the function arguments when creating the cache key, which prevents them from staying in the memory forever even after all other references to them are gone. This is a proof of concept for http://bugs.python.org/issue19859
author Radomir Dopieralski <devel@sheep.art.pl>
date Tue, 03 Dec 2013 16:02:48 +0100
parents 72d7b2185771
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
86467
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
1 #. Copyright (C) 2005-2010 Gregory P. Smith (greg@krypto.org)
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
2 # Licensed to PSF under a Contributor Agreement.
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
3 #
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
4
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
5 __doc__ = """hashlib module - A common interface to many hash functions.
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
6
42918
9220931e4741 More changes needed to make things work once bytes and str are truly divorced.
Guido van Rossum <guido@python.org>
parents: 42849
diff changeset
7 new(name, data=b'') - returns a new hash object implementing the
9220931e4741 More changes needed to make things work once bytes and str are truly divorced.
Guido van Rossum <guido@python.org>
parents: 42849
diff changeset
8 given hash function; initializing the hash
9220931e4741 More changes needed to make things work once bytes and str are truly divorced.
Guido van Rossum <guido@python.org>
parents: 42849
diff changeset
9 using the given binary data.
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
10
43294
78f1c329c983 Adds stand alone _md5 and _sha1 modules for use by hashlib on systems
Gregory P. Smith <greg@mad-scientist.com>
parents: 43020
diff changeset
11 Named constructor functions are also available, these are faster
78f1c329c983 Adds stand alone _md5 and _sha1 modules for use by hashlib on systems
Gregory P. Smith <greg@mad-scientist.com>
parents: 43020
diff changeset
12 than using new(name):
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
13
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
14 md5(), sha1(), sha224(), sha256(), sha384(), and sha512()
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
15
64488
49441e88ccc1 hashlib has two new constant attributes: algorithms_guaranteed and
Gregory P. Smith <greg@mad-scientist.com>
parents: 59212
diff changeset
16 More algorithms may be available on your platform but the above are guaranteed
49441e88ccc1 hashlib has two new constant attributes: algorithms_guaranteed and
Gregory P. Smith <greg@mad-scientist.com>
parents: 59212
diff changeset
17 to exist. See the algorithms_guaranteed and algorithms_available attributes
49441e88ccc1 hashlib has two new constant attributes: algorithms_guaranteed and
Gregory P. Smith <greg@mad-scientist.com>
parents: 59212
diff changeset
18 to find out what algorithm names can be passed to new().
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
19
45931
1657f0acca78 Merged revisions 61538-61540,61556,61559-61560,61563,61565,61571,61575-61576,61580-61582,61586,61591,61593,61595,61605-61606,61613-61616,61618,61621-61623,61625,61627,61631-61634 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43294
diff changeset
20 NOTE: If you want the adler32 or crc32 hash functions they are available in
1657f0acca78 Merged revisions 61538-61540,61556,61559-61560,61563,61565,61571,61575-61576,61580-61582,61586,61591,61593,61595,61605-61606,61613-61616,61618,61621-61623,61625,61627,61631-61634 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43294
diff changeset
21 the zlib module.
1657f0acca78 Merged revisions 61538-61540,61556,61559-61560,61563,61565,61571,61575-61576,61580-61582,61586,61591,61593,61595,61605-61606,61613-61616,61618,61621-61623,61625,61627,61631-61634 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43294
diff changeset
22
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
23 Choose your hash function wisely. Some have known collision weaknesses.
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
24 sha384 and sha512 will be slow on 32 bit platforms.
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
25
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
26 Hash objects have these methods:
43294
78f1c329c983 Adds stand alone _md5 and _sha1 modules for use by hashlib on systems
Gregory P. Smith <greg@mad-scientist.com>
parents: 43020
diff changeset
27 - update(arg): Update the hash object with the bytes in arg. Repeated calls
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
28 are equivalent to a single call with the concatenation of all
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
29 the arguments.
43294
78f1c329c983 Adds stand alone _md5 and _sha1 modules for use by hashlib on systems
Gregory P. Smith <greg@mad-scientist.com>
parents: 43020
diff changeset
30 - digest(): Return the digest of the bytes passed to the update() method
78f1c329c983 Adds stand alone _md5 and _sha1 modules for use by hashlib on systems
Gregory P. Smith <greg@mad-scientist.com>
parents: 43020
diff changeset
31 so far.
78f1c329c983 Adds stand alone _md5 and _sha1 modules for use by hashlib on systems
Gregory P. Smith <greg@mad-scientist.com>
parents: 43020
diff changeset
32 - hexdigest(): Like digest() except the digest is returned as a unicode
78f1c329c983 Adds stand alone _md5 and _sha1 modules for use by hashlib on systems
Gregory P. Smith <greg@mad-scientist.com>
parents: 43020
diff changeset
33 object of double length, containing only hexadecimal digits.
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
34 - copy(): Return a copy (clone) of the hash object. This can be used to
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
35 efficiently compute the digests of strings that share a common
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
36 initial substring.
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
37
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
38 For example, to obtain the digest of the string 'Nobody inspects the
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
39 spammish repetition':
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
40
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
41 >>> import hashlib
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
42 >>> m = hashlib.md5()
42918
9220931e4741 More changes needed to make things work once bytes and str are truly divorced.
Guido van Rossum <guido@python.org>
parents: 42849
diff changeset
43 >>> m.update(b"Nobody inspects")
9220931e4741 More changes needed to make things work once bytes and str are truly divorced.
Guido van Rossum <guido@python.org>
parents: 42849
diff changeset
44 >>> m.update(b" the spammish repetition")
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
45 >>> m.digest()
48916
7e194b9cc0a2 issue3715: docstring representation of hex escaped string needs to be double
Gregory P. Smith <greg@mad-scientist.com>
parents: 45931
diff changeset
46 b'\\xbbd\\x9c\\x83\\xdd\\x1e\\xa5\\xc9\\xd9\\xde\\xc9\\xa1\\x8d\\xf0\\xff\\xe9'
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
47
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
48 More condensed:
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
49
42918
9220931e4741 More changes needed to make things work once bytes and str are truly divorced.
Guido van Rossum <guido@python.org>
parents: 42849
diff changeset
50 >>> hashlib.sha224(b"Nobody inspects the spammish repetition").hexdigest()
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
51 'a4337bc45a8fc544c03f52dc550cd6e1e87021bc896588bd79e901e2'
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39782
diff changeset
52
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
53 """
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
54
55669
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
55 # This tuple and __get_builtin_constructor() must be modified if a new
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
56 # always available algorithm is added.
79502
11c9a894680e Issue #16113: integrade SHA-3 (Keccak) patch from http://hg.python.org/sandbox/cheimes
Christian Heimes <christian@cheimes.de>
parents: 78235
diff changeset
57 __always_supported = ('md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512',
11c9a894680e Issue #16113: integrade SHA-3 (Keccak) patch from http://hg.python.org/sandbox/cheimes
Christian Heimes <christian@cheimes.de>
parents: 78235
diff changeset
58 'sha3_224', 'sha3_256', 'sha3_384', 'sha3_512')
55669
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
59
67733
6f40b4709e9a Make the type consistent for hashlib algorithm constants. (Reviewed by Benjamin).
Raymond Hettinger <python@rcn.com>
parents: 64488
diff changeset
60 algorithms_guaranteed = set(__always_supported)
6f40b4709e9a Make the type consistent for hashlib algorithm constants. (Reviewed by Benjamin).
Raymond Hettinger <python@rcn.com>
parents: 64488
diff changeset
61 algorithms_available = set(__always_supported)
59212
9b4e0cd7817a Merged revisions 78528 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 55669
diff changeset
62
64488
49441e88ccc1 hashlib has two new constant attributes: algorithms_guaranteed and
Gregory P. Smith <greg@mad-scientist.com>
parents: 59212
diff changeset
63 __all__ = __always_supported + ('new', 'algorithms_guaranteed',
86467
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
64 'algorithms_available', 'pbkdf2_hmac')
55669
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
65
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
66
86569
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
67 __builtin_constructor_cache = {}
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
68
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
69 def __get_builtin_constructor(name):
86569
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
70 cache = __builtin_constructor_cache
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
71 constructor = cache.get(name)
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
72 if constructor is not None:
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
73 return constructor
70122
ad9249f35914 Fixes Issue #12059: Properly handle missing hash functions even when
Gregory P. Smith <greg@krypto.org>
parents: 67733
diff changeset
74 try:
ad9249f35914 Fixes Issue #12059: Properly handle missing hash functions even when
Gregory P. Smith <greg@krypto.org>
parents: 67733
diff changeset
75 if name in ('SHA1', 'sha1'):
ad9249f35914 Fixes Issue #12059: Properly handle missing hash functions even when
Gregory P. Smith <greg@krypto.org>
parents: 67733
diff changeset
76 import _sha1
86569
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
77 cache['SHA1'] = cache['sha1'] = _sha1.sha1
70122
ad9249f35914 Fixes Issue #12059: Properly handle missing hash functions even when
Gregory P. Smith <greg@krypto.org>
parents: 67733
diff changeset
78 elif name in ('MD5', 'md5'):
ad9249f35914 Fixes Issue #12059: Properly handle missing hash functions even when
Gregory P. Smith <greg@krypto.org>
parents: 67733
diff changeset
79 import _md5
86569
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
80 cache['MD5'] = cache['md5'] = _md5.md5
70122
ad9249f35914 Fixes Issue #12059: Properly handle missing hash functions even when
Gregory P. Smith <greg@krypto.org>
parents: 67733
diff changeset
81 elif name in ('SHA256', 'sha256', 'SHA224', 'sha224'):
ad9249f35914 Fixes Issue #12059: Properly handle missing hash functions even when
Gregory P. Smith <greg@krypto.org>
parents: 67733
diff changeset
82 import _sha256
86569
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
83 cache['SHA224'] = cache['sha224'] = _sha256.sha224
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
84 cache['SHA256'] = cache['sha256'] = _sha256.sha256
70122
ad9249f35914 Fixes Issue #12059: Properly handle missing hash functions even when
Gregory P. Smith <greg@krypto.org>
parents: 67733
diff changeset
85 elif name in ('SHA512', 'sha512', 'SHA384', 'sha384'):
ad9249f35914 Fixes Issue #12059: Properly handle missing hash functions even when
Gregory P. Smith <greg@krypto.org>
parents: 67733
diff changeset
86 import _sha512
86569
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
87 cache['SHA384'] = cache['sha384'] = _sha512.sha384
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
88 cache['SHA512'] = cache['sha512'] = _sha512.sha512
79502
11c9a894680e Issue #16113: integrade SHA-3 (Keccak) patch from http://hg.python.org/sandbox/cheimes
Christian Heimes <christian@cheimes.de>
parents: 78235
diff changeset
89 elif name in {'sha3_224', 'sha3_256', 'sha3_384', 'sha3_512',
11c9a894680e Issue #16113: integrade SHA-3 (Keccak) patch from http://hg.python.org/sandbox/cheimes
Christian Heimes <christian@cheimes.de>
parents: 78235
diff changeset
90 'SHA3_224', 'SHA3_256', 'SHA3_384', 'SHA3_512'}:
11c9a894680e Issue #16113: integrade SHA-3 (Keccak) patch from http://hg.python.org/sandbox/cheimes
Christian Heimes <christian@cheimes.de>
parents: 78235
diff changeset
91 import _sha3
86569
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
92 cache['SHA3_224'] = cache['sha3_224'] = _sha3.sha3_224
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
93 cache['SHA3_256'] = cache['sha3_256'] = _sha3.sha3_256
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
94 cache['SHA3_384'] = cache['sha3_384'] = _sha3.sha3_384
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
95 cache['SHA3_512'] = cache['sha3_512'] = _sha3.sha3_512
84442
7769c4d72806 Issue #18200: Back out usage of ModuleNotFoundError (8d28d44f3a9a)
Brett Cannon <brett@python.org>
parents: 84114
diff changeset
96 except ImportError:
70127
c238bddf3596 fix whitespace
Gregory P. Smith <greg@krypto.org>
parents: 70122
diff changeset
97 pass # no extension module, this hash is unsupported.
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
98
86569
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
99 constructor = cache.get(name)
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
100 if constructor is not None:
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
101 return constructor
72d7b2185771 Issue #18742: Rework the internal hashlib construtor to pave the road for ABCs.
Christian Heimes <christian@cheimes.de>
parents: 86467
diff changeset
102
78235
c97cfe04880f Consistently raise a TypeError when a non str is passed to hashlib.new
Gregory P. Smith <greg@krypto.org>
parents: 70127
diff changeset
103 raise ValueError('unsupported hash type ' + name)
55669
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
104
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
105
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
106 def __get_openssl_constructor(name):
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
107 try:
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
108 f = getattr(_hashlib, 'openssl_' + name)
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
109 # Allow the C module to raise ValueError. The function will be
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
110 # defined but the hash not actually available thanks to OpenSSL.
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
111 f()
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
112 # Use the C function directly (very fast)
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
113 return f
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
114 except (AttributeError, ValueError):
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
115 return __get_builtin_constructor(name)
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
116
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
117
42918
9220931e4741 More changes needed to make things work once bytes and str are truly divorced.
Guido van Rossum <guido@python.org>
parents: 42849
diff changeset
118 def __py_new(name, data=b''):
43294
78f1c329c983 Adds stand alone _md5 and _sha1 modules for use by hashlib on systems
Gregory P. Smith <greg@mad-scientist.com>
parents: 43020
diff changeset
119 """new(name, data=b'') - Return a new hashing object using the named algorithm;
42918
9220931e4741 More changes needed to make things work once bytes and str are truly divorced.
Guido van Rossum <guido@python.org>
parents: 42849
diff changeset
120 optionally initialized with data (which must be bytes).
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
121 """
42918
9220931e4741 More changes needed to make things work once bytes and str are truly divorced.
Guido van Rossum <guido@python.org>
parents: 42849
diff changeset
122 return __get_builtin_constructor(name)(data)
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
123
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
124
42918
9220931e4741 More changes needed to make things work once bytes and str are truly divorced.
Guido van Rossum <guido@python.org>
parents: 42849
diff changeset
125 def __hash_new(name, data=b''):
9220931e4741 More changes needed to make things work once bytes and str are truly divorced.
Guido van Rossum <guido@python.org>
parents: 42849
diff changeset
126 """new(name, data=b'') - Return a new hashing object using the named algorithm;
9220931e4741 More changes needed to make things work once bytes and str are truly divorced.
Guido van Rossum <guido@python.org>
parents: 42849
diff changeset
127 optionally initialized with data (which must be bytes).
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
128 """
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
129 try:
42918
9220931e4741 More changes needed to make things work once bytes and str are truly divorced.
Guido van Rossum <guido@python.org>
parents: 42849
diff changeset
130 return _hashlib.new(name, data)
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
131 except ValueError:
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
132 # If the _hashlib module (OpenSSL) doesn't support the named
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
133 # hash, try using our builtin implementations.
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
134 # This allows for SHA224/256 and SHA384/512 support even though
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
135 # the OpenSSL library prior to 0.9.8 doesn't provide them.
42918
9220931e4741 More changes needed to make things work once bytes and str are truly divorced.
Guido van Rossum <guido@python.org>
parents: 42849
diff changeset
136 return __get_builtin_constructor(name)(data)
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
137
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
138
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
139 try:
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
140 import _hashlib
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
141 new = __hash_new
55669
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
142 __get_hash = __get_openssl_constructor
64488
49441e88ccc1 hashlib has two new constant attributes: algorithms_guaranteed and
Gregory P. Smith <greg@mad-scientist.com>
parents: 59212
diff changeset
143 algorithms_available = algorithms_available.union(
49441e88ccc1 hashlib has two new constant attributes: algorithms_guaranteed and
Gregory P. Smith <greg@mad-scientist.com>
parents: 59212
diff changeset
144 _hashlib.openssl_md_meth_names)
84442
7769c4d72806 Issue #18200: Back out usage of ModuleNotFoundError (8d28d44f3a9a)
Brett Cannon <brett@python.org>
parents: 84114
diff changeset
145 except ImportError:
55669
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
146 new = __py_new
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
147 __get_hash = __get_builtin_constructor
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
148
86264
5fd56d6d3fce Issue #18582: Add 'pbkdf2_hmac' to the hashlib module.
Christian Heimes <christian@cheimes.de>
parents: 84442
diff changeset
149 try:
86467
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
150 # OpenSSL's PKCS5_PBKDF2_HMAC requires OpenSSL 1.0+ with HMAC and SHA
86264
5fd56d6d3fce Issue #18582: Add 'pbkdf2_hmac' to the hashlib module.
Christian Heimes <christian@cheimes.de>
parents: 84442
diff changeset
151 from _hashlib import pbkdf2_hmac
5fd56d6d3fce Issue #18582: Add 'pbkdf2_hmac' to the hashlib module.
Christian Heimes <christian@cheimes.de>
parents: 84442
diff changeset
152 except ImportError:
86467
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
153 _trans_5C = bytes((x ^ 0x5C) for x in range(256))
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
154 _trans_36 = bytes((x ^ 0x36) for x in range(256))
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
155
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
156 def pbkdf2_hmac(hash_name, password, salt, iterations, dklen=None):
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
157 """Password based key derivation function 2 (PKCS #5 v2.0)
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
158
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
159 This Python implementations based on the hmac module about as fast
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
160 as OpenSSL's PKCS5_PBKDF2_HMAC for short passwords and much faster
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
161 for long passwords.
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
162 """
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
163 if not isinstance(hash_name, str):
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
164 raise TypeError(hash_name)
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
165
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
166 if not isinstance(password, (bytes, bytearray)):
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
167 password = bytes(memoryview(password))
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
168 if not isinstance(salt, (bytes, bytearray)):
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
169 salt = bytes(memoryview(salt))
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
170
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
171 # Fast inline HMAC implementation
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
172 inner = new(hash_name)
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
173 outer = new(hash_name)
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
174 blocksize = getattr(inner, 'block_size', 64)
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
175 if len(password) > blocksize:
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
176 password = new(hash_name, password).digest()
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
177 password = password + b'\x00' * (blocksize - len(password))
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
178 inner.update(password.translate(_trans_36))
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
179 outer.update(password.translate(_trans_5C))
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
180
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
181 def prf(msg, inner=inner, outer=outer):
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
182 # PBKDF2_HMAC uses the password as key. We can re-use the same
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
183 # digest objects and and just update copies to skip initialization.
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
184 icpy = inner.copy()
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
185 ocpy = outer.copy()
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
186 icpy.update(msg)
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
187 ocpy.update(icpy.digest())
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
188 return ocpy.digest()
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
189
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
190 if iterations < 1:
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
191 raise ValueError(iterations)
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
192 if dklen is None:
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
193 dklen = outer.digest_size
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
194 if dklen < 1:
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
195 raise ValueError(dklen)
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
196
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
197 dkey = b''
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
198 loop = 1
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
199 from_bytes = int.from_bytes
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
200 while len(dkey) < dklen:
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
201 prev = prf(salt + loop.to_bytes(4, 'big'))
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
202 # endianess doesn't matter here as long to / from use the same
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
203 rkey = int.from_bytes(prev, 'big')
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
204 for i in range(iterations - 1):
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
205 prev = prf(prev)
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
206 # rkey = rkey ^ prev
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
207 rkey ^= from_bytes(prev, 'big')
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
208 loop += 1
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
209 dkey += rkey.to_bytes(inner.digest_size, 'big')
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
210
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
211 return dkey[:dklen]
e73627483d2f Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
Christian Heimes <christian@cheimes.de>
parents: 86264
diff changeset
212
86264
5fd56d6d3fce Issue #18582: Add 'pbkdf2_hmac' to the hashlib module.
Christian Heimes <christian@cheimes.de>
parents: 84442
diff changeset
213
55669
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
214 for __func_name in __always_supported:
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
215 # try them all, some may not work due to the OpenSSL
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
216 # version not supporting that algorithm.
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
217 try:
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
218 globals()[__func_name] = __get_hash(__func_name)
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
219 except ValueError:
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
220 import logging
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
221 logging.exception('code for hash %s was not found.', __func_name)
35375
624918e1c1b2 [ sf.net patch # 1121611 ]
Gregory P. Smith <greg@mad-scientist.com>
parents:
diff changeset
222
55669
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
223 # Cleanup locals()
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
224 del __always_supported, __func_name, __get_hash
da21b480ff05 Merged revisions 74479 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 48916
diff changeset
225 del __py_new, __hash_new, __get_openssl_constructor