Mercurial > cpython-lru_cache-weakref

annotate Lib/functools.py @ 70817:dd775b1561ba

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Fix markup.
author Georg Brandl <georg@python.org>
date Tue, 14 Jun 2011 21:09:55 +0200
parents 444574740f5d
children 5f4a871759ea
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
1 """functools.py - Tools for working with functions and callable objects
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents:
diff changeset
2 """
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents:
diff changeset
3 # Python module wrapper for _functools C module
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents:
diff changeset
4 # to allow utilities written in Python to be added
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents:
diff changeset
5 # to the functools module.
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents:
diff changeset
6 # Written by Nick Coghlan <ncoghlan at gmail.com>
63370
c55876a6fa3c Re-commit r83327 now that the release is done.
Georg Brandl <georg@python.org>
parents: 63357
diff changeset
7 # and Raymond Hettinger <python at rcn.com>
c55876a6fa3c Re-commit r83327 now that the release is done.
Georg Brandl <georg@python.org>
parents: 63357
diff changeset
8 # Copyright (C) 2006-2010 Python Software Foundation.
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
9 # See C source code for _functools credits/copyright
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents:
diff changeset
10
63370
c55876a6fa3c Re-commit r83327 now that the release is done.
Georg Brandl <georg@python.org>
parents: 63357
diff changeset
11 __all__ = ['update_wrapper', 'wraps', 'WRAPPER_ASSIGNMENTS', 'WRAPPER_UPDATES',
64630
470a5360570c add reduce and partial to __all__
Benjamin Peterson <benjamin@python.org>
parents: 64502
diff changeset
12 'total_ordering', 'cmp_to_key', 'lru_cache', 'reduce', 'partial']
63370
c55876a6fa3c Re-commit r83327 now that the release is done.
Georg Brandl <georg@python.org>
parents: 63357
diff changeset
13
39691
c4d8c054616c Part of SF patch #1513870 (the still relevant part) -- add reduce() to
Guido van Rossum <guido@python.org>
parents: 38673
diff changeset
14 from _functools import partial, reduce
66550
312b2916cfd4 Issue 10586: change the new functools.lru_cache implementation to expose the maximum and current cache sizes through the public statistics API. This API is now a single function that returns a named tuple.
Nick Coghlan <ncoghlan@gmail.com>
parents: 66441
diff changeset
15 from collections import OrderedDict, namedtuple
64016
d8d0abe61e83 Add locks to make the caches well behaved in multi-threaded code.
Raymond Hettinger <python@rcn.com>
parents: 63823
diff changeset
16 try:
d8d0abe61e83 Add locks to make the caches well behaved in multi-threaded code.
Raymond Hettinger <python@rcn.com>
parents: 63823
diff changeset
17 from _thread import allocate_lock as Lock
d8d0abe61e83 Add locks to make the caches well behaved in multi-threaded code.
Raymond Hettinger <python@rcn.com>
parents: 63823
diff changeset
18 except:
d8d0abe61e83 Add locks to make the caches well behaved in multi-threaded code.
Raymond Hettinger <python@rcn.com>
parents: 63823
diff changeset
19 from _dummy_thread import allocate_lock as Lock
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
20
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
21 # update_wrapper() and wraps() are tools to help write
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
22 # wrapper functions that can handle naive introspection
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
23
63810
15839ca43d96 Issue 8814: functools.wraps() did not copy __annotations__.
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
24 WRAPPER_ASSIGNMENTS = ('__module__', '__name__', '__doc__', '__annotations__')
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
25 WRAPPER_UPDATES = ('__dict__',)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
26 def update_wrapper(wrapper,
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
27 wrapped,
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
28 assigned = WRAPPER_ASSIGNMENTS,
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
29 updated = WRAPPER_UPDATES):
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
30 """Update a wrapper function to look like the wrapped function
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents:
diff changeset
31
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
32 wrapper is the function to be updated
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
33 wrapped is the original function
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
34 assigned is a tuple naming the attributes assigned directly
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
35 from the wrapped function to the wrapper function (defaults to
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
36 functools.WRAPPER_ASSIGNMENTS)
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39691
diff changeset
37 updated is a tuple naming the attributes of the wrapper that
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
38 are updated with the corresponding attribute from the wrapped
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
39 function (defaults to functools.WRAPPER_UPDATES)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
40 """
64102
77bf58296c87 Document and test the resolution of issue 3445 (tolerate missing attributes in functools.update_wrapper, previously implemented as a side effect of the __annotations__ copying patch) and implement issue 9567 (add a __wrapped__ attribute when using update_wrapper)
Nick Coghlan <ncoghlan@gmail.com>
parents: 64030
diff changeset
41 wrapper.__wrapped__ = wrapped
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
42 for attr in assigned:
64102
77bf58296c87 Document and test the resolution of issue 3445 (tolerate missing attributes in functools.update_wrapper, previously implemented as a side effect of the __annotations__ copying patch) and implement issue 9567 (add a __wrapped__ attribute when using update_wrapper)
Nick Coghlan <ncoghlan@gmail.com>
parents: 64030
diff changeset
43 try:
77bf58296c87 Document and test the resolution of issue 3445 (tolerate missing attributes in functools.update_wrapper, previously implemented as a side effect of the __annotations__ copying patch) and implement issue 9567 (add a __wrapped__ attribute when using update_wrapper)
Nick Coghlan <ncoghlan@gmail.com>
parents: 64030
diff changeset
44 value = getattr(wrapped, attr)
77bf58296c87 Document and test the resolution of issue 3445 (tolerate missing attributes in functools.update_wrapper, previously implemented as a side effect of the __annotations__ copying patch) and implement issue 9567 (add a __wrapped__ attribute when using update_wrapper)
Nick Coghlan <ncoghlan@gmail.com>
parents: 64030
diff changeset
45 except AttributeError:
77bf58296c87 Document and test the resolution of issue 3445 (tolerate missing attributes in functools.update_wrapper, previously implemented as a side effect of the __annotations__ copying patch) and implement issue 9567 (add a __wrapped__ attribute when using update_wrapper)
Nick Coghlan <ncoghlan@gmail.com>
parents: 64030
diff changeset
46 pass
77bf58296c87 Document and test the resolution of issue 3445 (tolerate missing attributes in functools.update_wrapper, previously implemented as a side effect of the __annotations__ copying patch) and implement issue 9567 (add a __wrapped__ attribute when using update_wrapper)
Nick Coghlan <ncoghlan@gmail.com>
parents: 64030
diff changeset
47 else:
77bf58296c87 Document and test the resolution of issue 3445 (tolerate missing attributes in functools.update_wrapper, previously implemented as a side effect of the __annotations__ copying patch) and implement issue 9567 (add a __wrapped__ attribute when using update_wrapper)
Nick Coghlan <ncoghlan@gmail.com>
parents: 64030
diff changeset
48 setattr(wrapper, attr, value)
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
49 for attr in updated:
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39691
diff changeset
50 getattr(wrapper, attr).update(getattr(wrapped, attr, {}))
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
51 # Return the wrapper so this can be used as a decorator via partial()
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
52 return wrapper
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
53
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
54 def wraps(wrapped,
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
55 assigned = WRAPPER_ASSIGNMENTS,
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
56 updated = WRAPPER_UPDATES):
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
57 """Decorator factory to apply update_wrapper() to a wrapper function
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
58
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
59 Returns a decorator that invokes update_wrapper() with the decorated
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
60 function as the wrapper argument and the arguments to wraps() as the
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
61 remaining arguments. Default arguments are as for update_wrapper().
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
62 This is a convenience function to simplify applying partial() to
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
63 update_wrapper().
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
64 """
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
65 return partial(update_wrapper, wrapped=wrapped,
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
66 assigned=assigned, updated=updated)
60370
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
67
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
68 def total_ordering(cls):
61761
82a4483d9c4a Merged revisions 80068 via svnmerge from
Georg Brandl <georg@python.org>
parents: 60479
diff changeset
69 """Class decorator that fills in missing ordering methods"""
60370
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
70 convert = {
67458
6492a713887e Issue #10042: Fixed the total_ordering decorator to handle cross-type
Raymond Hettinger <python@rcn.com>
parents: 66582
diff changeset
71 '__lt__': [('__gt__', lambda self, other: not (self < other or self == other)),
6492a713887e Issue #10042: Fixed the total_ordering decorator to handle cross-type
Raymond Hettinger <python@rcn.com>
parents: 66582
diff changeset
72 ('__le__', lambda self, other: self < other or self == other),
60370
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
73 ('__ge__', lambda self, other: not self < other)],
67458
6492a713887e Issue #10042: Fixed the total_ordering decorator to handle cross-type
Raymond Hettinger <python@rcn.com>
parents: 66582
diff changeset
74 '__le__': [('__ge__', lambda self, other: not self <= other or self == other),
6492a713887e Issue #10042: Fixed the total_ordering decorator to handle cross-type
Raymond Hettinger <python@rcn.com>
parents: 66582
diff changeset
75 ('__lt__', lambda self, other: self <= other and not self == other),
60370
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
76 ('__gt__', lambda self, other: not self <= other)],
67458
6492a713887e Issue #10042: Fixed the total_ordering decorator to handle cross-type
Raymond Hettinger <python@rcn.com>
parents: 66582
diff changeset
77 '__gt__': [('__lt__', lambda self, other: not (self > other or self == other)),
6492a713887e Issue #10042: Fixed the total_ordering decorator to handle cross-type
Raymond Hettinger <python@rcn.com>
parents: 66582
diff changeset
78 ('__ge__', lambda self, other: self > other or self == other),
60370
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
79 ('__le__', lambda self, other: not self > other)],
67458
6492a713887e Issue #10042: Fixed the total_ordering decorator to handle cross-type
Raymond Hettinger <python@rcn.com>
parents: 66582
diff changeset
80 '__ge__': [('__le__', lambda self, other: (not self >= other) or self == other),
6492a713887e Issue #10042: Fixed the total_ordering decorator to handle cross-type
Raymond Hettinger <python@rcn.com>
parents: 66582
diff changeset
81 ('__gt__', lambda self, other: self >= other and not self == other),
60370
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
82 ('__lt__', lambda self, other: not self >= other)]
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
83 }
64776
985072cd31d0 Improve comment
Raymond Hettinger <python@rcn.com>
parents: 64755
diff changeset
84 # Find user-defined comparisons (not those inherited from object).
64755
f643d53c51bd Future proof total_ordering against changes in methods defined on object.
Raymond Hettinger <python@rcn.com>
parents: 64642
diff changeset
85 roots = [op for op in convert if getattr(cls, op, None) is not getattr(object, op, None)]
60479
805512329c75 Issue 8361: Remove assert from functools.total_ordering
Raymond Hettinger <python@rcn.com>
parents: 60370
diff changeset
86 if not roots:
805512329c75 Issue 8361: Remove assert from functools.total_ordering
Raymond Hettinger <python@rcn.com>
parents: 60370
diff changeset
87 raise ValueError('must define at least one ordering operation: < > <= >=')
805512329c75 Issue 8361: Remove assert from functools.total_ordering
Raymond Hettinger <python@rcn.com>
parents: 60370
diff changeset
88 root = max(roots) # prefer __lt__ to __le__ to __gt__ to __ge__
60370
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
89 for opname, opfunc in convert[root]:
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
90 if opname not in roots:
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
91 opfunc.__name__ = opname
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
92 opfunc.__doc__ = getattr(int, opname).__doc__
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
93 setattr(cls, opname, opfunc)
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
94 return cls
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
95
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
96 def cmp_to_key(mycmp):
61761
82a4483d9c4a Merged revisions 80068 via svnmerge from
Georg Brandl <georg@python.org>
parents: 60479
diff changeset
97 """Convert a cmp= function into a key= function"""
60370
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
98 class K(object):
68824
97799a54728e Issue #11628: cmp_to_key should use __slots__.
Raymond Hettinger <python@rcn.com>
parents: 68257
diff changeset
99 __slots__ = ['obj']
69862
9f58612bc334 Sync-up minor code edits with the default branch.
Raymond Hettinger <python@rcn.com>
parents: 69810
diff changeset
100 def __init__(self, obj):
60370
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
101 self.obj = obj
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
102 def __lt__(self, other):
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
103 return mycmp(self.obj, other.obj) < 0
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
104 def __gt__(self, other):
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
105 return mycmp(self.obj, other.obj) > 0
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
106 def __eq__(self, other):
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
107 return mycmp(self.obj, other.obj) == 0
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
108 def __le__(self, other):
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
109 return mycmp(self.obj, other.obj) <= 0
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
110 def __ge__(self, other):
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
111 return mycmp(self.obj, other.obj) >= 0
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
112 def __ne__(self, other):
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
113 return mycmp(self.obj, other.obj) != 0
69810
2675415c57c1 Fix __hash__ in functools.cmp_to_key() to work with collections.Hashable.
Raymond Hettinger <python@rcn.com>
parents: 68824
diff changeset
114 __hash__ = None
60370
3ce66c6922ab Forward port total_ordering() and cmp_to_key().
Raymond Hettinger <python@rcn.com>
parents: 40450
diff changeset
115 return K
63370
c55876a6fa3c Re-commit r83327 now that the release is done.
Georg Brandl <georg@python.org>
parents: 63357
diff changeset
116
69150
a03fb2fc3ed8 Issue #11707: Fast C version of functools.cmp_to_key()
Raymond Hettinger <python@rcn.com>
parents: 68825
diff changeset
117 try:
a03fb2fc3ed8 Issue #11707: Fast C version of functools.cmp_to_key()
Raymond Hettinger <python@rcn.com>
parents: 68825
diff changeset
118 from _functools import cmp_to_key
a03fb2fc3ed8 Issue #11707: Fast C version of functools.cmp_to_key()
Raymond Hettinger <python@rcn.com>
parents: 68825
diff changeset
119 except ImportError:
a03fb2fc3ed8 Issue #11707: Fast C version of functools.cmp_to_key()
Raymond Hettinger <python@rcn.com>
parents: 68825
diff changeset
120 pass
a03fb2fc3ed8 Issue #11707: Fast C version of functools.cmp_to_key()
Raymond Hettinger <python@rcn.com>
parents: 68825
diff changeset
121
66573
2b2fc4262170 Add example, tighten text, and minor clean-ups.
Raymond Hettinger <python@rcn.com>
parents: 66552
diff changeset
122 _CacheInfo = namedtuple("CacheInfo", "hits misses maxsize currsize")
66550
312b2916cfd4 Issue 10586: change the new functools.lru_cache implementation to expose the maximum and current cache sizes through the public statistics API. This API is now a single function that returns a named tuple.
Nick Coghlan <ncoghlan@gmail.com>
parents: 66441
diff changeset
123
63370
c55876a6fa3c Re-commit r83327 now that the release is done.
Georg Brandl <georg@python.org>
parents: 63357
diff changeset
124 def lru_cache(maxsize=100):
63823
1086dad50c74 use same quoting as the rest of the file
Benjamin Peterson <benjamin@python.org>
parents: 63745
diff changeset
125 """Least-recently-used cache decorator.
63370
c55876a6fa3c Re-commit r83327 now that the release is done.
Georg Brandl <georg@python.org>
parents: 63357
diff changeset
126
66582
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
127 If *maxsize* is set to None, the LRU features are disabled and the cache
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
128 can grow without bound.
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
129
63370
c55876a6fa3c Re-commit r83327 now that the release is done.
Georg Brandl <georg@python.org>
parents: 63357
diff changeset
130 Arguments to the cached function must be hashable.
66441
03d933948bf3 Clean-up docstring, comments, and whitespace.
Raymond Hettinger <python@rcn.com>
parents: 64776
diff changeset
131
66578
5cb53f094c1a Doc and docstring nits.
Raymond Hettinger <python@rcn.com>
parents: 66573
diff changeset
132 View the cache statistics named tuple (hits, misses, maxsize, currsize) with
66552
f16bf4cae5d7 Neaten-up a bit.
Raymond Hettinger <python@rcn.com>
parents: 66550
diff changeset
133 f.cache_info(). Clear the cache and statistics with f.cache_clear().
66578
5cb53f094c1a Doc and docstring nits.
Raymond Hettinger <python@rcn.com>
parents: 66573
diff changeset
134 Access the underlying function with f.__wrapped__.
66441
03d933948bf3 Clean-up docstring, comments, and whitespace.
Raymond Hettinger <python@rcn.com>
parents: 64776
diff changeset
135
03d933948bf3 Clean-up docstring, comments, and whitespace.
Raymond Hettinger <python@rcn.com>
parents: 64776
diff changeset
136 See: http://en.wikipedia.org/wiki/Cache_algorithms#Least_Recently_Used
63370
c55876a6fa3c Re-commit r83327 now that the release is done.
Georg Brandl <georg@python.org>
parents: 63357
diff changeset
137
63823
1086dad50c74 use same quoting as the rest of the file
Benjamin Peterson <benjamin@python.org>
parents: 63745
diff changeset
138 """
66441
03d933948bf3 Clean-up docstring, comments, and whitespace.
Raymond Hettinger <python@rcn.com>
parents: 64776
diff changeset
139 # Users should only access the lru_cache through its public API:
66552
f16bf4cae5d7 Neaten-up a bit.
Raymond Hettinger <python@rcn.com>
parents: 66550
diff changeset
140 # cache_info, cache_clear, and f.__wrapped__
66441
03d933948bf3 Clean-up docstring, comments, and whitespace.
Raymond Hettinger <python@rcn.com>
parents: 64776
diff changeset
141 # The internals of the lru_cache are encapsulated for thread safety and
03d933948bf3 Clean-up docstring, comments, and whitespace.
Raymond Hettinger <python@rcn.com>
parents: 64776
diff changeset
142 # to allow the implementation to change (including a possible C version).
03d933948bf3 Clean-up docstring, comments, and whitespace.
Raymond Hettinger <python@rcn.com>
parents: 64776
diff changeset
143
03d933948bf3 Clean-up docstring, comments, and whitespace.
Raymond Hettinger <python@rcn.com>
parents: 64776
diff changeset
144 def decorating_function(user_function,
03d933948bf3 Clean-up docstring, comments, and whitespace.
Raymond Hettinger <python@rcn.com>
parents: 64776
diff changeset
145 tuple=tuple, sorted=sorted, len=len, KeyError=KeyError):
03d933948bf3 Clean-up docstring, comments, and whitespace.
Raymond Hettinger <python@rcn.com>
parents: 64776
diff changeset
146
66550
312b2916cfd4 Issue 10586: change the new functools.lru_cache implementation to expose the maximum and current cache sizes through the public statistics API. This API is now a single function that returns a named tuple.
Nick Coghlan <ncoghlan@gmail.com>
parents: 66441
diff changeset
147 hits = misses = 0
69862
9f58612bc334 Sync-up minor code edits with the default branch.
Raymond Hettinger <python@rcn.com>
parents: 69810
diff changeset
148 kwd_mark = (object(),) # separates positional and keyword args
67538
df34bb0869bc Add comment.
Raymond Hettinger <python@rcn.com>
parents: 67458
diff changeset
149 lock = Lock() # needed because ordereddicts aren't threadsafe
63370
c55876a6fa3c Re-commit r83327 now that the release is done.
Georg Brandl <georg@python.org>
parents: 63357
diff changeset
150
66582
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
151 if maxsize is None:
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
152 cache = dict() # simple cache without ordering or size limit
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
153
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
154 @wraps(user_function)
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
155 def wrapper(*args, **kwds):
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
156 nonlocal hits, misses
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
157 key = args
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
158 if kwds:
69862
9f58612bc334 Sync-up minor code edits with the default branch.
Raymond Hettinger <python@rcn.com>
parents: 69810
diff changeset
159 key += kwd_mark + tuple(sorted(kwds.items()))
66582
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
160 try:
64016
d8d0abe61e83 Add locks to make the caches well behaved in multi-threaded code.
Raymond Hettinger <python@rcn.com>
parents: 63823
diff changeset
161 result = cache[key]
66550
312b2916cfd4 Issue 10586: change the new functools.lru_cache implementation to expose the maximum and current cache sizes through the public statistics API. This API is now a single function that returns a named tuple.
Nick Coghlan <ncoghlan@gmail.com>
parents: 66441
diff changeset
162 hits += 1
66582
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
163 except KeyError:
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
164 result = user_function(*args, **kwds)
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
165 cache[key] = result
66550
312b2916cfd4 Issue 10586: change the new functools.lru_cache implementation to expose the maximum and current cache sizes through the public statistics API. This API is now a single function that returns a named tuple.
Nick Coghlan <ncoghlan@gmail.com>
parents: 66441
diff changeset
166 misses += 1
66582
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
167 return result
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
168 else:
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
169 cache = OrderedDict() # ordered least recent to most recent
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
170 cache_popitem = cache.popitem
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
171 cache_renew = cache.move_to_end
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
172
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
173 @wraps(user_function)
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
174 def wrapper(*args, **kwds):
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
175 nonlocal hits, misses
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
176 key = args
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
177 if kwds:
69862
9f58612bc334 Sync-up minor code edits with the default branch.
Raymond Hettinger <python@rcn.com>
parents: 69810
diff changeset
178 key += kwd_mark + tuple(sorted(kwds.items()))
66582
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
179 try:
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
180 with lock:
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
181 result = cache[key]
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
182 cache_renew(key) # record recent use of this key
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
183 hits += 1
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
184 except KeyError:
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
185 result = user_function(*args, **kwds)
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
186 with lock:
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
187 cache[key] = result # record recent use of this key
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
188 misses += 1
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
189 if len(cache) > maxsize:
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
190 cache_popitem(0) # purge least recently used cache entry
00c8cdf53936 Issue 10593: Adopt Nick's suggestion for an lru_cache with maxsize=None.
Raymond Hettinger <python@rcn.com>
parents: 66578
diff changeset
191 return result
63370
c55876a6fa3c Re-commit r83327 now that the release is done.
Georg Brandl <georg@python.org>
parents: 63357
diff changeset
192
66550
312b2916cfd4 Issue 10586: change the new functools.lru_cache implementation to expose the maximum and current cache sizes through the public statistics API. This API is now a single function that returns a named tuple.
Nick Coghlan <ncoghlan@gmail.com>
parents: 66441
diff changeset
193 def cache_info():
66552
f16bf4cae5d7 Neaten-up a bit.
Raymond Hettinger <python@rcn.com>
parents: 66550
diff changeset
194 """Report cache statistics"""
66550
312b2916cfd4 Issue 10586: change the new functools.lru_cache implementation to expose the maximum and current cache sizes through the public statistics API. This API is now a single function that returns a named tuple.
Nick Coghlan <ncoghlan@gmail.com>
parents: 66441
diff changeset
195 with lock:
66573
2b2fc4262170 Add example, tighten text, and minor clean-ups.
Raymond Hettinger <python@rcn.com>
parents: 66552
diff changeset
196 return _CacheInfo(hits, misses, maxsize, len(cache))
66550
312b2916cfd4 Issue 10586: change the new functools.lru_cache implementation to expose the maximum and current cache sizes through the public statistics API. This API is now a single function that returns a named tuple.
Nick Coghlan <ncoghlan@gmail.com>
parents: 66441
diff changeset
197
64448
61c44793238d Adopt more descriptive attribute names as suggested on python-dev.
Raymond Hettinger <python@rcn.com>
parents: 64382
diff changeset
198 def cache_clear():
63823
1086dad50c74 use same quoting as the rest of the file
Benjamin Peterson <benjamin@python.org>
parents: 63745
diff changeset
199 """Clear the cache and cache statistics"""
66550
312b2916cfd4 Issue 10586: change the new functools.lru_cache implementation to expose the maximum and current cache sizes through the public statistics API. This API is now a single function that returns a named tuple.
Nick Coghlan <ncoghlan@gmail.com>
parents: 66441
diff changeset
200 nonlocal hits, misses
64016
d8d0abe61e83 Add locks to make the caches well behaved in multi-threaded code.
Raymond Hettinger <python@rcn.com>
parents: 63823
diff changeset
201 with lock:
d8d0abe61e83 Add locks to make the caches well behaved in multi-threaded code.
Raymond Hettinger <python@rcn.com>
parents: 63823
diff changeset
202 cache.clear()
66550
312b2916cfd4 Issue 10586: change the new functools.lru_cache implementation to expose the maximum and current cache sizes through the public statistics API. This API is now a single function that returns a named tuple.
Nick Coghlan <ncoghlan@gmail.com>
parents: 66441
diff changeset
203 hits = misses = 0
63370
c55876a6fa3c Re-commit r83327 now that the release is done.
Georg Brandl <georg@python.org>
parents: 63357
diff changeset
204
66550
312b2916cfd4 Issue 10586: change the new functools.lru_cache implementation to expose the maximum and current cache sizes through the public statistics API. This API is now a single function that returns a named tuple.
Nick Coghlan <ncoghlan@gmail.com>
parents: 66441
diff changeset
205 wrapper.cache_info = cache_info
64448
61c44793238d Adopt more descriptive attribute names as suggested on python-dev.
Raymond Hettinger <python@rcn.com>
parents: 64382
diff changeset
206 wrapper.cache_clear = cache_clear
63370
c55876a6fa3c Re-commit r83327 now that the release is done.
Georg Brandl <georg@python.org>
parents: 63357
diff changeset
207 return wrapper
66441
03d933948bf3 Clean-up docstring, comments, and whitespace.
Raymond Hettinger <python@rcn.com>
parents: 64776
diff changeset
208
63370
c55876a6fa3c Re-commit r83327 now that the release is done.
Georg Brandl <georg@python.org>
parents: 63357
diff changeset
209 return decorating_function