Mercurial > cpython-lru_cache-weakref

annotate Tools/scripts/find_recursionlimit.py @ 74006:88198b93ff2f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #13560: Add PyUnicode_EncodeLocale() * Use PyUnicode_EncodeLocale() in time.strftime() if wcsftime() is not available * Document my last changes in Misc/NEWS
author Victor Stinner <victor.stinner@haypocalc.com>
date Sat, 17 Dec 2011 04:13:41 +0100
parents 3c9232a684ae
children 62dde5dd475e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
59522
fcfe324663a6 convert shebang lines: python -> python3
Benjamin Peterson <benjamin@python.org>
parents: 56889
diff changeset
1 #! /usr/bin/env python3
56889
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
2 """Find the maximum recursion limit that prevents interpreter termination.
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
3
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
4 This script finds the maximum safe recursion limit on a particular
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
5 platform. If you need to change the recursion limit on your system,
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
6 this script will tell you a safe upper bound. To use the new limit,
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
7 call sys.setrecursionlimit().
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
8
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
9 This module implements several ways to create infinite recursion in
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
10 Python. Different implementations end up pushing different numbers of
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
11 C stack frames, depending on how many calls through Python's abstract
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
12 C API occur.
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
13
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
14 After each round of tests, it prints a message:
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
15 "Limit of NNNN is fine".
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
16
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
17 The highest printed value of "NNNN" is therefore the highest potentially
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
18 safe limit for your system (which depends on the OS, architecture, but also
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
19 the compilation flags). Please note that it is practically impossible to
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
20 test all possible recursion paths in the interpreter, so the results of
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
21 this test should not be trusted blindly -- although they give a good hint
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
22 of which values are reasonable.
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
23
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
24 NOTE: When the C stack space allocated by your system is exceeded due
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
25 to excessive recursion, exact behaviour depends on the platform, although
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
26 the interpreter will always fail in a likely brutal way: either a
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
27 segmentation fault, a MemoryError, or just a silent abort.
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
28
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
29 NB: A program that does not use __methods__ can set a higher limit.
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
30 """
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
31
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
32 import sys
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
33 import itertools
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
34
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
35 class RecursiveBlowup1:
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
36 def __init__(self):
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
37 self.__init__()
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
38
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
39 def test_init():
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
40 return RecursiveBlowup1()
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
41
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
42 class RecursiveBlowup2:
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
43 def __repr__(self):
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
44 return repr(self)
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
45
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
46 def test_repr():
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
47 return repr(RecursiveBlowup2())
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
48
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
49 class RecursiveBlowup4:
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
50 def __add__(self, x):
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
51 return x + self
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
52
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
53 def test_add():
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
54 return RecursiveBlowup4() + RecursiveBlowup4()
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
55
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
56 class RecursiveBlowup5:
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
57 def __getattr__(self, attr):
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
58 return getattr(self, attr)
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
59
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
60 def test_getattr():
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
61 return RecursiveBlowup5().attr
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
62
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
63 class RecursiveBlowup6:
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
64 def __getitem__(self, item):
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
65 return self[item - 2] + self[item - 1]
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
66
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
67 def test_getitem():
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
68 return RecursiveBlowup6()[5]
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
69
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
70 def test_recurse():
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
71 return test_recurse()
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
72
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
73 def test_cpickle(_cache={}):
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
74 import io
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
75 try:
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
76 import _pickle
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
77 except ImportError:
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
78 print("cannot import _pickle, skipped!")
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
79 return
67725
bcf030f73eee Merged revisions 88147 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56889
diff changeset
80 k, l = None, None
56889
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
81 for n in itertools.count():
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
82 try:
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
83 l = _cache[n]
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
84 continue # Already tried and it works, let's save some time
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
85 except KeyError:
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
86 for i in range(100):
67725
bcf030f73eee Merged revisions 88147 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56889
diff changeset
87 l = [k, l]
bcf030f73eee Merged revisions 88147 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56889
diff changeset
88 k = {i: l}
56889
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
89 _pickle.Pickler(io.BytesIO(), protocol=-1).dump(l)
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
90 _cache[n] = l
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
91
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
92 def check_limit(n, test_func_name):
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
93 sys.setrecursionlimit(n)
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
94 if test_func_name.startswith("test_"):
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
95 print(test_func_name[5:])
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
96 else:
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
97 print(test_func_name)
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
98 test_func = globals()[test_func_name]
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
99 try:
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
100 test_func()
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
101 # AttributeError can be raised because of the way e.g. PyDict_GetItem()
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
102 # silences all exceptions and returns NULL, which is usually interpreted
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
103 # as "missing attribute".
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
104 except (RuntimeError, AttributeError):
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
105 pass
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
106 else:
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
107 print("Yikes!")
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
108
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
109 limit = 1000
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
110 while 1:
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
111 check_limit(limit, "test_recurse")
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
112 check_limit(limit, "test_add")
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
113 check_limit(limit, "test_repr")
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
114 check_limit(limit, "test_init")
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
115 check_limit(limit, "test_getattr")
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
116 check_limit(limit, "test_getitem")
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
117 check_limit(limit, "test_cpickle")
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
118 print("Limit of %d is fine" % limit)
029d69659b5f Merged revisions 75373 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
119 limit = limit + 100