Mercurial > cpython-lru_cache-weakref

annotate Tools/scripts/analyze_dxp.py @ 79675:c0a2294de787 2.7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #16199: Backport 2to3 typo fix from 3.2.
author Chris Jerdonek <chris.jerdonek@gmail.com>
date Thu, 11 Oct 2012 16:15:46 -0700
parents 7fb0abf928e0
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
71588
7fb0abf928e0 Fix missing or shebangs and executable bits for scripts (#10318)
Éric Araujo <merwok@netwok.org>
parents: 51872
diff changeset
1 #!/usr/bin/env python
51872
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
2 """
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
3 Some helper functions to analyze the output of sys.getdxp() (which is
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
4 only available if Python was built with -DDYNAMIC_EXECUTION_PROFILE).
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
5 These will tell you which opcodes have been executed most frequently
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
6 in the current process, and, if Python was also built with -DDXPAIRS,
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
7 will tell you which instruction _pairs_ were executed most frequently,
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
8 which may help in choosing new instructions.
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
9
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
10 If Python was built without -DDYNAMIC_EXECUTION_PROFILE, importing
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
11 this module will raise a RuntimeError.
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
12
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
13 If you're running a script you want to profile, a simple way to get
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
14 the common pairs is:
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
15
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
16 $ PYTHONPATH=$PYTHONPATH:<python_srcdir>/Tools/scripts \
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
17 ./python -i -O the_script.py --args
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
18 ...
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
19 > from analyze_dxp import *
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
20 > s = render_common_pairs()
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
21 > open('/tmp/some_file', 'w').write(s)
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
22 """
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
23
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
24 import copy
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
25 import opcode
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
26 import operator
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
27 import sys
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
28 import threading
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
29
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
30 if not hasattr(sys, "getdxp"):
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
31 raise RuntimeError("Can't import analyze_dxp: Python built without"
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
32 " -DDYNAMIC_EXECUTION_PROFILE.")
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
33
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
34
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
35 _profile_lock = threading.RLock()
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
36 _cumulative_profile = sys.getdxp()
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
37
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
38 # If Python was built with -DDXPAIRS, sys.getdxp() returns a list of
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
39 # lists of ints. Otherwise it returns just a list of ints.
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
40 def has_pairs(profile):
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
41 """Returns True if the Python that produced the argument profile
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
42 was built with -DDXPAIRS."""
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
43
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
44 return len(profile) > 0 and isinstance(profile[0], list)
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
45
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
46
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
47 def reset_profile():
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
48 """Forgets any execution profile that has been gathered so far."""
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
49 with _profile_lock:
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
50 sys.getdxp() # Resets the internal profile
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
51 global _cumulative_profile
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
52 _cumulative_profile = sys.getdxp() # 0s out our copy.
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
53
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
54
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
55 def merge_profile():
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
56 """Reads sys.getdxp() and merges it into this module's cached copy.
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
57
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
58 We need this because sys.getdxp() 0s itself every time it's called."""
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
59
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
60 with _profile_lock:
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
61 new_profile = sys.getdxp()
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
62 if has_pairs(new_profile):
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
63 for first_inst in range(len(_cumulative_profile)):
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
64 for second_inst in range(len(_cumulative_profile[first_inst])):
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
65 _cumulative_profile[first_inst][second_inst] += (
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
66 new_profile[first_inst][second_inst])
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
67 else:
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
68 for inst in range(len(_cumulative_profile)):
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
69 _cumulative_profile[inst] += new_profile[inst]
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
70
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
71
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
72 def snapshot_profile():
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
73 """Returns the cumulative execution profile until this call."""
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
74 with _profile_lock:
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
75 merge_profile()
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
76 return copy.deepcopy(_cumulative_profile)
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
77
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
78
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
79 def common_instructions(profile):
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
80 """Returns the most common opcodes in order of descending frequency.
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
81
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
82 The result is a list of tuples of the form
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
83 (opcode, opname, # of occurrences)
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
84
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
85 """
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
86 if has_pairs(profile) and profile:
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
87 inst_list = profile[-1]
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
88 else:
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
89 inst_list = profile
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
90 result = [(op, opcode.opname[op], count)
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
91 for op, count in enumerate(inst_list)
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
92 if count > 0]
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
93 result.sort(key=operator.itemgetter(2), reverse=True)
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
94 return result
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
95
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
96
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
97 def common_pairs(profile):
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
98 """Returns the most common opcode pairs in order of descending frequency.
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
99
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
100 The result is a list of tuples of the form
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
101 ((1st opcode, 2nd opcode),
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
102 (1st opname, 2nd opname),
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
103 # of occurrences of the pair)
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
104
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
105 """
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
106 if not has_pairs(profile):
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
107 return []
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
108 result = [((op1, op2), (opcode.opname[op1], opcode.opname[op2]), count)
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
109 # Drop the row of single-op profiles with [:-1]
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
110 for op1, op1profile in enumerate(profile[:-1])
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
111 for op2, count in enumerate(op1profile)
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
112 if count > 0]
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
113 result.sort(key=operator.itemgetter(2), reverse=True)
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
114 return result
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
115
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
116
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
117 def render_common_pairs(profile=None):
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
118 """Renders the most common opcode pairs to a string in order of
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
119 descending frequency.
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
120
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
121 The result is a series of lines of the form:
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
122 # of occurrences: ('1st opname', '2nd opname')
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
123
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
124 """
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
125 if profile is None:
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
126 profile = snapshot_profile()
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
127 def seq():
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
128 for _, ops, count in common_pairs(profile):
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
129 yield "%s: %s\n" % (count, ops)
3722ce458318 Merged revisions 69811,69947 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
130 return ''.join(seq())