Mercurial > cpython-lru_cache-weakref

annotate Lib/compiler/visitor.py @ 79675:c0a2294de787 2.7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #16199: Backport 2to3 typo fix from 3.2.
author Chris Jerdonek <chris.jerdonek@gmail.com>
date Thu, 11 Oct 2012 16:15:46 -0700
parents bdee45888b64
children 4cdf21a0dbbc
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11925
2a628067969b fix import to refer to compiler package
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11922
diff changeset
1 from compiler import ast
11922
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
2
19509
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
3 # XXX should probably rename ASTVisitor to ASTWalker
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
4 # XXX can it be made even more generic?
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
5
11922
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
6 class ASTVisitor:
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
7 """Performs a depth-first walk of the AST
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
8
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
9 The ASTVisitor will walk the AST, performing either a preorder or
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
10 postorder traversal depending on which method is called.
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
11
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
12 methods:
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
13 preorder(tree, visitor)
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
14 postorder(tree, visitor)
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
15 tree: an instance of ast.Node
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
16 visitor: an instance with visitXXX methods
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
17
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
18 The ASTVisitor is responsible for walking over the tree in the
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
19 correct order. For each node, it checks the visitor argument for
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
20 a method named 'visitNodeType' where NodeType is the name of the
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
21 node's class, e.g. Class. If the method exists, it is called
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
22 with the node as its sole argument.
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
23
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
24 The visitor method for a particular node type can control how
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
25 child nodes are visited during a preorder walk. (It can't control
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
26 the order during a postorder walk, because it is called _after_
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
27 the walk has occurred.) The ASTVisitor modifies the visitor
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
28 argument by adding a visit method to the visitor; this method can
23155
61eb7182535e Remove more out-of-date comments and clarify explanation of visit().
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23154
diff changeset
29 be used to visit a child node of arbitrary type.
11922
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
30 """
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
31
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
32 VERBOSE = 0
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
33
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
34 def __init__(self):
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
35 self.node = None
13941
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
36 self._cache = {}
11922
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
37
12008
74fb602b76c2 simplify visitor walker class
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11925
diff changeset
38 def default(self, node, *args):
19562
49bb41a69dfc Change default() to use getChildNodes() instead of getChildren()
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 19509
diff changeset
39 for child in node.getChildNodes():
49bb41a69dfc Change default() to use getChildNodes() instead of getChildren()
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 19509
diff changeset
40 self.dispatch(child, *args)
11922
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
41
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
42 def dispatch(self, node, *args):
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
43 self.node = node
15327
c1846d331de9 Small optimizations in dispatch method: 1) lookup node's __class__ once
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13941
diff changeset
44 klass = node.__class__
c1846d331de9 Small optimizations in dispatch method: 1) lookup node's __class__ once
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13941
diff changeset
45 meth = self._cache.get(klass, None)
13941
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
46 if meth is None:
15327
c1846d331de9 Small optimizations in dispatch method: 1) lookup node's __class__ once
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13941
diff changeset
47 className = klass.__name__
13941
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
48 meth = getattr(self.visitor, 'visit' + className, self.default)
15327
c1846d331de9 Small optimizations in dispatch method: 1) lookup node's __class__ once
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13941
diff changeset
49 self._cache[klass] = meth
19509
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
50 ## if self.VERBOSE > 0:
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
51 ## className = klass.__name__
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
52 ## if self.VERBOSE == 1:
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
53 ## if meth == 0:
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
54 ## print "dispatch", className
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
55 ## else:
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
56 ## print "dispatch", className, (meth and meth.__name__ or '')
17411
bd844b20b9c8 Add support for extra (*) arguments to preorder.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 15327
diff changeset
57 return meth(node, *args)
11922
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
58
17411
bd844b20b9c8 Add support for extra (*) arguments to preorder.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 15327
diff changeset
59 def preorder(self, tree, visitor, *args):
15327
c1846d331de9 Small optimizations in dispatch method: 1) lookup node's __class__ once
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13941
diff changeset
60 """Do preorder walk of tree using visitor"""
c1846d331de9 Small optimizations in dispatch method: 1) lookup node's __class__ once
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13941
diff changeset
61 self.visitor = visitor
19509
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
62 visitor.visit = self.dispatch
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
63 self.dispatch(tree, *args) # XXX *args make sense?
15327
c1846d331de9 Small optimizations in dispatch method: 1) lookup node's __class__ once
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13941
diff changeset
64
11922
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
65 class ExampleASTVisitor(ASTVisitor):
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
66 """Prints examples of the nodes that aren't visited
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
67
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
68 This visitor-driver is only useful for development, when it's
30130
82a1d3f210ad Comment typo fixes
Andrew M. Kuchling <amk@amk.ca>
parents: 23155
diff changeset
69 helpful to develop a visitor incrementally, and get feedback on what
11922
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
70 you still have to do.
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
71 """
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
72 examples = {}
20547
a12ccdbce8da Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 19562
diff changeset
73
11922
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
74 def dispatch(self, node, *args):
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
75 self.node = node
13941
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
76 meth = self._cache.get(node.__class__, None)
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
77 className = node.__class__.__name__
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
78 if meth is None:
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
79 meth = getattr(self.visitor, 'visit' + className, 0)
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
80 self._cache[node.__class__] = meth
12008
74fb602b76c2 simplify visitor walker class
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11925
diff changeset
81 if self.VERBOSE > 1:
74fb602b76c2 simplify visitor walker class
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11925
diff changeset
82 print "dispatch", className, (meth and meth.__name__ or '')
11922
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
83 if meth:
19509
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
84 meth(node, *args)
11922
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
85 elif self.VERBOSE > 0:
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
86 klass = node.__class__
39588
bdee45888b64 Stomp out more hsa_key() calls.
Guido van Rossum <guido@python.org>
parents: 30130
diff changeset
87 if klass not in self.examples:
13941
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
88 self.examples[klass] = klass
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
89 print
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
90 print self.visitor
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
91 print klass
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
92 for attr in dir(node):
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
93 if attr[0] != '_':
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
94 print "\t", "%-12.12s" % attr, getattr(node, attr)
d56dd1530ce1 Bring Tools/compiler almost up to date. Specifically:
Thomas Wouters <thomas@python.org>
parents: 12008
diff changeset
95 print
19509
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
96 return self.default(node, *args)
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
97
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
98 # XXX this is an API change
11922
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
99
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
100 _walker = ASTVisitor
19509
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
101 def walk(tree, visitor, walker=None, verbose=None):
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
102 if walker is None:
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
103 walker = _walker()
11922
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
104 if verbose is not None:
19509
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
105 walker.VERBOSE = verbose
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
106 walker.preorder(tree, visitor)
334f4178477a Two changes to visitor API:
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17411
diff changeset
107 return walker.visitor
11922
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
108
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
109 def dumpNode(node):
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
110 print node.__class__
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
111 for attr in dir(node):
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
112 if attr[0] != '_':
8cc4428c7bad factor out the tree walking/visitor code that was in compile.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
diff changeset
113 print "\t", "%-10.10s" % attr, getattr(node, attr)