Mercurial > cpython-lru_cache-weakref

annotate Lib/lib2to3/pytree.py @ 80661:94de77bd0b4b 2.7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #16194: document imp.load_dynamic problems when read several extension modules from the same binary object file.
author Andrew Svetlov <andrew.svetlov@gmail.com>
date Thu, 29 Nov 2012 19:51:28 +0200
parents cd8d3c2d3bce
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1 # Copyright 2006 Google, Inc. All Rights Reserved.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
2 # Licensed to PSF under a Contributor Agreement.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
3
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
4 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
5 Python parse tree definitions.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
6
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
7 This is a very concrete parse tree; we need to keep every token and
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
8 even the comments and whitespace between tokens.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
9
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
10 There's also a pattern matching implementation here.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
11 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
12
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
13 __author__ = "Guido van Rossum <guido@python.org>"
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
14
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
15 import sys
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
16 import warnings
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
17 from StringIO import StringIO
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
18
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
19 HUGE = 0x7FFFFFFF # maximum repeat count, default max
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
20
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
21 _type_reprs = {}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
22 def type_repr(type_num):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
23 global _type_reprs
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
24 if not _type_reprs:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
25 from .pygram import python_symbols
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
26 # printing tokens is possible but not as useful
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
27 # from .pgen2 import token // token.__dict__.items():
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
28 for name, val in python_symbols.__dict__.items():
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
29 if type(val) == int: _type_reprs[val] = name
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
30 return _type_reprs.setdefault(type_num, type_num)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
31
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
32 class Base(object):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
33
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
34 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
35 Abstract base class for Node and Leaf.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
36
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
37 This provides some default functionality and boilerplate using the
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
38 template pattern.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
39
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
40 A node may be a subnode of at most one parent.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
41 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
42
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
43 # Default values for instance variables
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
44 type = None # int: token number (< 256) or symbol number (>= 256)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
45 parent = None # Parent node pointer, or None
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
46 children = () # Tuple of subnodes
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
47 was_changed = False
65340
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
48 was_checked = False
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
49
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
50 def __new__(cls, *args, **kwds):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
51 """Constructor that prevents Base from being instantiated."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
52 assert cls is not Base, "Cannot instantiate Base"
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
53 return object.__new__(cls)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
54
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
55 def __eq__(self, other):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
56 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
57 Compare two nodes for equality.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
58
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
59 This calls the method _eq().
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
60 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
61 if self.__class__ is not other.__class__:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
62 return NotImplemented
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
63 return self._eq(other)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
64
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
65 __hash__ = None # For Py3 compatibility.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
66
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
67 def __ne__(self, other):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
68 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
69 Compare two nodes for inequality.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
70
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
71 This calls the method _eq().
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
72 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
73 if self.__class__ is not other.__class__:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
74 return NotImplemented
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
75 return not self._eq(other)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
76
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
77 def _eq(self, other):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
78 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
79 Compare two nodes for equality.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
80
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
81 This is called by __eq__ and __ne__. It is only called if the two nodes
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
82 have the same type. This must be implemented by the concrete subclass.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
83 Nodes should be considered equal if they have the same structure,
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
84 ignoring the prefix string and other context information.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
85 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
86 raise NotImplementedError
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
87
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
88 def clone(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
89 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
90 Return a cloned (deep) copy of self.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
91
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
92 This must be implemented by the concrete subclass.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
93 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
94 raise NotImplementedError
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
95
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
96 def post_order(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
97 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
98 Return a post-order iterator for the tree.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
99
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
100 This must be implemented by the concrete subclass.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
101 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
102 raise NotImplementedError
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
103
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
104 def pre_order(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
105 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
106 Return a pre-order iterator for the tree.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
107
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
108 This must be implemented by the concrete subclass.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
109 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
110 raise NotImplementedError
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
111
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
112 def set_prefix(self, prefix):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
113 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
114 Set the prefix for the node (see Leaf class).
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
115
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
116 DEPRECATED; use the prefix property directly.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
117 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
118 warnings.warn("set_prefix() is deprecated; use the prefix property",
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
119 DeprecationWarning, stacklevel=2)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
120 self.prefix = prefix
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
121
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
122 def get_prefix(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
123 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
124 Return the prefix for the node (see Leaf class).
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
125
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
126 DEPRECATED; use the prefix property directly.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
127 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
128 warnings.warn("get_prefix() is deprecated; use the prefix property",
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
129 DeprecationWarning, stacklevel=2)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
130 return self.prefix
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
131
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
132 def replace(self, new):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
133 """Replace this node with a new one in the parent."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
134 assert self.parent is not None, str(self)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
135 assert new is not None
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
136 if not isinstance(new, list):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
137 new = [new]
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
138 l_children = []
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
139 found = False
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
140 for ch in self.parent.children:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
141 if ch is self:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
142 assert not found, (self.parent.children, self, new)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
143 if new is not None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
144 l_children.extend(new)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
145 found = True
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
146 else:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
147 l_children.append(ch)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
148 assert found, (self.children, self, new)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
149 self.parent.changed()
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
150 self.parent.children = l_children
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
151 for x in new:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
152 x.parent = self.parent
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
153 self.parent = None
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
154
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
155 def get_lineno(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
156 """Return the line number which generated the invocant node."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
157 node = self
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
158 while not isinstance(node, Leaf):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
159 if not node.children:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
160 return
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
161 node = node.children[0]
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
162 return node.lineno
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
163
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
164 def changed(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
165 if self.parent:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
166 self.parent.changed()
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
167 self.was_changed = True
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
168
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
169 def remove(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
170 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
171 Remove the node from the tree. Returns the position of the node in its
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
172 parent's children before it was removed.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
173 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
174 if self.parent:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
175 for i, node in enumerate(self.parent.children):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
176 if node is self:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
177 self.parent.changed()
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
178 del self.parent.children[i]
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
179 self.parent = None
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
180 return i
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
181
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
182 @property
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
183 def next_sibling(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
184 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
185 The node immediately following the invocant in their parent's children
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
186 list. If the invocant does not have a next sibling, it is None
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
187 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
188 if self.parent is None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
189 return None
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
190
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
191 # Can't use index(); we need to test by identity
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
192 for i, child in enumerate(self.parent.children):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
193 if child is self:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
194 try:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
195 return self.parent.children[i+1]
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
196 except IndexError:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
197 return None
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
198
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
199 @property
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
200 def prev_sibling(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
201 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
202 The node immediately preceding the invocant in their parent's children
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
203 list. If the invocant does not have a previous sibling, it is None.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
204 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
205 if self.parent is None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
206 return None
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
207
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
208 # Can't use index(); we need to test by identity
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
209 for i, child in enumerate(self.parent.children):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
210 if child is self:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
211 if i == 0:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
212 return None
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
213 return self.parent.children[i-1]
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
214
65340
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
215 def leaves(self):
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
216 for child in self.children:
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
217 for x in child.leaves():
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
218 yield x
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
219
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
220 def depth(self):
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
221 if self.parent is None:
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
222 return 0
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
223 return 1 + self.parent.depth()
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
224
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
225 def get_suffix(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
226 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
227 Return the string immediately following the invocant node. This is
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
228 effectively equivalent to node.next_sibling.prefix
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
229 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
230 next_sib = self.next_sibling
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
231 if next_sib is None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
232 return u""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
233 return next_sib.prefix
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
234
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
235 if sys.version_info < (3, 0):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
236 def __str__(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
237 return unicode(self).encode("ascii")
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
238
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
239 class Node(Base):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
240
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
241 """Concrete implementation for interior nodes."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
242
65340
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
243 def __init__(self,type, children,
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
244 context=None,
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
245 prefix=None,
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
246 fixers_applied=None):
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
247 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
248 Initializer.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
249
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
250 Takes a type constant (a symbol number >= 256), a sequence of
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
251 child nodes, and an optional context keyword argument.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
252
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
253 As a side effect, the parent pointers of the children are updated.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
254 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
255 assert type >= 256, type
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
256 self.type = type
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
257 self.children = list(children)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
258 for ch in self.children:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
259 assert ch.parent is None, repr(ch)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
260 ch.parent = self
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
261 if prefix is not None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
262 self.prefix = prefix
65340
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
263 if fixers_applied:
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
264 self.fixers_applied = fixers_applied[:]
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
265 else:
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
266 self.fixers_applied = None
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
267
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
268 def __repr__(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
269 """Return a canonical string representation."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
270 return "%s(%s, %r)" % (self.__class__.__name__,
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
271 type_repr(self.type),
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
272 self.children)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
273
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
274 def __unicode__(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
275 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
276 Return a pretty string representation.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
277
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
278 This reproduces the input source exactly.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
279 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
280 return u"".join(map(unicode, self.children))
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
281
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
282 if sys.version_info > (3, 0):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
283 __str__ = __unicode__
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
284
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
285 def _eq(self, other):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
286 """Compare two nodes for equality."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
287 return (self.type, self.children) == (other.type, other.children)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
288
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
289 def clone(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
290 """Return a cloned (deep) copy of self."""
65340
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
291 return Node(self.type, [ch.clone() for ch in self.children],
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
292 fixers_applied=self.fixers_applied)
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
293
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
294 def post_order(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
295 """Return a post-order iterator for the tree."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
296 for child in self.children:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
297 for node in child.post_order():
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
298 yield node
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
299 yield self
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
300
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
301 def pre_order(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
302 """Return a pre-order iterator for the tree."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
303 yield self
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
304 for child in self.children:
63841
feffc1a2599c Merged revisions 82530-82531,82779,82855,83740,83789-83791,83797-83801,83803,83811,83827,83844 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 61375
diff changeset
305 for node in child.pre_order():
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
306 yield node
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
307
61376
e92fb95f2317 Merged revisions 80934 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 58737
diff changeset
308 def _prefix_getter(self):
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
309 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
310 The whitespace and comments preceding this node in the input.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
311 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
312 if not self.children:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
313 return ""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
314 return self.children[0].prefix
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
315
61376
e92fb95f2317 Merged revisions 80934 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 58737
diff changeset
316 def _prefix_setter(self, prefix):
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
317 if self.children:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
318 self.children[0].prefix = prefix
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
319
61376
e92fb95f2317 Merged revisions 80934 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 58737
diff changeset
320 prefix = property(_prefix_getter, _prefix_setter)
e92fb95f2317 Merged revisions 80934 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 58737
diff changeset
321
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
322 def set_child(self, i, child):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
323 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
324 Equivalent to 'node.children[i] = child'. This method also sets the
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
325 child's parent attribute appropriately.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
326 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
327 child.parent = self
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
328 self.children[i].parent = None
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
329 self.children[i] = child
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
330 self.changed()
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
331
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
332 def insert_child(self, i, child):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
333 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
334 Equivalent to 'node.children.insert(i, child)'. This method also sets
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
335 the child's parent attribute appropriately.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
336 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
337 child.parent = self
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
338 self.children.insert(i, child)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
339 self.changed()
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
340
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
341 def append_child(self, child):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
342 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
343 Equivalent to 'node.children.append(child)'. This method also sets the
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
344 child's parent attribute appropriately.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
345 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
346 child.parent = self
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
347 self.children.append(child)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
348 self.changed()
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
349
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
350
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
351 class Leaf(Base):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
352
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
353 """Concrete implementation for leaf nodes."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
354
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
355 # Default values for instance variables
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
356 _prefix = "" # Whitespace and comments preceding this token in the input
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
357 lineno = 0 # Line where this token starts in the input
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
358 column = 0 # Column where this token tarts in the input
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
359
65340
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
360 def __init__(self, type, value,
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
361 context=None,
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
362 prefix=None,
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
363 fixers_applied=[]):
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
364 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
365 Initializer.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
366
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
367 Takes a type constant (a token number < 256), a string value, and an
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
368 optional context keyword argument.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
369 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
370 assert 0 <= type < 256, type
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
371 if context is not None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
372 self._prefix, (self.lineno, self.column) = context
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
373 self.type = type
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
374 self.value = value
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
375 if prefix is not None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
376 self._prefix = prefix
65340
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
377 self.fixers_applied = fixers_applied[:]
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
378
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
379 def __repr__(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
380 """Return a canonical string representation."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
381 return "%s(%r, %r)" % (self.__class__.__name__,
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
382 self.type,
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
383 self.value)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
384
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
385 def __unicode__(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
386 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
387 Return a pretty string representation.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
388
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
389 This reproduces the input source exactly.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
390 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
391 return self.prefix + unicode(self.value)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
392
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
393 if sys.version_info > (3, 0):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
394 __str__ = __unicode__
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
395
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
396 def _eq(self, other):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
397 """Compare two nodes for equality."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
398 return (self.type, self.value) == (other.type, other.value)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
399
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
400 def clone(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
401 """Return a cloned (deep) copy of self."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
402 return Leaf(self.type, self.value,
65340
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
403 (self.prefix, (self.lineno, self.column)),
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
404 fixers_applied=self.fixers_applied)
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
405
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
406 def leaves(self):
7d1e453b120b Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 63841
diff changeset
407 yield self
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
408
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
409 def post_order(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
410 """Return a post-order iterator for the tree."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
411 yield self
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
412
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
413 def pre_order(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
414 """Return a pre-order iterator for the tree."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
415 yield self
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
416
61376
e92fb95f2317 Merged revisions 80934 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 58737
diff changeset
417 def _prefix_getter(self):
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
418 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
419 The whitespace and comments preceding this token in the input.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
420 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
421 return self._prefix
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
422
61376
e92fb95f2317 Merged revisions 80934 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 58737
diff changeset
423 def _prefix_setter(self, prefix):
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
424 self.changed()
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
425 self._prefix = prefix
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
426
61376
e92fb95f2317 Merged revisions 80934 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 58737
diff changeset
427 prefix = property(_prefix_getter, _prefix_setter)
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
428
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
429 def convert(gr, raw_node):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
430 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
431 Convert raw node information to a Node or Leaf instance.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
432
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
433 This is passed to the parser driver which calls it whenever a reduction of a
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
434 grammar rule produces a new complete node, so that the tree is build
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
435 strictly bottom-up.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
436 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
437 type, value, context, children = raw_node
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
438 if children or type in gr.number2symbol:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
439 # If there's exactly one child, return that child instead of
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
440 # creating a new node.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
441 if len(children) == 1:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
442 return children[0]
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
443 return Node(type, children, context=context)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
444 else:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
445 return Leaf(type, value, context=context)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
446
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
447
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
448 class BasePattern(object):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
449
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
450 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
451 A pattern is a tree matching pattern.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
452
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
453 It looks for a specific node type (token or symbol), and
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
454 optionally for a specific content.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
455
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
456 This is an abstract base class. There are three concrete
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
457 subclasses:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
458
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
459 - LeafPattern matches a single leaf node;
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
460 - NodePattern matches a single node (usually non-leaf);
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
461 - WildcardPattern matches a sequence of nodes of variable length.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
462 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
463
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
464 # Defaults for instance variables
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
465 type = None # Node type (token if < 256, symbol if >= 256)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
466 content = None # Optional content matching pattern
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
467 name = None # Optional name used to store match in results dict
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
468
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
469 def __new__(cls, *args, **kwds):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
470 """Constructor that prevents BasePattern from being instantiated."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
471 assert cls is not BasePattern, "Cannot instantiate BasePattern"
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
472 return object.__new__(cls)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
473
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
474 def __repr__(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
475 args = [type_repr(self.type), self.content, self.name]
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
476 while args and args[-1] is None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
477 del args[-1]
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
478 return "%s(%s)" % (self.__class__.__name__, ", ".join(map(repr, args)))
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
479
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
480 def optimize(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
481 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
482 A subclass can define this as a hook for optimizations.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
483
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
484 Returns either self or another node with the same effect.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
485 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
486 return self
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
487
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
488 def match(self, node, results=None):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
489 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
490 Does this pattern exactly match a node?
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
491
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
492 Returns True if it matches, False if not.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
493
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
494 If results is not None, it must be a dict which will be
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
495 updated with the nodes matching named subpatterns.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
496
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
497 Default implementation for non-wildcard patterns.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
498 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
499 if self.type is not None and node.type != self.type:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
500 return False
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
501 if self.content is not None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
502 r = None
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
503 if results is not None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
504 r = {}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
505 if not self._submatch(node, r):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
506 return False
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
507 if r:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
508 results.update(r)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
509 if results is not None and self.name:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
510 results[self.name] = node
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
511 return True
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
512
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
513 def match_seq(self, nodes, results=None):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
514 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
515 Does this pattern exactly match a sequence of nodes?
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
516
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
517 Default implementation for non-wildcard patterns.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
518 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
519 if len(nodes) != 1:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
520 return False
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
521 return self.match(nodes[0], results)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
522
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
523 def generate_matches(self, nodes):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
524 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
525 Generator yielding all matches for this pattern.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
526
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
527 Default implementation for non-wildcard patterns.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
528 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
529 r = {}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
530 if nodes and self.match(nodes[0], r):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
531 yield 1, r
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
532
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
533
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
534 class LeafPattern(BasePattern):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
535
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
536 def __init__(self, type=None, content=None, name=None):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
537 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
538 Initializer. Takes optional type, content, and name.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
539
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
540 The type, if given must be a token type (< 256). If not given,
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
541 this matches any *leaf* node; the content may still be required.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
542
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
543 The content, if given, must be a string.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
544
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
545 If a name is given, the matching node is stored in the results
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
546 dict under that key.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
547 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
548 if type is not None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
549 assert 0 <= type < 256, type
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
550 if content is not None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
551 assert isinstance(content, basestring), repr(content)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
552 self.type = type
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
553 self.content = content
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
554 self.name = name
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
555
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
556 def match(self, node, results=None):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
557 """Override match() to insist on a leaf node."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
558 if not isinstance(node, Leaf):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
559 return False
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
560 return BasePattern.match(self, node, results)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
561
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
562 def _submatch(self, node, results=None):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
563 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
564 Match the pattern's content to the node's children.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
565
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
566 This assumes the node type matches and self.content is not None.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
567
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
568 Returns True if it matches, False if not.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
569
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
570 If results is not None, it must be a dict which will be
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
571 updated with the nodes matching named subpatterns.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
572
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
573 When returning False, the results dict may still be updated.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
574 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
575 return self.content == node.value
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
576
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
577
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
578 class NodePattern(BasePattern):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
579
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
580 wildcards = False
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
581
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
582 def __init__(self, type=None, content=None, name=None):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
583 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
584 Initializer. Takes optional type, content, and name.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
585
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
586 The type, if given, must be a symbol type (>= 256). If the
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
587 type is None this matches *any* single node (leaf or not),
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
588 except if content is not None, in which it only matches
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
589 non-leaf nodes that also match the content pattern.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
590
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
591 The content, if not None, must be a sequence of Patterns that
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
592 must match the node's children exactly. If the content is
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
593 given, the type must not be None.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
594
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
595 If a name is given, the matching node is stored in the results
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
596 dict under that key.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
597 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
598 if type is not None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
599 assert type >= 256, type
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
600 if content is not None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
601 assert not isinstance(content, basestring), repr(content)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
602 content = list(content)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
603 for i, item in enumerate(content):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
604 assert isinstance(item, BasePattern), (i, item)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
605 if isinstance(item, WildcardPattern):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
606 self.wildcards = True
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
607 self.type = type
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
608 self.content = content
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
609 self.name = name
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
610
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
611 def _submatch(self, node, results=None):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
612 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
613 Match the pattern's content to the node's children.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
614
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
615 This assumes the node type matches and self.content is not None.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
616
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
617 Returns True if it matches, False if not.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
618
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
619 If results is not None, it must be a dict which will be
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
620 updated with the nodes matching named subpatterns.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
621
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
622 When returning False, the results dict may still be updated.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
623 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
624 if self.wildcards:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
625 for c, r in generate_matches(self.content, node.children):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
626 if c == len(node.children):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
627 if results is not None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
628 results.update(r)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
629 return True
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
630 return False
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
631 if len(self.content) != len(node.children):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
632 return False
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
633 for subpattern, child in zip(self.content, node.children):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
634 if not subpattern.match(child, results):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
635 return False
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
636 return True
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
637
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
638
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
639 class WildcardPattern(BasePattern):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
640
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
641 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
642 A wildcard pattern can match zero or more nodes.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
643
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
644 This has all the flexibility needed to implement patterns like:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
645
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
646 .* .+ .? .{m,n}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
647 (a b c | d e | f)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
648 (...)* (...)+ (...)? (...){m,n}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
649
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
650 except it always uses non-greedy matching.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
651 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
652
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
653 def __init__(self, content=None, min=0, max=HUGE, name=None):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
654 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
655 Initializer.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
656
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
657 Args:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
658 content: optional sequence of subsequences of patterns;
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
659 if absent, matches one node;
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
660 if present, each subsequence is an alternative [*]
68588
cd8d3c2d3bce #11565: Fix several typos. Patch by Piotr Kasprzyk.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 68309
diff changeset
661 min: optional minimum number of times to match, default 0
cd8d3c2d3bce #11565: Fix several typos. Patch by Piotr Kasprzyk.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 68309
diff changeset
662 max: optional maximum number of times to match, default HUGE
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
663 name: optional name assigned to this match
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
664
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
665 [*] Thus, if content is [[a, b, c], [d, e], [f, g, h]] this is
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
666 equivalent to (a b c | d e | f g h); if content is None,
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
667 this is equivalent to '.' in regular expression terms.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
668 The min and max parameters work as follows:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
669 min=0, max=maxint: .*
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
670 min=1, max=maxint: .+
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
671 min=0, max=1: .?
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
672 min=1, max=1: .
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
673 If content is not None, replace the dot with the parenthesized
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
674 list of alternatives, e.g. (a b c | d e | f g h)*
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
675 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
676 assert 0 <= min <= max <= HUGE, (min, max)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
677 if content is not None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
678 content = tuple(map(tuple, content)) # Protect against alterations
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
679 # Check sanity of alternatives
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
680 assert len(content), repr(content) # Can't have zero alternatives
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
681 for alt in content:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
682 assert len(alt), repr(alt) # Can have empty alternatives
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
683 self.content = content
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
684 self.min = min
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
685 self.max = max
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
686 self.name = name
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
687
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
688 def optimize(self):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
689 """Optimize certain stacked wildcard patterns."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
690 subpattern = None
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
691 if (self.content is not None and
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
692 len(self.content) == 1 and len(self.content[0]) == 1):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
693 subpattern = self.content[0][0]
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
694 if self.min == 1 and self.max == 1:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
695 if self.content is None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
696 return NodePattern(name=self.name)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
697 if subpattern is not None and self.name == subpattern.name:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
698 return subpattern.optimize()
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
699 if (self.min <= 1 and isinstance(subpattern, WildcardPattern) and
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
700 subpattern.min <= 1 and self.name == subpattern.name):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
701 return WildcardPattern(subpattern.content,
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
702 self.min*subpattern.min,
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
703 self.max*subpattern.max,
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
704 subpattern.name)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
705 return self
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
706
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
707 def match(self, node, results=None):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
708 """Does this pattern exactly match a node?"""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
709 return self.match_seq([node], results)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
710
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
711 def match_seq(self, nodes, results=None):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
712 """Does this pattern exactly match a sequence of nodes?"""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
713 for c, r in self.generate_matches(nodes):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
714 if c == len(nodes):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
715 if results is not None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
716 results.update(r)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
717 if self.name:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
718 results[self.name] = list(nodes)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
719 return True
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
720 return False
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
721
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
722 def generate_matches(self, nodes):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
723 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
724 Generator yielding matches for a sequence of nodes.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
725
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
726 Args:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
727 nodes: sequence of nodes
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
728
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
729 Yields:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
730 (count, results) tuples where:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
731 count: the match comprises nodes[:count];
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
732 results: dict containing named submatches.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
733 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
734 if self.content is None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
735 # Shortcut for special case (see __init__.__doc__)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
736 for count in xrange(self.min, 1 + min(len(nodes), self.max)):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
737 r = {}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
738 if self.name:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
739 r[self.name] = nodes[:count]
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
740 yield count, r
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
741 elif self.name == "bare_name":
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
742 yield self._bare_name_matches(nodes)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
743 else:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
744 # The reason for this is that hitting the recursion limit usually
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
745 # results in some ugly messages about how RuntimeErrors are being
68309
8bea7b156485 only do this sys.stderr replacing on CPython
Benjamin Peterson <benjamin@python.org>
parents: 68256
diff changeset
746 # ignored. We don't do this on non-CPython implementation because
8bea7b156485 only do this sys.stderr replacing on CPython
Benjamin Peterson <benjamin@python.org>
parents: 68256
diff changeset
747 # they don't have this problem.
8bea7b156485 only do this sys.stderr replacing on CPython
Benjamin Peterson <benjamin@python.org>
parents: 68256
diff changeset
748 if hasattr(sys, "getrefcount"):
8bea7b156485 only do this sys.stderr replacing on CPython
Benjamin Peterson <benjamin@python.org>
parents: 68256
diff changeset
749 save_stderr = sys.stderr
8bea7b156485 only do this sys.stderr replacing on CPython
Benjamin Peterson <benjamin@python.org>
parents: 68256
diff changeset
750 sys.stderr = StringIO()
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
751 try:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
752 for count, r in self._recursive_matches(nodes, 0):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
753 if self.name:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
754 r[self.name] = nodes[:count]
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
755 yield count, r
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
756 except RuntimeError:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
757 # We fall back to the iterative pattern matching scheme if the recursive
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
758 # scheme hits the recursion limit.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
759 for count, r in self._iterative_matches(nodes):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
760 if self.name:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
761 r[self.name] = nodes[:count]
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
762 yield count, r
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
763 finally:
68309
8bea7b156485 only do this sys.stderr replacing on CPython
Benjamin Peterson <benjamin@python.org>
parents: 68256
diff changeset
764 if hasattr(sys, "getrefcount"):
8bea7b156485 only do this sys.stderr replacing on CPython
Benjamin Peterson <benjamin@python.org>
parents: 68256
diff changeset
765 sys.stderr = save_stderr
58737
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
766
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
767 def _iterative_matches(self, nodes):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
768 """Helper to iteratively yield the matches."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
769 nodelen = len(nodes)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
770 if 0 >= self.min:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
771 yield 0, {}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
772
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
773 results = []
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
774 # generate matches that use just one alt from self.content
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
775 for alt in self.content:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
776 for c, r in generate_matches(alt, nodes):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
777 yield c, r
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
778 results.append((c, r))
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
779
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
780 # for each match, iterate down the nodes
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
781 while results:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
782 new_results = []
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
783 for c0, r0 in results:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
784 # stop if the entire set of nodes has been matched
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
785 if c0 < nodelen and c0 <= self.max:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
786 for alt in self.content:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
787 for c1, r1 in generate_matches(alt, nodes[c0:]):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
788 if c1 > 0:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
789 r = {}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
790 r.update(r0)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
791 r.update(r1)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
792 yield c0 + c1, r
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
793 new_results.append((c0 + c1, r))
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
794 results = new_results
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
795
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
796 def _bare_name_matches(self, nodes):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
797 """Special optimized matcher for bare_name."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
798 count = 0
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
799 r = {}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
800 done = False
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
801 max = len(nodes)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
802 while not done and count < max:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
803 done = True
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
804 for leaf in self.content:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
805 if leaf[0].match(nodes[count], r):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
806 count += 1
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
807 done = False
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
808 break
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
809 r[self.name] = nodes[:count]
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
810 return count, r
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
811
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
812 def _recursive_matches(self, nodes, count):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
813 """Helper to recursively yield the matches."""
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
814 assert self.content is not None
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
815 if count >= self.min:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
816 yield 0, {}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
817 if count < self.max:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
818 for alt in self.content:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
819 for c0, r0 in generate_matches(alt, nodes):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
820 for c1, r1 in self._recursive_matches(nodes[c0:], count+1):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
821 r = {}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
822 r.update(r0)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
823 r.update(r1)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
824 yield c0 + c1, r
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
825
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
826
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
827 class NegatedPattern(BasePattern):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
828
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
829 def __init__(self, content=None):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
830 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
831 Initializer.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
832
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
833 The argument is either a pattern or None. If it is None, this
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
834 only matches an empty sequence (effectively '$' in regex
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
835 lingo). If it is not None, this matches whenever the argument
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
836 pattern doesn't have any matches.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
837 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
838 if content is not None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
839 assert isinstance(content, BasePattern), repr(content)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
840 self.content = content
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
841
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
842 def match(self, node):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
843 # We never match a node in its entirety
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
844 return False
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
845
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
846 def match_seq(self, nodes):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
847 # We only match an empty sequence of nodes in its entirety
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
848 return len(nodes) == 0
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
849
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
850 def generate_matches(self, nodes):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
851 if self.content is None:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
852 # Return a match if there is an empty sequence
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
853 if len(nodes) == 0:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
854 yield 0, {}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
855 else:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
856 # Return a match if the argument pattern has no matches
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
857 for c, r in self.content.generate_matches(nodes):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
858 return
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
859 yield 0, {}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
860
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
861
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
862 def generate_matches(patterns, nodes):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
863 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
864 Generator yielding matches for a sequence of patterns and nodes.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
865
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
866 Args:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
867 patterns: a sequence of patterns
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
868 nodes: a sequence of nodes
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
869
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
870 Yields:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
871 (count, results) tuples where:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
872 count: the entire sequence of patterns matches nodes[:count];
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
873 results: dict containing named submatches.
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
874 """
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
875 if not patterns:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
876 yield 0, {}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
877 else:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
878 p, rest = patterns[0], patterns[1:]
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
879 for c0, r0 in p.generate_matches(nodes):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
880 if not rest:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
881 yield c0, r0
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
882 else:
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
883 for c1, r1 in generate_matches(rest, nodes[c0:]):
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
884 r = {}
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
885 r.update(r0)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
886 r.update(r1)
fab380674959 copy lib2to3 from the trunk
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
887 yield c0 + c1, r