Mercurial > cpython-lru_cache-weakref

annotate Lib/ast.py @ 79691:05c8928362ee

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Merge __next__ method link fixes with 3.3.
author Ezio Melotti <ezio.melotti@gmail.com>
date Fri, 12 Oct 2012 13:46:37 +0300
parents b23ad0a6cf87
children 03f92a9f0875
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
47555
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
1 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
2 ast
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
3 ~~~
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
4
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
5 The `ast` module helps Python applications to process trees of the Python
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
6 abstract syntax grammar. The abstract syntax itself might change with
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
7 each Python release; this module helps to find out programmatically what
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
8 the current grammar looks like and allows modifications of it.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
9
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
10 An abstract syntax tree can be generated by passing `ast.PyCF_ONLY_AST` as
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
11 a flag to the `compile()` builtin function or by using the `parse()`
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
12 function from this module. The result will be a tree of objects whose
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
13 classes all inherit from `ast.AST`.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
14
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
15 A modified abstract syntax tree can be compiled into a Python code object
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
16 using the built-in `compile()` function.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
17
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
18 Additionally various helper functions are provided that make working with
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
19 the trees simpler. The main intention of the helper functions and this
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
20 module in general is to provide an easy to use interface for libraries
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
21 that work tightly with the python syntax (template engines for example).
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
22
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
23
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
24 :copyright: Copyright 2008 by Armin Ronacher.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
25 :license: Python License.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
26 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
27 from _ast import *
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
28
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
29
67748
3b93d0099ced Issue #11000 ast.parse doc fix (r88172) + tix fix (r88170)
Terry Reedy <tjreedy@udel.edu>
parents: 67484
diff changeset
30 def parse(source, filename='<unknown>', mode='exec'):
47555
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
31 """
67748
3b93d0099ced Issue #11000 ast.parse doc fix (r88172) + tix fix (r88170)
Terry Reedy <tjreedy@udel.edu>
parents: 67484
diff changeset
32 Parse the source into an AST node.
3b93d0099ced Issue #11000 ast.parse doc fix (r88172) + tix fix (r88170)
Terry Reedy <tjreedy@udel.edu>
parents: 67484
diff changeset
33 Equivalent to compile(source, filename, mode, PyCF_ONLY_AST).
47555
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
34 """
67748
3b93d0099ced Issue #11000 ast.parse doc fix (r88172) + tix fix (r88170)
Terry Reedy <tjreedy@udel.edu>
parents: 67484
diff changeset
35 return compile(source, filename, mode, PyCF_ONLY_AST)
47555
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
36
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
37
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
38 def literal_eval(node_or_string):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
39 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
40 Safely evaluate an expression node or a string containing a Python
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
41 expression. The string or node provided may only consist of the following
69605
281c090ad3b9 Add missing types to docstring of ast.literal_eval.
Éric Araujo <merwok@netwok.org>
parents: 68257
diff changeset
42 Python literal structures: strings, bytes, numbers, tuples, lists, dicts,
281c090ad3b9 Add missing types to docstring of ast.literal_eval.
Éric Araujo <merwok@netwok.org>
parents: 68257
diff changeset
43 sets, booleans, and None.
47555
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
44 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
45 _safe_names = {'None': None, 'True': True, 'False': False}
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
46 if isinstance(node_or_string, str):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
47 node_or_string = parse(node_or_string, mode='eval')
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
48 if isinstance(node_or_string, Expression):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
49 node_or_string = node_or_string.body
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
50 def _convert(node):
62976
4dd9852dae9b allow byte literals
Benjamin Peterson <benjamin@python.org>
parents: 62958
diff changeset
51 if isinstance(node, (Str, Bytes)):
47555
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
52 return node.s
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
53 elif isinstance(node, Num):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
54 return node.n
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
55 elif isinstance(node, Tuple):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
56 return tuple(map(_convert, node.elts))
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
57 elif isinstance(node, List):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
58 return list(map(_convert, node.elts))
62958
90bf0631bfb8 Allow set literals in literal_eval().
Georg Brandl <georg@python.org>
parents: 50812
diff changeset
59 elif isinstance(node, Set):
90bf0631bfb8 Allow set literals in literal_eval().
Georg Brandl <georg@python.org>
parents: 50812
diff changeset
60 return set(map(_convert, node.elts))
47555
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
61 elif isinstance(node, Dict):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
62 return dict((_convert(k), _convert(v)) for k, v
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
63 in zip(node.keys, node.values))
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
64 elif isinstance(node, Name):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
65 if node.id in _safe_names:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
66 return _safe_names[node.id]
65172
884c71cd8dc6 Fix handling on negative numbers in ast.literal_eval().
Raymond Hettinger <python@rcn.com>
parents: 64316
diff changeset
67 elif isinstance(node, UnaryOp) and \
884c71cd8dc6 Fix handling on negative numbers in ast.literal_eval().
Raymond Hettinger <python@rcn.com>
parents: 64316
diff changeset
68 isinstance(node.op, (UAdd, USub)) and \
884c71cd8dc6 Fix handling on negative numbers in ast.literal_eval().
Raymond Hettinger <python@rcn.com>
parents: 64316
diff changeset
69 isinstance(node.operand, (Num, UnaryOp, BinOp)):
884c71cd8dc6 Fix handling on negative numbers in ast.literal_eval().
Raymond Hettinger <python@rcn.com>
parents: 64316
diff changeset
70 operand = _convert(node.operand)
884c71cd8dc6 Fix handling on negative numbers in ast.literal_eval().
Raymond Hettinger <python@rcn.com>
parents: 64316
diff changeset
71 if isinstance(node.op, UAdd):
884c71cd8dc6 Fix handling on negative numbers in ast.literal_eval().
Raymond Hettinger <python@rcn.com>
parents: 64316
diff changeset
72 return + operand
884c71cd8dc6 Fix handling on negative numbers in ast.literal_eval().
Raymond Hettinger <python@rcn.com>
parents: 64316
diff changeset
73 else:
884c71cd8dc6 Fix handling on negative numbers in ast.literal_eval().
Raymond Hettinger <python@rcn.com>
parents: 64316
diff changeset
74 return - operand
50812
a04b1c7167ba Merged revisions 68630 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49587
diff changeset
75 elif isinstance(node, BinOp) and \
a04b1c7167ba Merged revisions 68630 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49587
diff changeset
76 isinstance(node.op, (Add, Sub)) and \
65172
884c71cd8dc6 Fix handling on negative numbers in ast.literal_eval().
Raymond Hettinger <python@rcn.com>
parents: 64316
diff changeset
77 isinstance(node.right, (Num, UnaryOp, BinOp)) and \
884c71cd8dc6 Fix handling on negative numbers in ast.literal_eval().
Raymond Hettinger <python@rcn.com>
parents: 64316
diff changeset
78 isinstance(node.left, (Num, UnaryOp, BinOp)):
884c71cd8dc6 Fix handling on negative numbers in ast.literal_eval().
Raymond Hettinger <python@rcn.com>
parents: 64316
diff changeset
79 left = _convert(node.left)
884c71cd8dc6 Fix handling on negative numbers in ast.literal_eval().
Raymond Hettinger <python@rcn.com>
parents: 64316
diff changeset
80 right = _convert(node.right)
50812
a04b1c7167ba Merged revisions 68630 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49587
diff changeset
81 if isinstance(node.op, Add):
a04b1c7167ba Merged revisions 68630 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49587
diff changeset
82 return left + right
a04b1c7167ba Merged revisions 68630 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49587
diff changeset
83 else:
a04b1c7167ba Merged revisions 68630 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49587
diff changeset
84 return left - right
65172
884c71cd8dc6 Fix handling on negative numbers in ast.literal_eval().
Raymond Hettinger <python@rcn.com>
parents: 64316
diff changeset
85 raise ValueError('malformed node or string: ' + repr(node))
47555
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
86 return _convert(node_or_string)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
87
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
88
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
89 def dump(node, annotate_fields=True, include_attributes=False):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
90 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
91 Return a formatted dump of the tree in *node*. This is mainly useful for
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
92 debugging purposes. The returned string will show the names and the values
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
93 for fields. This makes the code impossible to evaluate, so if evaluation is
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
94 wanted *annotate_fields* must be set to False. Attributes such as line
48410
81fd8a8baa9a Merged revisions 64434-64435,64440-64443,64445,64447-64448,64450,64452,64455,64461,64464,64466,64468 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 47555
diff changeset
95 numbers and column offsets are not dumped by default. If this is wanted,
47555
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
96 *include_attributes* can be set to True.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
97 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
98 def _format(node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
99 if isinstance(node, AST):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
100 fields = [(a, _format(b)) for a, b in iter_fields(node)]
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
101 rv = '%s(%s' % (node.__class__.__name__, ', '.join(
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
102 ('%s=%s' % field for field in fields)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
103 if annotate_fields else
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
104 (b for a, b in fields)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
105 ))
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
106 if include_attributes and node._attributes:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
107 rv += fields and ', ' or ' '
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
108 rv += ', '.join('%s=%s' % (a, _format(getattr(node, a)))
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
109 for a in node._attributes)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
110 return rv + ')'
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
111 elif isinstance(node, list):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
112 return '[%s]' % ', '.join(_format(x) for x in node)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
113 return repr(node)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
114 if not isinstance(node, AST):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
115 raise TypeError('expected AST, got %r' % node.__class__.__name__)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
116 return _format(node)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
117
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
118
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
119 def copy_location(new_node, old_node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
120 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
121 Copy source location (`lineno` and `col_offset` attributes) from
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
122 *old_node* to *new_node* if possible, and return *new_node*.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
123 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
124 for attr in 'lineno', 'col_offset':
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
125 if attr in old_node._attributes and attr in new_node._attributes \
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
126 and hasattr(old_node, attr):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
127 setattr(new_node, attr, getattr(old_node, attr))
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
128 return new_node
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
129
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
130
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
131 def fix_missing_locations(node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
132 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
133 When you compile a node tree with compile(), the compiler expects lineno and
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
134 col_offset attributes for every node that supports them. This is rather
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
135 tedious to fill in for generated nodes, so this helper adds these attributes
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
136 recursively where not already set, by setting them to the values of the
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
137 parent node. It works recursively starting at *node*.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
138 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
139 def _fix(node, lineno, col_offset):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
140 if 'lineno' in node._attributes:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
141 if not hasattr(node, 'lineno'):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
142 node.lineno = lineno
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
143 else:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
144 lineno = node.lineno
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
145 if 'col_offset' in node._attributes:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
146 if not hasattr(node, 'col_offset'):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
147 node.col_offset = col_offset
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
148 else:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
149 col_offset = node.col_offset
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
150 for child in iter_child_nodes(node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
151 _fix(child, lineno, col_offset)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
152 _fix(node, 1, 0)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
153 return node
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
154
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
155
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
156 def increment_lineno(node, n=1):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
157 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
158 Increment the line number of each node in the tree starting at *node* by *n*.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
159 This is useful to "move code" to a different location in a file.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
160 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
161 for child in walk(node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
162 if 'lineno' in child._attributes:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
163 child.lineno = getattr(child, 'lineno', 0) + n
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
164 return node
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
165
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
166
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
167 def iter_fields(node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
168 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
169 Yield a tuple of ``(fieldname, value)`` for each field in ``node._fields``
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
170 that is present on *node*.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
171 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
172 for field in node._fields:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
173 try:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
174 yield field, getattr(node, field)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
175 except AttributeError:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
176 pass
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
177
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
178
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
179 def iter_child_nodes(node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
180 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
181 Yield all direct child nodes of *node*, that is, all fields that are nodes
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
182 and all items of fields that are lists of nodes.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
183 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
184 for name, field in iter_fields(node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
185 if isinstance(field, AST):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
186 yield field
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
187 elif isinstance(field, list):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
188 for item in field:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
189 if isinstance(item, AST):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
190 yield item
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
191
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
192
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
193 def get_docstring(node, clean=True):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
194 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
195 Return the docstring for the given node or None if no docstring can
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
196 be found. If the node provided does not have docstrings a TypeError
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
197 will be raised.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
198 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
199 if not isinstance(node, (FunctionDef, ClassDef, Module)):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
200 raise TypeError("%r can't have docstrings" % node.__class__.__name__)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
201 if node.body and isinstance(node.body[0], Expr) and \
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
202 isinstance(node.body[0].value, Str):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
203 if clean:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
204 import inspect
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
205 return inspect.cleandoc(node.body[0].value.s)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
206 return node.body[0].value.s
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
207
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
208
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
209 def walk(node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
210 """
67484
2e74369b4f40 Merged revisions 87876-87877 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63079
diff changeset
211 Recursively yield all descendant nodes in the tree starting at *node*
2e74369b4f40 Merged revisions 87876-87877 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63079
diff changeset
212 (including *node* itself), in no specified order. This is useful if you
2e74369b4f40 Merged revisions 87876-87877 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63079
diff changeset
213 only want to modify nodes in place and don't care about the context.
47555
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
214 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
215 from collections import deque
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
216 todo = deque([node])
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
217 while todo:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
218 node = todo.popleft()
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
219 todo.extend(iter_child_nodes(node))
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
220 yield node
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
221
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
222
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
223 class NodeVisitor(object):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
224 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
225 A node visitor base class that walks the abstract syntax tree and calls a
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
226 visitor function for every node found. This function may return a value
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
227 which is forwarded by the `visit` method.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
228
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
229 This class is meant to be subclassed, with the subclass adding visitor
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
230 methods.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
231
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
232 Per default the visitor functions for the nodes are ``'visit_'`` +
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
233 class name of the node. So a `TryFinally` node visit function would
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
234 be `visit_TryFinally`. This behavior can be changed by overriding
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
235 the `visit` method. If no visitor function exists for a node
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
236 (return value `None`) the `generic_visit` visitor is used instead.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
237
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
238 Don't use the `NodeVisitor` if you want to apply changes to nodes during
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
239 traversing. For this a special visitor exists (`NodeTransformer`) that
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
240 allows modifications.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
241 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
242
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
243 def visit(self, node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
244 """Visit a node."""
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
245 method = 'visit_' + node.__class__.__name__
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
246 visitor = getattr(self, method, self.generic_visit)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
247 return visitor(node)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
248
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
249 def generic_visit(self, node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
250 """Called if no explicit visitor function exists for a node."""
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
251 for field, value in iter_fields(node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
252 if isinstance(value, list):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
253 for item in value:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
254 if isinstance(item, AST):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
255 self.visit(item)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
256 elif isinstance(value, AST):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
257 self.visit(value)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
258
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
259
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
260 class NodeTransformer(NodeVisitor):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
261 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
262 A :class:`NodeVisitor` subclass that walks the abstract syntax tree and
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
263 allows modification of nodes.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
264
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
265 The `NodeTransformer` will walk the AST and use the return value of the
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
266 visitor methods to replace or remove the old node. If the return value of
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
267 the visitor method is ``None``, the node will be removed from its location,
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
268 otherwise it is replaced with the return value. The return value may be the
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
269 original node in which case no replacement takes place.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
270
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
271 Here is an example transformer that rewrites all occurrences of name lookups
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
272 (``foo``) to ``data['foo']``::
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
273
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
274 class RewriteName(NodeTransformer):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
275
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
276 def visit_Name(self, node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
277 return copy_location(Subscript(
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
278 value=Name(id='data', ctx=Load()),
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
279 slice=Index(value=Str(s=node.id)),
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
280 ctx=node.ctx
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
281 ), node)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
282
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
283 Keep in mind that if the node you're operating on has child nodes you must
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
284 either transform the child nodes yourself or call the :meth:`generic_visit`
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
285 method for the node first.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
286
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
287 For nodes that were part of a collection of statements (that applies to all
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
288 statement nodes), the visitor may also return a list of nodes rather than
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
289 just a single node.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
290
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
291 Usually you use the transformer like this::
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
292
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
293 node = YourTransformer().visit(node)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
294 """
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
295
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
296 def generic_visit(self, node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
297 for field, old_value in iter_fields(node):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
298 old_value = getattr(node, field, None)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
299 if isinstance(old_value, list):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
300 new_values = []
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
301 for value in old_value:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
302 if isinstance(value, AST):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
303 value = self.visit(value)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
304 if value is None:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
305 continue
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
306 elif not isinstance(value, AST):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
307 new_values.extend(value)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
308 continue
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
309 new_values.append(value)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
310 old_value[:] = new_values
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
311 elif isinstance(old_value, AST):
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
312 new_node = self.visit(old_value)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
313 if new_node is None:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
314 delattr(node, field)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
315 else:
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
316 setattr(node, field, new_node)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
Georg Brandl <georg@python.org>
parents:
diff changeset
317 return node