cpython-lru_cache-weakref: Doc/lib/compiler.tex annotate

annotate Doc/lib/compiler.tex @ 21467:4e35fa5d5708 legacy-trunk

Add documentation for the help() built-in; contributed by Ka-Ping Yee. This is part of SF patch #494622.

author	Fred Drake <fdrake@acm.org>
date	Tue, 18 Dec 2001 16:31:08 +0000
parents	0fbc9289a4c7
children	7863ebcff121

rev	line source
20136 fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	1 \chapter{Python compiler package \label{compiler}}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	2
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	3 \sectionauthor{Jeremy Hylton}{jeremy@zope.com}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	4
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	5
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	6 The Python compiler package is a tool for analyzing Python source code
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	7 and generating Python bytecode. The compiler contains libraries to
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	8 generate an abstract syntax tree from Python source code and to
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	9 generate Python bytecode from the tree.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	10
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	11 The \refmodule{compiler} package is a Python source to bytecode
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	12 translator written in Python. It uses the built-in parser and
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	13 standard \refmodule{parser} module to generated a concrete syntax
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	14 tree. This tree is used to generate an abstract syntax tree (AST) and
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	15 then Python bytecode.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	16
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	17 The full functionality of the package duplicates the builtin compiler
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	18 provided with the Python interpreter. It is intended to match its
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	19 behavior almost exactly. Why implement another compiler that does the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	20 same thing? The package is useful for a variety of purposes. It can
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	21 be modified more easily than the builtin compiler. The AST it
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	22 generates is useful for analyzing Python source code.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	23
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	24 This chapter explains how the various components of the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	25 \refmodule{compiler} package work. It blends reference material with
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	26 a tutorial.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	27
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	28 The following modules are part of the \refmodule{compiler} package:
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	29
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	30 \localmoduletable
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	31
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	32
20839 f6c7460ccaf8 Make sure there are no weird nesting behaviors; the PDF version will not Fred Drake <fdrake@acm.org> parents: 20136 diff changeset	33 \section{The basic interface}
20136 fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	34
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	35 \declaremodule{}{compiler}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	36
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	37 The top-level of the package defines four functions. If you import
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	38 \module{compiler}, you will get these functions and a collection of
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	39 modules contained in the package.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	40
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	41 \begin{funcdesc}{parse}{buf}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	42 Returns an abstract syntax tree for the Python source code in \var{buf}.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	43 The function raises SyntaxError if there is an error in the source
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	44 code. The return value is a \class{compiler.ast.Module} instance that
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	45 contains the tree.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	46 \end{funcdesc}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	47
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	48 \begin{funcdesc}{parseFile}{path}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	49 Return an abstract syntax tree for the Python source code in the file
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	50 specified by \var{path}. It is equivalent to
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	51 \code{parse(open(\var{path}).read())}.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	52 \end{funcdesc}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	53
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	54 \begin{funcdesc}{walk}{ast, visitor\optional{, verbose}}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	55 Do a pre-order walk over the abstract syntax tree \var{ast}. Call the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	56 appropriate method on the \var{visitor} instance for each node
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	57 encountered.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	58 \end{funcdesc}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	59
21180 0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	60 \begin{funcdesc}{compile}{source, filename, mode, flags=None,
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	61 dont_inherit=None}
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	62 Compile the string \var{source}, a Python module, statement or
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	63 expression, into a code object that can be executed by the exec
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	64 statement or \function{eval()}. This function is a replacement for the
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	65 built-in \function{compile()} function.
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	66
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	67 The \var{filename} will be used for run-time error messages.
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	68
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	69 The \var{mode} must be 'exec' to compile a module, 'single' to compile a
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	70 single (interactive) statement, or 'eval' to compile an expression.
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	71
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	72 The \var{flags} and \var{dont_inherit} arguments affect future-related
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	73 statements, but are not supported yet.
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	74 \end{funcdesc}
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	75
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	76 \begin{funcdesc}{compileFile}{source}
0fbc9289a4c7 Update docs to reflect new compile() and compileFile() Jeremy Hylton <jeremy@alum.mit.edu> parents: 20839 diff changeset	77 Compiles the file \var{source} and generates a .pyc file.
20136 fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	78 \end{funcdesc}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	79
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	80 The \module{compiler} package contains the following modules:
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	81 \refmodule[compiler.ast]{ast}, \module{consts}, \module{future},
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	82 \module{misc}, \module{pyassem}, \module{pycodegen}, \module{symbols},
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	83 \module{transformer}, and \refmodule[compiler.visitor]{visitor}.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	84
20839 f6c7460ccaf8 Make sure there are no weird nesting behaviors; the PDF version will not Fred Drake <fdrake@acm.org> parents: 20136 diff changeset	85 \section{Limitations}
20136 fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	86
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	87 There are some problems with the error checking of the compiler
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	88 package. The interpreter detects syntax errors in two distinct
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	89 phases. One set of errors is detected by the interpreter's parser,
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	90 the other set by the compiler. The compiler package relies on the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	91 interpreter's parser, so it get the first phases of error checking for
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	92 free. It implements the second phase itself, and that implement is
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	93 incomplete. For example, the compiler package does not raise an error
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	94 if a name appears more than once in an argument list:
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	95 \code{def f(x, x): ...}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	96
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	97 A future version of the compiler should fix these problems.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	98
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	99 \section{Python Abstract Syntax}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	100
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	101 The \module{compiler.ast} module defines an abstract syntax for
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	102 Python. In the abstract syntax tree, each node represents a syntactic
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	103 construct. The root of the tree is \class{Module} object.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	104
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	105 The abstract syntax offers a higher level interface to parsed Python
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	106 source code. The \ulink{\module{parser}}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	107 {http://www.python.org/doc/current/lib/module-parser.html}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	108 module and the compiler written in C for the Python interpreter use a
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	109 concrete syntax tree. The concrete syntax is tied closely to the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	110 grammar description used for the Python parser. Instead of a single
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	111 node for a construct, there are often several levels of nested nodes
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	112 that are introduced by Python's precedence rules.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	113
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	114 The abstract syntax tree is created by the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	115 \module{compiler.transformer} module. The transformer relies on the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	116 builtin Python parser to generate a concrete syntax tree. It
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	117 generates an abstract syntax tree from the concrete tree.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	118
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	119 The \module{transformer} module was created by Greg
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	120 Stein\index{Stein, Greg} and Bill Tutt\index{Tutt, Bill} for an
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	121 experimental Python-to-C compiler. The current version contains a
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	122 number of modifications and improvements, but the basic form of the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	123 abstract syntax and of the transformer are due to Stein and Tutt.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	124
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	125 \subsection{AST Nodes}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	126
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	127 \declaremodule{}{compiler.ast}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	128
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	129 The \module{compiler.ast} module is generated from a text file that
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	130 describes each node type and its elements. Each node type is
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	131 represented as a class that inherits from the abstract base class
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	132 \class{compiler.ast.Node} and defines a set of named attributes for
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	133 child nodes.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	134
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	135 \begin{classdesc}{Node}{}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	136
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	137 The \class{Node} instances are created automatically by the parser
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	138 generator. The recommended interface for specific \class{Node}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	139 instances is to use the public attributes to access child nodes. A
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	140 public attribute may be bound to a single node or to a sequence of
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	141 nodes, depending on the \class{Node} type. For example, the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	142 \member{bases} attribute of the \class{Class} node, is bound to a
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	143 list of base class nodes, and the \member{doc} attribute is bound to
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	144 a single node.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	145
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	146 Each \class{Node} instance has a \member{lineno} attribute which may
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	147 be \code{None}. XXX Not sure what the rules are for which nodes
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	148 will have a useful lineno.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	149 \end{classdesc}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	150
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	151 All \class{Node} objects offer the following methods:
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	152
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	153 \begin{methoddesc}{getChildren}{}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	154 Returns a flattened list of the child nodes and objects in the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	155 order they occur. Specifically, the order of the nodes is the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	156 order in which they appear in the Python grammar. Not all of the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	157 children are \class{Node} instances. The names of functions and
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	158 classes, for example, are plain strings.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	159 \end{methoddesc}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	160
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	161 \begin{methoddesc}{getChildNodes}{}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	162 Returns a flattened list of the child nodes in the order they
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	163 occur. This method is like \method{getChildren()}, except that it
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	164 only returns those children that are \class{Node} instances.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	165 \end{methoddesc}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	166
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	167 Two examples illustrate the general structure of \class{Node}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	168 classes. The \keyword{while} statement is defined by the following
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	169 grammar production:
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	170
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	171 \begin{verbatim}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	172 while_stmt: "while" expression ":" suite
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	173 ["else" ":" suite]
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	174 \end{verbatim}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	175
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	176 The \class{While} node has three attributes: \member{test},
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	177 \member{body}, and \member{else_}. (If the natural name for an
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	178 attribute is also a Python reserved word, it can't be used as an
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	179 attribute name. An underscore is appended to the word to make it a
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	180 legal identifier, hence \member{else_} instead of \keyword{else}.)
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	181
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	182 The \keyword{if} statement is more complicated because it can include
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	183 several tests.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	184
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	185 \begin{verbatim}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	186 if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	187 \end{verbatim}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	188
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	189 The \class{If} node only defines two attributes: \member{tests} and
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	190 \member{else_}. The \member{tests} attribute is a sequence of test
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	191 expression, consequent body pairs. There is one pair for each
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	192 \keyword{if}/\keyword{elif} clause. The first element of the pair is
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	193 the test expression. The second elements is a \class{Stmt} node that
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	194 contains the code to execute if the test is true.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	195
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	196 The \method{getChildren()} method of \class{If} returns a flat list of
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	197 child nodes. If there are three \keyword{if}/\keyword{elif} clauses
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	198 and no \keyword{else} clause, then \method{getChildren()} will return
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	199 a list of six elements: the first test expression, the first
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	200 \class{Stmt}, the second text expression, etc.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	201
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	202 The following table lists each of the \class{Node} subclasses defined
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	203 in \module{compiler.ast} and each of the public attributes available
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	204 on their instances. The values of most of the attributes are
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	205 themselves \class{Node} instances or sequences of instances. When the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	206 value is something other than an instance, the type is noted in the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	207 comment. The attributes are listed in the order in which they are
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	208 returned by \method{getChildren()} and \method{getChildNodes()}.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	209
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	210 \input{asttable}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	211
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	212
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	213 \subsection{Assignment nodes}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	214
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	215 There is a collection of nodes used to represent assignments. Each
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	216 assignment statement in the source code becomes a single
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	217 \class{Assign} node in the AST. The \member{nodes} attribute is a
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	218 list that contains a node for each assignment target. This is
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	219 necessary because assignment can be chained, e.g. \code{a = b = 2}.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	220 Each \class{Node} in the list will be one of the following classes:
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	221 \class{AssAttr}, \class{AssList}, \class{AssName}, or
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	222 \class{AssTuple}.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	223
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	224 Each target assignment node will describe the kind of object being
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	225 assigned to: \class{AssName} for a simple name, e.g. \code{a = 1}.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	226 \class{AssAttr} for an attribute assigned, e.g. \code{a.x = 1}.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	227 \class{AssList} and \class{AssTuple} for list and tuple expansion
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	228 respectively, e.g. \code{a, b, c = a_tuple}.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	229
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	230 The target assignment nodes also have a \member{flags} attribute that
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	231 indicates whether the node is being used for assignment or in a delete
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	232 statement. The \class{AssName} is also used to represent a delete
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	233 statement, e.g. \class{del x}.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	234
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	235 When an expression contains several attribute references, an
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	236 assignment or delete statement will contain only one \class{AssAttr}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	237 node -- for the final attribute reference. The other attribute
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	238 references will be represented as \class{Getattr} nodes in the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	239 \member{expr} attribute of the \class{AssAttr} instance.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	240
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	241 \subsection{Examples}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	242
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	243 This section shows several simple examples of ASTs for Python source
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	244 code. The examples demonstrate how to use the \function{parse()}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	245 function, what the repr of an AST looks like, and how to access
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	246 attributes of an AST node.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	247
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	248 The first module defines a single function. Assume it is stored in
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	249 \file{/tmp/doublelib.py}.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	250
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	251 \begin{verbatim}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	252 """This is an example module.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	253
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	254 This is the docstring.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	255 """
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	256
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	257 def double(x):
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	258 "Return twice the argument"
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	259 return x * 2
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	260 \end{verbatim}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	261
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	262 In the interactive interpreter session below, I have reformatted the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	263 long AST reprs for readability. The AST reprs use unqualified class
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	264 names. If you want to create an instance from a repr, you must import
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	265 the class names from the \module{compiler.ast} module.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	266
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	267 \begin{verbatim}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	268 >>> import compiler
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	269 >>> mod = compiler.parseFile("/tmp/doublelib.py")
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	270 >>> mod
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	271 Module('This is an example module.\n\nThis is the docstring.\n',
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	272 Stmt([Function('double', ['x'], [], 0, 'Return twice the argument',
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	273 Stmt([Return(Mul((Name('x'), Const(2))))]))]))
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	274 >>> from compiler.ast import *
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	275 >>> Module('This is an example module.\n\nThis is the docstring.\n',
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	276 ... Stmt([Function('double', ['x'], [], 0, 'Return twice the argument',
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	277 ... Stmt([Return(Mul((Name('x'), Const(2))))]))]))
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	278 Module('This is an example module.\n\nThis is the docstring.\n',
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	279 Stmt([Function('double', ['x'], [], 0, 'Return twice the argument',
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	280 Stmt([Return(Mul((Name('x'), Const(2))))]))]))
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	281 >>> mod.doc
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	282 'This is an example module.\n\nThis is the docstring.\n'
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	283 >>> for node in mod.node.nodes:
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	284 ... print node
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	285 ...
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	286 Function('double', ['x'], [], 0, 'Return twice the argument',
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	287 Stmt([Return(Mul((Name('x'), Const(2))))]))
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	288 >>> func = mod.node.nodes[0]
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	289 >>> func.code
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	290 Stmt([Return(Mul((Name('x'), Const(2))))])
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	291 \end{verbatim}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	292
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	293 \section{Using Visitors to Walk ASTs}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	294
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	295 \declaremodule{}{compiler.visitor}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	296
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	297 The visitor pattern is ... The \refmodule{compiler} package uses a
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	298 variant on the visitor pattern that takes advantage of Python's
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	299 introspection features to elminiate the need for much of the visitor's
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	300 infrastructure.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	301
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	302 The classes being visited do not need to be programmed to accept
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	303 visitors. The visitor need only define visit methods for classes it
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	304 is specifically interested in; a default visit method can handle the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	305 rest.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	306
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	307 XXX The magic \method{visit()} method for visitors.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	308
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	309 \begin{funcdesc}{walk}{tree, visitor\optional{, verbose}}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	310 \end{funcdesc}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	311
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	312 \begin{classdesc}{ASTVisitor}{}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	313
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	314 The \class{ASTVisitor} is responsible for walking over the tree in the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	315 correct order. A walk begins with a call to \method{preorder()}. For
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	316 each node, it checks the \var{visitor} argument to \method{preorder()}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	317 for a method named `visitNodeType,' where NodeType is the name of the
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	318 node's class, e.g. for a \class{While} node a \method{visitWhile()}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	319 would be called. If the method exists, it is called with the node as
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	320 its first argument.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	321
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	322 The visitor method for a particular node type can control how child
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	323 nodes are visited during the walk. The \class{ASTVisitor} modifies
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	324 the visitor argument by adding a visit method to the visitor; this
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	325 method can be used to visit a particular child node. If no visitor is
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	326 found for a particular node type, the \method{default()} method is
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	327 called.
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	328 \end{classdesc}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	329
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	330 \class{ASTVisitor} objects have the following methods:
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	331
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	332 XXX describe extra arguments
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	333
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	334 \begin{methoddesc}{default}{node\optional{, \moreargs}}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	335 \end{methoddesc}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	336
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	337 \begin{methoddesc}{dispatch}{node\optional{, \moreargs}}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	338 \end{methoddesc}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	339
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	340 \begin{methoddesc}{preorder}{tree, visitor}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	341 \end{methoddesc}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	342
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	343
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	344 \section{Bytecode Generation}
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	345
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	346 The code generator is a visitor that emits bytecodes. Each visit method
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	347 can call the \method{emit()} method to emit a new bytecode. The basic
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	348 code generator is specialized for modules, classes, and functions. An
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	349 assembler converts that emitted instructions to the low-level bytecode
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	350 format. It handles things like generator of constant lists of code
fcc216c58c87 Migrate the compiler documentation from the Tools/compiler/doc/ directory. Fred Drake <fdrake@acm.org> parents: diff changeset	351 objects and calculation of jump offsets.

Mercurial > cpython-lru_cache-weakref

annotate Doc/lib/compiler.tex @ 21467:4e35fa5d5708 legacy-trunk