cpython-lru_cache-weakref: Doc/library/parser.rst annotate

annotate Doc/library/parser.rst @ 78069:a8aa77458e12 3.2

Add c_size_t and c_ssize_t to the ctypes summary table.

author	Antoine Pitrou <solipsis@pitrou.net>
date	Thu, 12 Jul 2012 20:31:50 +0200
parents	3c9232a684ae
children

rev	line source
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	1 :mod:`parser` --- Access Python parse trees
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	2 ===========================================
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	3
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	4 .. module:: parser
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	5 :synopsis: Access parse trees for Python source code.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	6 .. moduleauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	7 .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	8
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	9
44396 6556f3f22c19 Merged revisions 59605-59624 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43737 diff changeset	10 .. Copyright 1995 Virginia Polytechnic Institute and State University and Fred
6556f3f22c19 Merged revisions 59605-59624 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43737 diff changeset	11 L. Drake, Jr. This copyright notice must be distributed on all copies, but
6556f3f22c19 Merged revisions 59605-59624 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43737 diff changeset	12 this document otherwise may be distributed as part of the Python
6556f3f22c19 Merged revisions 59605-59624 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43737 diff changeset	13 distribution. No fee may be charged for this document in any representation,
6556f3f22c19 Merged revisions 59605-59624 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43737 diff changeset	14 either on paper or electronically. This restriction does not affect other
6556f3f22c19 Merged revisions 59605-59624 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43737 diff changeset	15 elements in a distributed package in any way.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	16
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	17 .. index:: single: parsing; Python source code
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	18
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	19 The :mod:`parser` module provides an interface to Python's internal parser and
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	20 byte-code compiler. The primary purpose for this interface is to allow Python
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	21 code to edit the parse tree of a Python expression and create executable code
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	22 from this. This is better than trying to parse and modify an arbitrary Python
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	23 code fragment as a string because parsing is performed in a manner identical to
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	24 the code forming the application. It is also faster.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	25
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	26 .. note::
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	27
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	28 From Python 2.5 onward, it's much more convenient to cut in at the Abstract
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	29 Syntax Tree (AST) generation and compilation stage, using the :mod:`ast`
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	30 module.
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	31
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	32 There are a few things to note about this module which are important to making
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	33 use of the data structures created. This is not a tutorial on editing the parse
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	34 trees for Python code, but some examples of using the :mod:`parser` module are
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	35 presented.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	36
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	37 Most importantly, a good understanding of the Python grammar processed by the
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	38 internal parser is required. For full information on the language syntax, refer
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	39 to :ref:`reference-index`. The parser
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	40 itself is created from a grammar specification defined in the file
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	41 :file:`Grammar/Grammar` in the standard Python distribution. The parse trees
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	42 stored in the ST objects created by this module are the actual output from the
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	43 internal parser when created by the :func:`expr` or :func:`suite` functions,
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	44 described below. The ST objects created by :func:`sequence2st` faithfully
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	45 simulate those structures. Be aware that the values of the sequences which are
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	46 considered "correct" will vary from one version of Python to another as the
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	47 formal grammar for the language is revised. However, transporting code from one
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	48 Python version to another as source text will always allow correct parse trees
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	49 to be created in the target version, with the only restriction being that
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	50 migrating to an older version of the interpreter will not support more recent
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	51 language constructs. The parse trees are not typically compatible from one
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	52 version to another, whereas source code has always been forward-compatible.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	53
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	54 Each element of the sequences returned by :func:`st2list` or :func:`st2tuple`
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	55 has a simple form. Sequences representing non-terminal elements in the grammar
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	56 always have a length greater than one. The first element is an integer which
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	57 identifies a production in the grammar. These integers are given symbolic names
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	58 in the C header file :file:`Include/graminit.h` and the Python module
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	59 :mod:`symbol`. Each additional element of the sequence represents a component
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	60 of the production as recognized in the input string: these are always sequences
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	61 which have the same form as the parent. An important aspect of this structure
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	62 which should be noted is that keywords used to identify the parent node type,
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	63 such as the keyword :keyword:`if` in an :const:`if_stmt`, are included in the
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	64 node tree without any special treatment. For example, the :keyword:`if` keyword
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	65 is represented by the tuple ``(1, 'if')``, where ``1`` is the numeric value
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	66 associated with all :const:`NAME` tokens, including variable and function names
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	67 defined by the user. In an alternate form returned when line number information
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	68 is requested, the same token might be represented as ``(1, 'if', 12)``, where
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	69 the ``12`` represents the line number at which the terminal symbol was found.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	70
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	71 Terminal elements are represented in much the same way, but without any child
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	72 elements and the addition of the source text which was identified. The example
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	73 of the :keyword:`if` keyword above is representative. The various types of
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	74 terminal symbols are defined in the C header file :file:`Include/token.h` and
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	75 the Python module :mod:`token`.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	76
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	77 The ST objects are not required to support the functionality of this module,
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	78 but are provided for three purposes: to allow an application to amortize the
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	79 cost of processing complex parse trees, to provide a parse tree representation
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	80 which conserves memory space when compared to the Python list or tuple
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	81 representation, and to ease the creation of additional modules in C which
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	82 manipulate parse trees. A simple "wrapper" class may be created in Python to
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	83 hide the use of ST objects.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	84
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	85 The :mod:`parser` module defines functions for a few distinct purposes. The
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	86 most important purposes are to create ST objects and to convert ST objects to
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	87 other representations such as parse trees and compiled code objects, but there
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	88 are also functions which serve to query the type of parse tree represented by an
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	89 ST object.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	90
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	91
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	92 .. seealso::
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	93
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	94 Module :mod:`symbol`
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	95 Useful constants representing internal nodes of the parse tree.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	96
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	97 Module :mod:`token`
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	98 Useful constants representing leaf nodes of the parse tree and functions for
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	99 testing node values.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	100
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	101
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	102 .. _creating-sts:
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	103
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	104 Creating ST Objects
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	105 -------------------
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	106
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	107 ST objects may be created from source code or from a parse tree. When creating
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	108 an ST object from source, different functions are used to create the ``'eval'``
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	109 and ``'exec'`` forms.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	110
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	111
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	112 .. function:: expr(source)
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	113
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	114 The :func:`expr` function parses the parameter source as if it were an input
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	115 to ``compile(source, 'file.py', 'eval')``. If the parse succeeds, an ST object
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	116 is created to hold the internal parse tree representation, otherwise an
65116 90e1735df4d9 Merged revisions 83561,83563,83565-83566,83569,83571,83574-83575,83580,83584,83599,83612,83659,83977,84015-84018,84020,84141 via svnmerge from Georg Brandl <georg@python.org> parents: 50569 diff changeset	117 appropriate exception is raised.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	118
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	119
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	120 .. function:: suite(source)
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	121
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	122 The :func:`suite` function parses the parameter source as if it were an input
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	123 to ``compile(source, 'file.py', 'exec')``. If the parse succeeds, an ST object
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	124 is created to hold the internal parse tree representation, otherwise an
65116 90e1735df4d9 Merged revisions 83561,83563,83565-83566,83569,83571,83574-83575,83580,83584,83599,83612,83659,83977,84015-84018,84020,84141 via svnmerge from Georg Brandl <georg@python.org> parents: 50569 diff changeset	125 appropriate exception is raised.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	126
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	127
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	128 .. function:: sequence2st(sequence)
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	129
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	130 This function accepts a parse tree represented as a sequence and builds an
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	131 internal representation if possible. If it can validate that the tree conforms
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	132 to the Python grammar and all nodes are valid node types in the host version of
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	133 Python, an ST object is created from the internal representation and returned
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	134 to the called. If there is a problem creating the internal representation, or
65116 90e1735df4d9 Merged revisions 83561,83563,83565-83566,83569,83571,83574-83575,83580,83584,83599,83612,83659,83977,84015-84018,84020,84141 via svnmerge from Georg Brandl <georg@python.org> parents: 50569 diff changeset	135 if the tree cannot be validated, a :exc:`ParserError` exception is raised. An
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	136 ST object created this way should not be assumed to compile correctly; normal
65116 90e1735df4d9 Merged revisions 83561,83563,83565-83566,83569,83571,83574-83575,83580,83584,83599,83612,83659,83977,84015-84018,84020,84141 via svnmerge from Georg Brandl <georg@python.org> parents: 50569 diff changeset	137 exceptions raised by compilation may still be initiated when the ST object is
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	138 passed to :func:`compilest`. This may indicate problems not related to syntax
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	139 (such as a :exc:`MemoryError` exception), but may also be due to constructs such
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	140 as the result of parsing ``del f(0)``, which escapes the Python parser but is
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	141 checked by the bytecode compiler.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	142
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	143 Sequences representing terminal tokens may be represented as either two-element
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	144 lists of the form ``(1, 'name')`` or as three-element lists of the form ``(1,
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	145 'name', 56)``. If the third element is present, it is assumed to be a valid
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	146 line number. The line number may be specified for any subset of the terminal
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	147 symbols in the input tree.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	148
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	149
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	150 .. function:: tuple2st(sequence)
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	151
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	152 This is the same function as :func:`sequence2st`. This entry point is
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	153 maintained for backward compatibility.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	154
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	155
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	156 .. _converting-sts:
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	157
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	158 Converting ST Objects
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	159 ---------------------
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	160
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	161 ST objects, regardless of the input used to create them, may be converted to
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	162 parse trees represented as list- or tuple- trees, or may be compiled into
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	163 executable code objects. Parse trees may be extracted with or without line
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	164 numbering information.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	165
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	166
55802 106ee4eb5970 Switch more function arguments docs to new-style. Georg Brandl <georg@python.org> parents: 50569 diff changeset	167 .. function:: st2list(st, line_info=False, col_info=False)
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	168
48558 fcbd9f4f095c Remove "ast" aliases from the parser module. Georg Brandl <georg@python.org> parents: 47555 diff changeset	169 This function accepts an ST object from the caller in st and returns a
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	170 Python list representing the equivalent parse tree. The resulting list
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	171 representation can be used for inspection or the creation of a new parse tree in
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	172 list form. This function does not fail so long as memory is available to build
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	173 the list representation. If the parse tree will only be used for inspection,
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	174 :func:`st2tuple` should be used instead to reduce memory consumption and
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	175 fragmentation. When the list representation is required, this function is
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	176 significantly faster than retrieving a tuple representation and converting that
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	177 to nested lists.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	178
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	179 If line_info is true, line number information will be included for all
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	180 terminal tokens as a third element of the list representing the token. Note
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	181 that the line number provided specifies the line on which the token ends.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	182 This information is omitted if the flag is false or omitted.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	183
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	184
55802 106ee4eb5970 Switch more function arguments docs to new-style. Georg Brandl <georg@python.org> parents: 50569 diff changeset	185 .. function:: st2tuple(st, line_info=False, col_info=False)
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	186
48558 fcbd9f4f095c Remove "ast" aliases from the parser module. Georg Brandl <georg@python.org> parents: 47555 diff changeset	187 This function accepts an ST object from the caller in st and returns a
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	188 Python tuple representing the equivalent parse tree. Other than returning a
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	189 tuple instead of a list, this function is identical to :func:`st2list`.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	190
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	191 If line_info is true, line number information will be included for all
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	192 terminal tokens as a third element of the list representing the token. This
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	193 information is omitted if the flag is false or omitted.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	194
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	195
55802 106ee4eb5970 Switch more function arguments docs to new-style. Georg Brandl <georg@python.org> parents: 50569 diff changeset	196 .. function:: compilest(st, filename='<syntax-tree>')
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	197
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	198 .. index::
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	199 builtin: exec
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	200 builtin: eval
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	201
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	202 The Python byte compiler can be invoked on an ST object to produce code objects
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	203 which can be used as part of a call to the built-in :func:`exec` or :func:`eval`
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	204 functions. This function provides the interface to the compiler, passing the
48558 fcbd9f4f095c Remove "ast" aliases from the parser module. Georg Brandl <georg@python.org> parents: 47555 diff changeset	205 internal parse tree from st to the parser, using the source file name
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	206 specified by the filename parameter. The default value supplied for filename
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	207 indicates that the source was an ST object.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	208
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	209 Compiling an ST object may result in exceptions related to compilation; an
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	210 example would be a :exc:`SyntaxError` caused by the parse tree for ``del f(0)``:
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	211 this statement is considered legal within the formal grammar for Python but is
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	212 not a legal language construct. The :exc:`SyntaxError` raised for this
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	213 condition is actually generated by the Python byte-compiler normally, which is
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	214 why it can be raised at this point by the :mod:`parser` module. Most causes of
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	215 compilation failure can be diagnosed programmatically by inspection of the parse
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	216 tree.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	217
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	218
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	219 .. _querying-sts:
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	220
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	221 Queries on ST Objects
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	222 ---------------------
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	223
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	224 Two functions are provided which allow an application to determine if an ST was
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	225 created as an expression or a suite. Neither of these functions can be used to
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	226 determine if an ST was created from source code via :func:`expr` or
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	227 :func:`suite` or from a parse tree via :func:`sequence2st`.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	228
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	229
48558 fcbd9f4f095c Remove "ast" aliases from the parser module. Georg Brandl <georg@python.org> parents: 47555 diff changeset	230 .. function:: isexpr(st)
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	231
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	232 .. index:: builtin: compile
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	233
48558 fcbd9f4f095c Remove "ast" aliases from the parser module. Georg Brandl <georg@python.org> parents: 47555 diff changeset	234 When st represents an ``'eval'`` form, this function returns true, otherwise
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	235 it returns false. This is useful, since code objects normally cannot be queried
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	236 for this information using existing built-in functions. Note that the code
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	237 objects created by :func:`compilest` cannot be queried like this either, and
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	238 are identical to those created by the built-in :func:`compile` function.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	239
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	240
48558 fcbd9f4f095c Remove "ast" aliases from the parser module. Georg Brandl <georg@python.org> parents: 47555 diff changeset	241 .. function:: issuite(st)
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	242
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	243 This function mirrors :func:`isexpr` in that it reports whether an ST object
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	244 represents an ``'exec'`` form, commonly known as a "suite." It is not safe to
48558 fcbd9f4f095c Remove "ast" aliases from the parser module. Georg Brandl <georg@python.org> parents: 47555 diff changeset	245 assume that this function is equivalent to ``not isexpr(st)``, as additional
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	246 syntactic fragments may be supported in the future.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	247
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	248
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	249 .. _st-errors:
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	250
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	251 Exceptions and Error Handling
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	252 -----------------------------
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	253
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	254 The parser module defines a single exception, but may also pass other built-in
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	255 exceptions from other portions of the Python runtime environment. See each
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	256 function for information about the exceptions it can raise.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	257
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	258
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	259 .. exception:: ParserError
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	260
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	261 Exception raised when a failure occurs within the parser module. This is
65116 90e1735df4d9 Merged revisions 83561,83563,83565-83566,83569,83571,83574-83575,83580,83584,83599,83612,83659,83977,84015-84018,84020,84141 via svnmerge from Georg Brandl <georg@python.org> parents: 50569 diff changeset	262 generally produced for validation failures rather than the built-in
90e1735df4d9 Merged revisions 83561,83563,83565-83566,83569,83571,83574-83575,83580,83584,83599,83612,83659,83977,84015-84018,84020,84141 via svnmerge from Georg Brandl <georg@python.org> parents: 50569 diff changeset	263 :exc:`SyntaxError` raised during normal parsing. The exception argument is
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	264 either a string describing the reason of the failure or a tuple containing a
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	265 sequence causing the failure from a parse tree passed to :func:`sequence2st`
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	266 and an explanatory string. Calls to :func:`sequence2st` need to be able to
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	267 handle either type of exception, while calls to other functions in the module
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	268 will only need to be aware of the simple string values.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	269
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	270 Note that the functions :func:`compilest`, :func:`expr`, and :func:`suite` may
66566 340369fb2e70 Merged revisions 86892 via svnmerge from Éric Araujo <merwok@netwok.org> parents: 66472 diff changeset	271 raise exceptions which are normally raised by the parsing and compilation
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	272 process. These include the built in exceptions :exc:`MemoryError`,
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	273 :exc:`OverflowError`, :exc:`SyntaxError`, and :exc:`SystemError`. In these
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	274 cases, these exceptions carry all the meaning normally associated with them.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	275 Refer to the descriptions of each function for detailed information.
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	276
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	277
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	278 .. _st-objects:
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	279
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	280 ST Objects
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	281 ----------
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	282
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	283 Ordered and equality comparisons are supported between ST objects. Pickling of
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	284 ST objects (using the :mod:`pickle` module) is also supported.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	285
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	286
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	287 .. data:: STType
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	288
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	289 The type of the objects returned by :func:`expr`, :func:`suite` and
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	290 :func:`sequence2st`.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	291
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	292 ST objects have the following methods:
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	293
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	294
55802 106ee4eb5970 Switch more function arguments docs to new-style. Georg Brandl <georg@python.org> parents: 50569 diff changeset	295 .. method:: ST.compile(filename='<syntax-tree>')
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	296
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	297 Same as ``compilest(st, filename)``.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	298
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	299
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	300 .. method:: ST.isexpr()
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	301
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	302 Same as ``isexpr(st)``.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	303
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	304
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	305 .. method:: ST.issuite()
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	306
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	307 Same as ``issuite(st)``.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	308
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	309
55802 106ee4eb5970 Switch more function arguments docs to new-style. Georg Brandl <georg@python.org> parents: 50569 diff changeset	310 .. method:: ST.tolist(line_info=False, col_info=False)
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	311
55802 106ee4eb5970 Switch more function arguments docs to new-style. Georg Brandl <georg@python.org> parents: 50569 diff changeset	312 Same as ``st2list(st, line_info, col_info)``.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	313
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	314
55802 106ee4eb5970 Switch more function arguments docs to new-style. Georg Brandl <georg@python.org> parents: 50569 diff changeset	315 .. method:: ST.totuple(line_info=False, col_info=False)
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	316
55802 106ee4eb5970 Switch more function arguments docs to new-style. Georg Brandl <georg@python.org> parents: 50569 diff changeset	317 Same as ``st2tuple(st, line_info, col_info)``.
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	318
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	319
66472 a9f0ec604c8a Merged revisions 85572-85573,85606,85609-85622,85624,85626-85627,85629,85631,85633,85635-85636,85638-85639,85641-85642 via svnmerge from Georg Brandl <georg@python.org> parents: 65116 diff changeset	320 Example: Emulation of :func:`compile`
a9f0ec604c8a Merged revisions 85572-85573,85606,85609-85622,85624,85626-85627,85629,85631,85633,85635-85636,85638-85639,85641-85642 via svnmerge from Georg Brandl <georg@python.org> parents: 65116 diff changeset	321 -------------------------------------
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	322
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	323 While many useful operations may take place between parsing and bytecode
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	324 generation, the simplest operation is to do nothing. For this purpose, using
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	325 the :mod:`parser` module to produce an intermediate data structure is equivalent
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	326 to the code ::
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	327
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	328 >>> code = compile('a + 5', 'file.py', 'eval')
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	329 >>> a = 5
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	330 >>> eval(code)
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	331 10
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	332
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	333 The equivalent operation using the :mod:`parser` module is somewhat longer, and
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	334 allows the intermediate internal parse tree to be retained as an ST object::
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	335
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	336 >>> import parser
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	337 >>> st = parser.expr('a + 5')
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	338 >>> code = st.compile('file.py')
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	339 >>> a = 5
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	340 >>> eval(code)
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	341 10
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	342
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	343 An application which needs both ST and code objects can package this code into
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	344 readily available functions::
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	345
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	346 import parser
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	347
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	348 def load_suite(source_string):
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	349 st = parser.suite(source_string)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	350 return st, st.compile()
42604 9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	351
9e1529bf0442 Move the 3k reST doc tree in place. Georg Brandl <georg@python.org> parents: diff changeset	352 def load_expression(source_string):
47555 7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	353 st = parser.expr(source_string)
7d824fd131ad Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from Georg Brandl <georg@python.org> parents: 44396 diff changeset	354 return st, st.compile()

Mercurial > cpython-lru_cache-weakref

annotate Doc/library/parser.rst @ 78069:a8aa77458e12 3.2