Mercurial > cpython-lru_cache-weakref

annotate Doc/lib/libtoken.tex @ 28646:193b8aec00ce legacy-trunk

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Changes from Jonathan Riehl to allow his pgen extension (PEP 269) to work. This includes some more code that used to be part of pgen in the main parser; I'm okay with that. I'll see if the Windows build needs work next.
author Guido van Rossum <guido@python.org>
date Thu, 17 Apr 2003 14:55:42 +0000
parents bc4f89eefb04
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
8918
acfffe14628e New section header style.
Fred Drake <fdrake@acm.org>
parents: 8716
diff changeset
1 \section{\module{token} ---
10251
d83935d14084 Shorten a few section titles.
Fred Drake <fdrake@acm.org>
parents: 8918
diff changeset
2 Constants used with Python parse trees}
d83935d14084 Shorten a few section titles.
Fred Drake <fdrake@acm.org>
parents: 8918
diff changeset
3
8716
760dcf197048 Adjusted to use the new module synopsis support macros.
Fred Drake <fdrake@acm.org>
parents: 7877
diff changeset
4 \declaremodule{standard}{token}
760dcf197048 Adjusted to use the new module synopsis support macros.
Fred Drake <fdrake@acm.org>
parents: 7877
diff changeset
5 \modulesynopsis{Constants representing terminal nodes of the parse tree.}
10251
d83935d14084 Shorten a few section titles.
Fred Drake <fdrake@acm.org>
parents: 8918
diff changeset
6 \sectionauthor{Fred L. Drake, Jr.}{fdrake@acm.org}
8716
760dcf197048 Adjusted to use the new module synopsis support macros.
Fred Drake <fdrake@acm.org>
parents: 7877
diff changeset
7
7363
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
8
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
9 This module provides constants which represent the numeric values of
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
10 leaf nodes of the parse tree (terminal tokens). Refer to the file
13625
fdfd1049f36e Rob W. W. Hooft's spelling fixes for the Library Reference. I hope
Thomas Wouters <thomas@python.org>
parents: 10251
diff changeset
11 \file{Grammar/Grammar} in the Python distribution for the definitions
7363
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
12 of the names in the context of the language grammar. The specific
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
13 numeric values which the names map to may change between Python
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
14 versions.
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
15
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
16 This module also provides one data object and some functions. The
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
17 functions mirror definitions in the Python C header files.
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
18
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
19
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
20
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
21 \begin{datadesc}{tok_name}
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
22 Dictionary mapping the numeric values of the constants defined in this
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
23 module back to name strings, allowing more human-readable
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
24 representation of parse trees to be generated.
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
25 \end{datadesc}
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
26
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
27 \begin{funcdesc}{ISTERMINAL}{x}
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
28 Return true for terminal token values.
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
29 \end{funcdesc}
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
30
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
31 \begin{funcdesc}{ISNONTERMINAL}{x}
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
32 Return true for non-terminal token values.
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
33 \end{funcdesc}
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
34
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
35 \begin{funcdesc}{ISEOF}{x}
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
36 Return true if \var{x} is the marker indicating the end of input.
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
37 \end{funcdesc}
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
38
15261
bc4f89eefb04 Make all the \seemodule explanations consistent: start with a capitalized
Fred Drake <fdrake@acm.org>
parents: 13625
diff changeset
39
7363
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
40 \begin{seealso}
15261
bc4f89eefb04 Make all the \seemodule explanations consistent: start with a capitalized
Fred Drake <fdrake@acm.org>
parents: 13625
diff changeset
41 \seemodule{parser}{The second example for the \refmodule{parser}
bc4f89eefb04 Make all the \seemodule explanations consistent: start with a capitalized
Fred Drake <fdrake@acm.org>
parents: 13625
diff changeset
42 module shows how to use the \module{symbol}
bc4f89eefb04 Make all the \seemodule explanations consistent: start with a capitalized
Fred Drake <fdrake@acm.org>
parents: 13625
diff changeset
43 module.}
7363
f4a4723b9553 Separate the parse, symbol, and token documentation into separate files.
Fred Drake <fdrake@acm.org>
parents:
diff changeset
44 \end{seealso}