Mercurial > cpython-lru_cache-weakref

annotate Lib/irix6/panelparser.py @ 5817:48b94de85bf3 legacy-trunk

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Extensive changes by AMK.
author Guido van Rossum <guido@python.org>
date Thu, 17 Jul 1997 16:29:42 +0000
parents 6ce61f212c3e
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4964
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
1 # Module 'parser'
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
2 #
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
3 # Parse S-expressions output by the Panel Editor
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
4 # (which is written in Scheme so it can't help writing S-expressions).
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
5 #
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
6 # See notes at end of file.
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
7
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
8
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
9 whitespace = ' \t\n'
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
10 operators = '()\''
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
11 separators = operators + whitespace + ';' + '"'
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
12
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
13
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
14 # Tokenize a string.
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
15 # Return a list of tokens (strings).
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
16 #
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
17 def tokenize_string(s):
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
18 tokens = []
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
19 while s:
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
20 c = s[:1]
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
21 if c in whitespace:
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
22 s = s[1:]
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
23 elif c == ';':
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
24 s = ''
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
25 elif c == '"':
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
26 n = len(s)
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
27 i = 1
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
28 while i < n:
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
29 c = s[i]
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
30 i = i+1
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
31 if c == '"': break
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
32 if c == '\\': i = i+1
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
33 tokens.append(s[:i])
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
34 s = s[i:]
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
35 elif c in operators:
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
36 tokens.append(c)
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
37 s = s[1:]
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
38 else:
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
39 n = len(s)
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
40 i = 1
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
41 while i < n:
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
42 if s[i] in separators: break
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
43 i = i+1
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
44 tokens.append(s[:i])
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
45 s = s[i:]
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
46 return tokens
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
47
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
48
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
49 # Tokenize a whole file (given as file object, not as file name).
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
50 # Return a list of tokens (strings).
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
51 #
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
52 def tokenize_file(fp):
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
53 tokens = []
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
54 while 1:
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
55 line = fp.readline()
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
56 if not line: break
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
57 tokens = tokens + tokenize_string(line)
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
58 return tokens
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
59
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
60
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
61 # Exception raised by parse_exr.
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
62 #
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
63 syntax_error = 'syntax error'
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
64
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
65
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
66 # Parse an S-expression.
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
67 # Input is a list of tokens as returned by tokenize_*().
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
68 # Return a pair (expr, tokens)
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
69 # where expr is a list representing the s-expression,
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
70 # and tokens contains the remaining tokens.
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
71 # May raise syntax_error.
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
72 #
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
73 def parse_expr(tokens):
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
74 if (not tokens) or tokens[0] <> '(':
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
75 raise syntax_error, 'expected "("'
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
76 tokens = tokens[1:]
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
77 expr = []
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
78 while 1:
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
79 if not tokens:
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
80 raise syntax_error, 'missing ")"'
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
81 if tokens[0] == ')':
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
82 return expr, tokens[1:]
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
83 elif tokens[0] == '(':
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
84 subexpr, tokens = parse_expr(tokens)
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
85 expr.append(subexpr)
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
86 else:
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
87 expr.append(tokens[0])
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
88 tokens = tokens[1:]
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
89
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
90
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
91 # Parse a file (given as file object, not as file name).
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
92 # Return a list of parsed S-expressions found at the top level.
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
93 #
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
94 def parse_file(fp):
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
95 tokens = tokenize_file(fp)
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
96 exprlist = []
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
97 while tokens:
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
98 expr, tokens = parse_expr(tokens)
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
99 exprlist.append(expr)
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
100 return exprlist
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
101
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
102
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
103 # EXAMPLE:
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
104 #
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
105 # The input
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
106 # '(hip (hop hur-ray))'
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
107 #
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
108 # passed to tokenize_string() returns the token list
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
109 # ['(', 'hip', '(', 'hop', 'hur-ray', ')', ')']
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
110 #
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
111 # When this is passed to parse_expr() it returns the expression
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
112 # ['hip', ['hop', 'hur-ray']]
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
113 # plus an empty token list (because there are no tokens left.
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
114 #
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
115 # When a file containing the example is passed to parse_file() it returns
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
116 # a list whose only element is the output of parse_expr() above:
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
117 # [['hip', ['hop', 'hur-ray']]]
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
118
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
119
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
120 # TOKENIZING:
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
121 #
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
122 # Comments start with semicolon (;) and continue till the end of the line.
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
123 #
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
124 # Tokens are separated by whitespace, except the following characters
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
125 # always form a separate token (outside strings):
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
126 # ( ) '
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
127 # Strings are enclosed in double quotes (") and backslash (\) is used
6ce61f212c3e Irix 6 specific files -- thanks Sjoerd!
Guido van Rossum <guido@python.org>
parents:
diff changeset
128 # as escape character in strings.