Mercurial > cpython-lru_cache-weakref
annotate Lib/distutils/text_file.py @ 79675:c0a2294de787 2.7
Issue #16199: Backport 2to3 typo fix from 3.2.
| author | Chris Jerdonek <chris.jerdonek@gmail.com> |
|---|---|
| date | Thu, 11 Oct 2012 16:15:46 -0700 |
| parents | 3e2b7965ada9 |
| children |
| rev | line source |
|---|---|
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
1 """text_file |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
2 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
3 provides the TextFile class, which gives an interface to text files |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
4 that (optionally) takes care of stripping comments, ignoring blank |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
5 lines, and joining lines with backslashes.""" |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
6 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
7 __revision__ = "$Id$" |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
8 |
|
57847
3e2b7965ada9
massive import cleaning in Distutils
Tarek Ziadé <ziade.tarek@gmail.com>
parents:
51792
diff
changeset
|
9 import sys |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
10 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
11 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
12 class TextFile: |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
13 |
| 11342 | 14 """Provides a file-like object that takes care of all the things you |
| 15 commonly want to do when processing a text file that has some | |
|
14640
3fefeafa3f4a
[change from 2000/04/17, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
11342
diff
changeset
|
16 line-by-line syntax: strip comments (as long as "#" is your |
|
3fefeafa3f4a
[change from 2000/04/17, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
11342
diff
changeset
|
17 comment character), skip blank lines, join adjacent lines by |
|
3fefeafa3f4a
[change from 2000/04/17, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
11342
diff
changeset
|
18 escaping the newline (ie. backslash at end of line), strip |
|
3fefeafa3f4a
[change from 2000/04/17, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
11342
diff
changeset
|
19 leading and/or trailing whitespace. All of these are optional |
|
3fefeafa3f4a
[change from 2000/04/17, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
11342
diff
changeset
|
20 and independently controllable. |
| 11342 | 21 |
| 22 Provides a 'warn()' method so you can generate warning messages that | |
| 23 report physical line number, even if the logical line in question | |
| 24 spans multiple physical lines. Also provides 'unreadline()' for | |
| 25 implementing line-at-a-time lookahead. | |
| 26 | |
| 27 Constructor is called as: | |
| 28 | |
| 29 TextFile (filename=None, file=None, **options) | |
| 30 | |
| 31 It bombs (RuntimeError) if both 'filename' and 'file' are None; | |
| 32 'filename' should be a string, and 'file' a file object (or | |
| 33 something that provides 'readline()' and 'close()' methods). It is | |
| 34 recommended that you supply at least 'filename', so that TextFile | |
| 35 can include it in warning messages. If 'file' is not supplied, | |
| 36 TextFile creates its own using the 'open()' builtin. | |
| 37 | |
| 38 The options are all boolean, and affect the value returned by | |
| 39 'readline()': | |
| 40 strip_comments [default: true] | |
| 41 strip from "#" to end-of-line, as well as any whitespace | |
| 42 leading up to the "#" -- unless it is escaped by a backslash | |
| 43 lstrip_ws [default: false] | |
| 44 strip leading whitespace from each line before returning it | |
| 45 rstrip_ws [default: true] | |
| 46 strip trailing whitespace (including line terminator!) from | |
| 47 each line before returning it | |
| 48 skip_blanks [default: true} | |
| 49 skip lines that are empty *after* stripping comments and | |
|
14640
3fefeafa3f4a
[change from 2000/04/17, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
11342
diff
changeset
|
50 whitespace. (If both lstrip_ws and rstrip_ws are false, |
| 11342 | 51 then some lines may consist of solely whitespace: these will |
| 52 *not* be skipped, even if 'skip_blanks' is true.) | |
| 53 join_lines [default: false] | |
| 54 if a backslash is the last non-newline character on a line | |
| 55 after stripping comments and whitespace, join the following line | |
| 56 to it to form one "logical line"; if N consecutive lines end | |
| 57 with a backslash, then N+1 physical lines will be joined to | |
| 58 form one logical line. | |
|
14640
3fefeafa3f4a
[change from 2000/04/17, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
11342
diff
changeset
|
59 collapse_join [default: false] |
|
3fefeafa3f4a
[change from 2000/04/17, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
11342
diff
changeset
|
60 strip leading whitespace from lines that are joined to their |
|
3fefeafa3f4a
[change from 2000/04/17, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
11342
diff
changeset
|
61 predecessor; only matters if (join_lines and not lstrip_ws) |
| 11342 | 62 |
| 63 Note that since 'rstrip_ws' can strip the trailing newline, the | |
| 64 semantics of 'readline()' must differ from those of the builtin file | |
| 65 object's 'readline()' method! In particular, 'readline()' returns | |
| 66 None for end-of-file: an empty string might just be a blank line (or | |
| 67 an all-whitespace line), if 'rstrip_ws' is true but 'skip_blanks' is | |
| 68 not.""" | |
| 69 | |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
70 default_options = { 'strip_comments': 1, |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
71 'skip_blanks': 1, |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
72 'lstrip_ws': 0, |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
73 'rstrip_ws': 1, |
|
14640
3fefeafa3f4a
[change from 2000/04/17, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
11342
diff
changeset
|
74 'join_lines': 0, |
|
3fefeafa3f4a
[change from 2000/04/17, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
11342
diff
changeset
|
75 'collapse_join': 0, |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
76 } |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
77 |
|
10447
d3762933bff8
Changes to allow passing an open file to the constructor (to support
Greg Ward <gward@python.net>
parents:
10360
diff
changeset
|
78 def __init__ (self, filename=None, file=None, **options): |
| 11342 | 79 """Construct a new TextFile object. At least one of 'filename' |
| 80 (a string) and 'file' (a file-like object) must be supplied. | |
| 81 They keyword argument options are described above and affect | |
| 82 the values returned by 'readline()'.""" | |
|
10447
d3762933bff8
Changes to allow passing an open file to the constructor (to support
Greg Ward <gward@python.net>
parents:
10360
diff
changeset
|
83 |
|
d3762933bff8
Changes to allow passing an open file to the constructor (to support
Greg Ward <gward@python.net>
parents:
10360
diff
changeset
|
84 if filename is None and file is None: |
|
d3762933bff8
Changes to allow passing an open file to the constructor (to support
Greg Ward <gward@python.net>
parents:
10360
diff
changeset
|
85 raise RuntimeError, \ |
| 21267 | 86 "you must supply either or both of 'filename' and 'file'" |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
87 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
88 # set values for all options -- either from client option hash |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
89 # or fallback to default_options |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
90 for opt in self.default_options.keys(): |
|
45413
8a187f562959
Removed uses of dict.has_key() from distutils, and uses of
Guido van Rossum <guido@python.org>
parents:
26210
diff
changeset
|
91 if opt in options: |
|
11185
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
92 setattr (self, opt, options[opt]) |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
93 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
94 else: |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
95 setattr (self, opt, self.default_options[opt]) |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
96 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
97 # sanity check client option hash |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
98 for opt in options.keys(): |
|
45413
8a187f562959
Removed uses of dict.has_key() from distutils, and uses of
Guido van Rossum <guido@python.org>
parents:
26210
diff
changeset
|
99 if opt not in self.default_options: |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
100 raise KeyError, "invalid TextFile option '%s'" % opt |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
101 |
|
10447
d3762933bff8
Changes to allow passing an open file to the constructor (to support
Greg Ward <gward@python.net>
parents:
10360
diff
changeset
|
102 if file is None: |
|
d3762933bff8
Changes to allow passing an open file to the constructor (to support
Greg Ward <gward@python.net>
parents:
10360
diff
changeset
|
103 self.open (filename) |
|
d3762933bff8
Changes to allow passing an open file to the constructor (to support
Greg Ward <gward@python.net>
parents:
10360
diff
changeset
|
104 else: |
|
d3762933bff8
Changes to allow passing an open file to the constructor (to support
Greg Ward <gward@python.net>
parents:
10360
diff
changeset
|
105 self.filename = filename |
|
d3762933bff8
Changes to allow passing an open file to the constructor (to support
Greg Ward <gward@python.net>
parents:
10360
diff
changeset
|
106 self.file = file |
|
d3762933bff8
Changes to allow passing an open file to the constructor (to support
Greg Ward <gward@python.net>
parents:
10360
diff
changeset
|
107 self.current_line = 0 # assuming that file is at BOF! |
|
10483
7474ad7e0a44
Added 'linestart' array and 'unreadline()' method (makes parsing a lot easier).
Greg Ward <gward@python.net>
parents:
10447
diff
changeset
|
108 |
|
10492
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
109 # 'linebuf' is a stack of lines that will be emptied before we |
|
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
110 # actually read from the file; it's only populated by an |
|
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
111 # 'unreadline()' operation |
|
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
112 self.linebuf = [] |
| 21267 | 113 |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
114 |
|
10447
d3762933bff8
Changes to allow passing an open file to the constructor (to support
Greg Ward <gward@python.net>
parents:
10360
diff
changeset
|
115 def open (self, filename): |
| 11342 | 116 """Open a new file named 'filename'. This overrides both the |
| 117 'filename' and 'file' arguments to the constructor.""" | |
| 118 | |
|
10447
d3762933bff8
Changes to allow passing an open file to the constructor (to support
Greg Ward <gward@python.net>
parents:
10360
diff
changeset
|
119 self.filename = filename |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
120 self.file = open (self.filename, 'r') |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
121 self.current_line = 0 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
122 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
123 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
124 def close (self): |
| 11342 | 125 """Close the current file and forget everything we know about it |
| 126 (filename, current line number).""" | |
| 127 | |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
128 self.file.close () |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
129 self.file = None |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
130 self.filename = None |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
131 self.current_line = None |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
132 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
133 |
|
14641
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
134 def gen_error (self, msg, line=None): |
|
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
135 outmsg = [] |
|
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
136 if line is None: |
|
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
137 line = self.current_line |
|
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
138 outmsg.append(self.filename + ", ") |
|
57847
3e2b7965ada9
massive import cleaning in Distutils
Tarek Ziadé <ziade.tarek@gmail.com>
parents:
51792
diff
changeset
|
139 if isinstance(line, (list, tuple)): |
|
14641
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
140 outmsg.append("lines %d-%d: " % tuple (line)) |
|
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
141 else: |
|
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
142 outmsg.append("line %d: " % line) |
|
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
143 outmsg.append(str(msg)) |
|
57847
3e2b7965ada9
massive import cleaning in Distutils
Tarek Ziadé <ziade.tarek@gmail.com>
parents:
51792
diff
changeset
|
144 return ''.join(outmsg) |
|
14641
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
145 |
|
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
146 |
|
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
147 def error (self, msg, line=None): |
|
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
148 raise ValueError, "error: " + self.gen_error(msg, line) |
|
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
149 |
|
10610
3f1e72c71b3a
'warn()' method now takes an optional line number.
Greg Ward <gward@python.net>
parents:
10492
diff
changeset
|
150 def warn (self, msg, line=None): |
| 11342 | 151 """Print (to stderr) a warning message tied to the current logical |
| 152 line in the current file. If the current logical line in the | |
| 153 file spans multiple physical lines, the warning refers to the | |
| 154 whole range, eg. "lines 3-5". If 'line' supplied, it overrides | |
| 155 the current line number; it may be a list or tuple to indicate a | |
| 156 range of physical lines, or an integer for a single physical | |
| 157 line.""" | |
|
14641
8a7d47d07981
[change from 2000/08/11, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
14640
diff
changeset
|
158 sys.stderr.write("warning: " + self.gen_error(msg, line) + "\n") |
| 9904 | 159 |
| 160 | |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
161 def readline (self): |
| 11342 | 162 """Read and return a single logical line from the current file (or |
| 163 from an internal buffer if lines have previously been "unread" | |
| 164 with 'unreadline()'). If the 'join_lines' option is true, this | |
| 165 may involve reading multiple physical lines concatenated into a | |
| 166 single string. Updates the current line number, so calling | |
| 167 'warn()' after 'readline()' emits a warning about the physical | |
| 168 line(s) just read. Returns None on end-of-file, since the empty | |
| 169 string can occur if 'rstrip_ws' is true but 'strip_blanks' is | |
| 170 not.""" | |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
171 |
|
10492
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
172 # If any "unread" lines waiting in 'linebuf', return the top |
|
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
173 # one. (We don't actually buffer read-ahead data -- lines only |
|
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
174 # get put in 'linebuf' if the client explicitly does an |
|
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
175 # 'unreadline()'. |
|
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
176 if self.linebuf: |
|
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
177 line = self.linebuf[-1] |
|
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
178 del self.linebuf[-1] |
|
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
179 return line |
|
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
180 |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
181 buildup_line = '' |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
182 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
183 while 1: |
|
11185
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
184 # read the line, make it None if EOF |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
185 line = self.file.readline() |
|
11185
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
186 if line == '': line = None |
|
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
187 |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
188 if self.strip_comments and line: |
|
11185
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
189 |
|
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
190 # Look for the first "#" in the line. If none, never |
|
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
191 # mind. If we find one and it's the first character, or |
|
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
192 # is not preceded by "\", then it starts a comment -- |
|
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
193 # strip the comment, strip whitespace before it, and |
|
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
194 # carry on. Otherwise, it's just an escaped "#", so |
|
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
195 # unescape it (and any other escaped "#"'s that might be |
|
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
196 # lurking in there) and otherwise leave the line alone. |
|
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
197 |
|
57847
3e2b7965ada9
massive import cleaning in Distutils
Tarek Ziadé <ziade.tarek@gmail.com>
parents:
51792
diff
changeset
|
198 pos = line.find("#") |
|
11185
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
199 if pos == -1: # no "#" -- no comments |
|
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
200 pass |
|
14643
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
201 |
|
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
202 # It's definitely a comment -- either "#" is the first |
|
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
203 # character, or it's elsewhere and unescaped. |
|
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
204 elif pos == 0 or line[pos-1] != "\\": |
| 11342 | 205 # Have to preserve the trailing newline, because it's |
| 206 # the job of a later step (rstrip_ws) to remove it -- | |
| 207 # and if rstrip_ws is false, we'd better preserve it! | |
| 208 # (NB. this means that if the final line is all comment | |
| 209 # and has no trailing newline, we will think that it's | |
|
11185
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
210 # EOF; I think that's OK.) |
| 11342 | 211 eol = (line[-1] == '\n') and '\n' or '' |
| 212 line = line[0:pos] + eol | |
| 21267 | 213 |
|
14643
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
214 # If all that's left is whitespace, then skip line |
|
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
215 # *now*, before we try to join it to 'buildup_line' -- |
|
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
216 # that way constructs like |
|
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
217 # hello \\ |
|
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
218 # # comment that should be ignored |
|
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
219 # there |
|
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
220 # result in "hello there". |
|
57847
3e2b7965ada9
massive import cleaning in Distutils
Tarek Ziadé <ziade.tarek@gmail.com>
parents:
51792
diff
changeset
|
221 if line.strip() == "": |
|
14643
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
222 continue |
|
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
223 |
|
11185
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
224 else: # it's an escaped "#" |
|
57847
3e2b7965ada9
massive import cleaning in Distutils
Tarek Ziadé <ziade.tarek@gmail.com>
parents:
51792
diff
changeset
|
225 line = line.replace("\\#", "#") |
| 21267 | 226 |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
227 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
228 # did previous line end with a backslash? then accumulate |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
229 if self.join_lines and buildup_line: |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
230 # oops: end of file |
|
11185
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
231 if line is None: |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
232 self.warn ("continuation line immediately precedes " |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
233 "end-of-file") |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
234 return buildup_line |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
235 |
|
14640
3fefeafa3f4a
[change from 2000/04/17, propagating now to distutils copy]
Greg Ward <gward@python.net>
parents:
11342
diff
changeset
|
236 if self.collapse_join: |
|
57847
3e2b7965ada9
massive import cleaning in Distutils
Tarek Ziadé <ziade.tarek@gmail.com>
parents:
51792
diff
changeset
|
237 line = line.lstrip() |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
238 line = buildup_line + line |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
239 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
240 # careful: pay attention to line number when incrementing it |
|
57847
3e2b7965ada9
massive import cleaning in Distutils
Tarek Ziadé <ziade.tarek@gmail.com>
parents:
51792
diff
changeset
|
241 if isinstance(self.current_line, list): |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
242 self.current_line[1] = self.current_line[1] + 1 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
243 else: |
|
14643
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
244 self.current_line = [self.current_line, |
|
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
245 self.current_line+1] |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
246 # just an ordinary line, read it as usual |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
247 else: |
|
11185
dbb8083f0ce7
Allow comment characters (#) to be escaped:
Greg Ward <gward@python.net>
parents:
10610
diff
changeset
|
248 if line is None: # eof |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
249 return None |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
250 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
251 # still have to be careful about incrementing the line number! |
|
57847
3e2b7965ada9
massive import cleaning in Distutils
Tarek Ziadé <ziade.tarek@gmail.com>
parents:
51792
diff
changeset
|
252 if isinstance(self.current_line, list): |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
253 self.current_line = self.current_line[1] + 1 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
254 else: |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
255 self.current_line = self.current_line + 1 |
| 21267 | 256 |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
257 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
258 # strip whitespace however the client wants (leading and |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
259 # trailing, or one or the other, or neither) |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
260 if self.lstrip_ws and self.rstrip_ws: |
|
57847
3e2b7965ada9
massive import cleaning in Distutils
Tarek Ziadé <ziade.tarek@gmail.com>
parents:
51792
diff
changeset
|
261 line = line.strip() |
| 11342 | 262 elif self.lstrip_ws: |
|
57847
3e2b7965ada9
massive import cleaning in Distutils
Tarek Ziadé <ziade.tarek@gmail.com>
parents:
51792
diff
changeset
|
263 line = line.lstrip() |
| 11342 | 264 elif self.rstrip_ws: |
|
57847
3e2b7965ada9
massive import cleaning in Distutils
Tarek Ziadé <ziade.tarek@gmail.com>
parents:
51792
diff
changeset
|
265 line = line.rstrip() |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
266 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
267 # blank line (whether we rstrip'ed or not)? skip to next line |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
268 # if appropriate |
| 14642 | 269 if (line == '' or line == '\n') and self.skip_blanks: |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
270 continue |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
271 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
272 if self.join_lines: |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
273 if line[-1] == '\\': |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
274 buildup_line = line[:-1] |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
275 continue |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
276 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
277 if line[-2:] == '\\\n': |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
278 buildup_line = line[0:-2] + '\n' |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
279 continue |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
280 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
281 # well, I guess there's some actual content there: return it |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
282 return line |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
283 |
|
14643
b954c33f0b4b
Changed so lines that are all comment (or just whitespace + comment)
Greg Ward <gward@python.net>
parents:
14642
diff
changeset
|
284 # readline () |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
285 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
286 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
287 def readlines (self): |
| 11342 | 288 """Read and return the list of all logical lines remaining in the |
| 289 current file.""" | |
| 290 | |
|
9862
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
291 lines = [] |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
292 while 1: |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
293 line = self.readline() |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
294 if line is None: |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
295 return lines |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
296 lines.append (line) |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
297 |
|
a7d1cf95dadc
Added: mems.lib.text_file: provides TextFile class for parsing text
Greg Ward <gward@python.net>
parents:
diff
changeset
|
298 |
|
10492
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
299 def unreadline (self, line): |
| 11342 | 300 """Push 'line' (a string) onto an internal buffer that will be |
| 301 checked by future 'readline()' calls. Handy for implementing | |
| 302 a parser with line-at-a-time lookahead.""" | |
| 303 | |
|
10492
43052d5b9808
Replaced the last attempt at an "unreadline" with one that actually
Greg Ward <gward@python.net>
parents:
10483
diff
changeset
|
304 self.linebuf.append (line) |
