Mercurial > cpython-lru_cache-weakref

annotate Lib/gzip.py @ 80661:94de77bd0b4b 2.7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #16194: document imp.load_dynamic problems when read several extension modules from the same binary object file.
author Andrew Svetlov <andrew.svetlov@gmail.com>
date Thu, 29 Nov 2012 19:51:28 +0200
parents 735ef27fa231
children 214d8909513d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11708
4a23cc49e2b0 Actually, the previous batch's comment should have been different;
Guido van Rossum <guido@python.org>
parents: 11706
diff changeset
1 """Functions that read and write gzipped files.
4a23cc49e2b0 Actually, the previous batch's comment should have been different;
Guido van Rossum <guido@python.org>
parents: 11706
diff changeset
2
11706
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 11692
diff changeset
3 The user of the file doesn't have to worry about the compression,
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 11692
diff changeset
4 but random access is not allowed."""
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 11692
diff changeset
5
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 11692
diff changeset
6 # based on Andrew Kuchling's minigzip.py distributed with the zlib module
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 11692
diff changeset
7
56960
b571c6685229 Issue #4750: Store the basename of the original filename in
Lars Gustäbel <lars@gustaebel.de>
parents: 53901
diff changeset
8 import struct, sys, time, os
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
9 import zlib
58137
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
10 import io
5872
dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates
Guido van Rossum <guido@python.org>
parents: 5386
diff changeset
11 import __builtin__
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
12
16160
46920b60cd33 added a few more __all__ lists
Skip Montanaro <skip@pobox.com>
parents: 15868
diff changeset
13 __all__ = ["GzipFile","open"]
46920b60cd33 added a few more __all__ lists
Skip Montanaro <skip@pobox.com>
parents: 15868
diff changeset
14
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
15 FTEXT, FHCRC, FEXTRA, FNAME, FCOMMENT = 1, 2, 4, 8, 16
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
16
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
17 READ, WRITE = 1, 2
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
18
10569
f58046339cbf Two different changes.
Guido van Rossum <guido@python.org>
parents: 10508
diff changeset
19 def write32u(output, value):
26019
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
20 # The L format writes the bit pattern correctly whether signed
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
21 # or unsigned.
10569
f58046339cbf Two different changes.
Guido van Rossum <guido@python.org>
parents: 10508
diff changeset
22 output.write(struct.pack("<L", value))
f58046339cbf Two different changes.
Guido van Rossum <guido@python.org>
parents: 10508
diff changeset
23
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
24 def read32(input):
46033
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to
Gregory P. Smith <greg@mad-scientist.com>
parents: 45933
diff changeset
25 return struct.unpack("<I", input.read(4))[0]
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
26
10508
d79989a435d9 Oops, missed mode parameter to open().
Fred Drake <fdrake@acm.org>
parents: 10507
diff changeset
27 def open(filename, mode="rb", compresslevel=9):
23565
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
28 """Shorthand for GzipFile(filename, mode, compresslevel).
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
29
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
30 The filename argument is required; mode defaults to 'rb'
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
31 and compresslevel defaults to 9.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
32
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
33 """
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
34 return GzipFile(filename, mode, compresslevel)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
35
58137
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
36 class GzipFile(io.BufferedIOBase):
23565
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
37 """The GzipFile class simulates most of the methods of a file object with
24601
262ad16b4de0 Remove mention of deprecated xreadlines method.
Guido van Rossum <guido@python.org>
parents: 23565
diff changeset
38 the exception of the readinto() and truncate() methods.
23565
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
39
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
40 """
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
41
5872
dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates
Guido van Rossum <guido@python.org>
parents: 5386
diff changeset
42 myfileobj = None
35050
e79793a8eeea [Bug #1074261, patch #1074381] Restrict the size of chunks read from the file in order to avoid overflow or huge memory consumption. Patch by Mark Eichin
Andrew M. Kuchling <amk@amk.ca>
parents: 34818
diff changeset
43 max_read_chunk = 10 * 1024 * 1024 # 10Mb
5872
dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates
Guido van Rossum <guido@python.org>
parents: 5386
diff changeset
44
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
45 def __init__(self, filename=None, mode=None,
50600
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
46 compresslevel=9, fileobj=None, mtime=None):
23565
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
47 """Constructor for the GzipFile class.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
48
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
49 At least one of fileobj and filename must be given a
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
50 non-trivial value.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
51
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
52 The new class instance is based on fileobj, which can be a regular
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
53 file, a StringIO object, or any other object which simulates a file.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
54 It defaults to None, in which case filename is opened to provide
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
55 a file object.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
56
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
57 When fileobj is not None, the filename argument is only used to be
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
58 included in the gzip file header, which may includes the original
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
59 filename of the uncompressed file. It defaults to the filename of
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
60 fileobj, if discernible; otherwise, it defaults to the empty string,
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
61 and in this case the original filename is not included in the header.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
62
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
63 The mode argument can be any of 'r', 'rb', 'a', 'ab', 'w', or 'wb',
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
64 depending on whether the file will be read or written. The default
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
65 is the mode of fileobj if discernible; otherwise, the default is 'rb'.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
66 Be aware that only the 'rb', 'ab', and 'wb' values should be used
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
67 for cross-platform portability.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
68
80384
735ef27fa231 Issue #15677: Document that zlib and gzip accept a compression level of 0 to mean 'no compression'.
Nadeem Vawda <nadeem.vawda@gmail.com>
parents: 80328
diff changeset
69 The compresslevel argument is an integer from 0 to 9 controlling the
23565
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
70 level of compression; 1 is fastest and produces the least compression,
80384
735ef27fa231 Issue #15677: Document that zlib and gzip accept a compression level of 0 to mean 'no compression'.
Nadeem Vawda <nadeem.vawda@gmail.com>
parents: 80328
diff changeset
71 and 9 is slowest and produces the most compression. 0 is no compression
735ef27fa231 Issue #15677: Document that zlib and gzip accept a compression level of 0 to mean 'no compression'.
Nadeem Vawda <nadeem.vawda@gmail.com>
parents: 80328
diff changeset
72 at all. The default is 9.
23565
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
73
50600
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
74 The mtime argument is an optional numeric timestamp to be written
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
75 to the stream when compressing. All gzip compressed streams
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
76 are required to contain a timestamp. If omitted or None, the
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
77 current time is used. This module ignores the timestamp when
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
78 decompressing; however, some programs, such as gunzip, make use
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
79 of it. The format of the timestamp is the same as that of the
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
80 return value of time.time() and of the st_mtime member of the
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
81 object returned by os.stat().
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
82
23565
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
83 """
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2).
Raymond Hettinger <python@rcn.com>
parents: 23518
diff changeset
84
79877
e647229c422b Issue #5148: Ignore 'U' in mode given to gzip.open() and gzip.GzipFile().
Nadeem Vawda <nadeem.vawda@gmail.com>
parents: 74514
diff changeset
85 # Make sure we don't inadvertently enable universal newlines on the
e647229c422b Issue #5148: Ignore 'U' in mode given to gzip.open() and gzip.GzipFile().
Nadeem Vawda <nadeem.vawda@gmail.com>
parents: 74514
diff changeset
86 # underlying file object - in read mode, this causes data corruption.
e647229c422b Issue #5148: Ignore 'U' in mode given to gzip.open() and gzip.GzipFile().
Nadeem Vawda <nadeem.vawda@gmail.com>
parents: 74514
diff changeset
87 if mode:
e647229c422b Issue #5148: Ignore 'U' in mode given to gzip.open() and gzip.GzipFile().
Nadeem Vawda <nadeem.vawda@gmail.com>
parents: 74514
diff changeset
88 mode = mode.replace('U', '')
23518
c03e8899d4b1 force gzip module to open files using 'b'inary mode.
Skip Montanaro <skip@pobox.com>
parents: 23075
diff changeset
89 # guarantee the file is opened in binary mode on platforms
c03e8899d4b1 force gzip module to open files using 'b'inary mode.
Skip Montanaro <skip@pobox.com>
parents: 23075
diff changeset
90 # that care about that sort of thing
c03e8899d4b1 force gzip module to open files using 'b'inary mode.
Skip Montanaro <skip@pobox.com>
parents: 23075
diff changeset
91 if mode and 'b' not in mode:
c03e8899d4b1 force gzip module to open files using 'b'inary mode.
Skip Montanaro <skip@pobox.com>
parents: 23075
diff changeset
92 mode += 'b'
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
93 if fileobj is None:
10507
753f3f95c50f Made the default mode 'rb' instead of 'r', for better cross-platform
Fred Drake <fdrake@acm.org>
parents: 10470
diff changeset
94 fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
5872
dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates
Guido van Rossum <guido@python.org>
parents: 5386
diff changeset
95 if filename is None:
74514
a08e9e84f33f Issue #13781: Fix GzipFile to work with os.fdopen()'d file objects.
Nadeem Vawda <nadeem.vawda@gmail.com>
parents: 68256
diff changeset
96 # Issue #13781: os.fdopen() creates a fileobj with a bogus name
a08e9e84f33f Issue #13781: Fix GzipFile to work with os.fdopen()'d file objects.
Nadeem Vawda <nadeem.vawda@gmail.com>
parents: 68256
diff changeset
97 # attribute. Avoid saving this in the gzip header's filename field.
a08e9e84f33f Issue #13781: Fix GzipFile to work with os.fdopen()'d file objects.
Nadeem Vawda <nadeem.vawda@gmail.com>
parents: 68256
diff changeset
98 if hasattr(fileobj, 'name') and fileobj.name != '<fdopen>':
a08e9e84f33f Issue #13781: Fix GzipFile to work with os.fdopen()'d file objects.
Nadeem Vawda <nadeem.vawda@gmail.com>
parents: 68256
diff changeset
99 filename = fileobj.name
a08e9e84f33f Issue #13781: Fix GzipFile to work with os.fdopen()'d file objects.
Nadeem Vawda <nadeem.vawda@gmail.com>
parents: 68256
diff changeset
100 else:
a08e9e84f33f Issue #13781: Fix GzipFile to work with os.fdopen()'d file objects.
Nadeem Vawda <nadeem.vawda@gmail.com>
parents: 68256
diff changeset
101 filename = ''
5872
dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates
Guido van Rossum <guido@python.org>
parents: 5386
diff changeset
102 if mode is None:
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
103 if hasattr(fileobj, 'mode'): mode = fileobj.mode
10507
753f3f95c50f Made the default mode 'rb' instead of 'r', for better cross-platform
Fred Drake <fdrake@acm.org>
parents: 10470
diff changeset
104 else: mode = 'rb'
5872
dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates
Guido van Rossum <guido@python.org>
parents: 5386
diff changeset
105
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
106 if mode[0:1] == 'r':
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
107 self.mode = READ
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
108 # Set flag indicating start of a new member
22867
e2f2187c3d2c Partial introduction of bools where appropriate.
Guido van Rossum <guido@python.org>
parents: 22417
diff changeset
109 self._new_member = True
58137
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
110 # Buffer data read from gzip file. extrastart is offset in
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
111 # stream where buffer starts. extrasize is number of
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
112 # bytes remaining in buffer from current stream position.
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
113 self.extrabuf = ""
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
114 self.extrasize = 0
58137
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
115 self.extrastart = 0
40839
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The
Lars Gustäbel <lars@gustaebel.de>
parents: 40334
diff changeset
116 self.name = filename
38237
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
117 # Starts small, scales exponentially
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
118 self.min_readsize = 100
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
119
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
120 elif mode[0:1] == 'w' or mode[0:1] == 'a':
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
121 self.mode = WRITE
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
122 self._init_write(filename)
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
123 self.compress = zlib.compressobj(compresslevel,
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
124 zlib.DEFLATED,
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
125 -zlib.MAX_WBITS,
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
126 zlib.DEF_MEM_LEVEL,
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
127 0)
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
128 else:
22216
64c731d180e9 Patch #443899: Check modes on files before performing operations.
Martin v. Löwis <martin@v.loewis.de>
parents: 20418
diff changeset
129 raise IOError, "Mode " + mode + " not supported"
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
130
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
131 self.fileobj = fileobj
19088
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
132 self.offset = 0
50600
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
133 self.mtime = mtime
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
134
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
135 if self.mode == WRITE:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
136 self._write_gzip_header()
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
137
40839
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The
Lars Gustäbel <lars@gustaebel.de>
parents: 40334
diff changeset
138 @property
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The
Lars Gustäbel <lars@gustaebel.de>
parents: 40334
diff changeset
139 def filename(self):
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The
Lars Gustäbel <lars@gustaebel.de>
parents: 40334
diff changeset
140 import warnings
53903
a36fa2ce61b1 Merged revisions 72458 via svnmerge from
Philip Jenvey <pjenvey@underboss.org>
parents: 47256
diff changeset
141 warnings.warn("use the name attribute", DeprecationWarning, 2)
40839
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The
Lars Gustäbel <lars@gustaebel.de>
parents: 40334
diff changeset
142 if self.mode == WRITE and self.name[-3:] != ".gz":
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The
Lars Gustäbel <lars@gustaebel.de>
parents: 40334
diff changeset
143 return self.name + ".gz"
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The
Lars Gustäbel <lars@gustaebel.de>
parents: 40334
diff changeset
144 return self.name
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The
Lars Gustäbel <lars@gustaebel.de>
parents: 40334
diff changeset
145
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
146 def __repr__(self):
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
147 s = repr(self.fileobj)
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
148 return '<gzip ' + s[1:-1] + ' ' + hex(id(self)) + '>'
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
149
65152
32dcccdc73f8 Merged revisions 85291 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61220
diff changeset
150 def _check_closed(self):
32dcccdc73f8 Merged revisions 85291 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61220
diff changeset
151 """Raises a ValueError if the underlying file object has been closed.
32dcccdc73f8 Merged revisions 85291 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61220
diff changeset
152
32dcccdc73f8 Merged revisions 85291 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61220
diff changeset
153 """
32dcccdc73f8 Merged revisions 85291 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61220
diff changeset
154 if self.closed:
32dcccdc73f8 Merged revisions 85291 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61220
diff changeset
155 raise ValueError('I/O operation on closed file.')
32dcccdc73f8 Merged revisions 85291 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61220
diff changeset
156
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
157 def _init_write(self, filename):
40839
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The
Lars Gustäbel <lars@gustaebel.de>
parents: 40334
diff changeset
158 self.name = filename
46033
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to
Gregory P. Smith <greg@mad-scientist.com>
parents: 45933
diff changeset
159 self.crc = zlib.crc32("") & 0xffffffffL
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
160 self.size = 0
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
161 self.writebuf = []
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
162 self.bufsize = 0
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
163
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
164 def _write_gzip_header(self):
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
165 self.fileobj.write('\037\213') # magic header
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
166 self.fileobj.write('\010') # compression method
56960
b571c6685229 Issue #4750: Store the basename of the original filename in
Lars Gustäbel <lars@gustaebel.de>
parents: 53901
diff changeset
167 fname = os.path.basename(self.name)
40840
d4768bff643e Strip the '.gz' extension from the filename that is written to the
Lars Gustäbel <lars@gustaebel.de>
parents: 40839
diff changeset
168 if fname.endswith(".gz"):
d4768bff643e Strip the '.gz' extension from the filename that is written to the
Lars Gustäbel <lars@gustaebel.de>
parents: 40839
diff changeset
169 fname = fname[:-3]
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
170 flags = 0
40840
d4768bff643e Strip the '.gz' extension from the filename that is written to the
Lars Gustäbel <lars@gustaebel.de>
parents: 40839
diff changeset
171 if fname:
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
172 flags = FNAME
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
173 self.fileobj.write(chr(flags))
50600
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
174 mtime = self.mtime
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
175 if mtime is None:
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
176 mtime = time.time()
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
177 write32u(self.fileobj, long(mtime))
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
178 self.fileobj.write('\002')
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
179 self.fileobj.write('\377')
40840
d4768bff643e Strip the '.gz' extension from the filename that is written to the
Lars Gustäbel <lars@gustaebel.de>
parents: 40839
diff changeset
180 if fname:
d4768bff643e Strip the '.gz' extension from the filename that is written to the
Lars Gustäbel <lars@gustaebel.de>
parents: 40839
diff changeset
181 self.fileobj.write(fname + '\000')
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
182
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
183 def _init_read(self):
46033
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to
Gregory P. Smith <greg@mad-scientist.com>
parents: 45933
diff changeset
184 self.crc = zlib.crc32("") & 0xffffffffL
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
185 self.size = 0
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
186
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
187 def _read_gzip_header(self):
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
188 magic = self.fileobj.read(2)
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
189 if magic != '\037\213':
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
190 raise IOError, 'Not a gzipped file'
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
191 method = ord( self.fileobj.read(1) )
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
192 if method != 8:
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
193 raise IOError, 'Unknown compression method'
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
194 flag = ord( self.fileobj.read(1) )
50600
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
195 self.mtime = read32(self.fileobj)
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
196 # extraflag = self.fileobj.read(1)
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
197 # os = self.fileobj.read(1)
50600
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream.
Antoine Pitrou <solipsis@pitrou.net>
parents: 47256
diff changeset
198 self.fileobj.read(2)
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
199
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
200 if flag & FEXTRA:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
201 # Read & discard the extra field, if present
26019
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
202 xlen = ord(self.fileobj.read(1))
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
203 xlen = xlen + 256*ord(self.fileobj.read(1))
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
204 self.fileobj.read(xlen)
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
205 if flag & FNAME:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
206 # Read and discard a null-terminated string containing the filename
22867
e2f2187c3d2c Partial introduction of bools where appropriate.
Guido van Rossum <guido@python.org>
parents: 22417
diff changeset
207 while True:
26019
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
208 s = self.fileobj.read(1)
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
209 if not s or s=='\000':
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
210 break
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
211 if flag & FCOMMENT:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
212 # Read and discard a null-terminated string containing a comment
22867
e2f2187c3d2c Partial introduction of bools where appropriate.
Guido van Rossum <guido@python.org>
parents: 22417
diff changeset
213 while True:
26019
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
214 s = self.fileobj.read(1)
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
215 if not s or s=='\000':
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
216 break
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
217 if flag & FHCRC:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
218 self.fileobj.read(2) # Read & discard the 16-bit header CRC
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
219
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
220 def write(self,data):
65152
32dcccdc73f8 Merged revisions 85291 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61220
diff changeset
221 self._check_closed()
22216
64c731d180e9 Patch #443899: Check modes on files before performing operations.
Martin v. Löwis <martin@v.loewis.de>
parents: 20418
diff changeset
222 if self.mode != WRITE:
64c731d180e9 Patch #443899: Check modes on files before performing operations.
Martin v. Löwis <martin@v.loewis.de>
parents: 20418
diff changeset
223 import errno
64c731d180e9 Patch #443899: Check modes on files before performing operations.
Martin v. Löwis <martin@v.loewis.de>
parents: 20418
diff changeset
224 raise IOError(errno.EBADF, "write() on read-only GzipFile object")
23075
b7e87732f6ab Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 22867
diff changeset
225
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
226 if self.fileobj is None:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
227 raise ValueError, "write() on closed GzipFile object"
58137
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
228
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
229 # Convert data type if called by io.BufferedWriter.
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
230 if isinstance(data, memoryview):
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
231 data = data.tobytes()
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
232
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
233 if len(data) > 0:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
234 self.size = self.size + len(data)
46033
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to
Gregory P. Smith <greg@mad-scientist.com>
parents: 45933
diff changeset
235 self.crc = zlib.crc32(data, self.crc) & 0xffffffffL
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
236 self.fileobj.write( self.compress.compress(data) )
19088
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
237 self.offset += len(data)
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
238
58137
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
239 return len(data)
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
240
11692
0dd520f4dce8 Make read() and readlines() conform more to the file object interface:
Guido van Rossum <guido@python.org>
parents: 11255
diff changeset
241 def read(self, size=-1):
65152
32dcccdc73f8 Merged revisions 85291 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61220
diff changeset
242 self._check_closed()
22216
64c731d180e9 Patch #443899: Check modes on files before performing operations.
Martin v. Löwis <martin@v.loewis.de>
parents: 20418
diff changeset
243 if self.mode != READ:
64c731d180e9 Patch #443899: Check modes on files before performing operations.
Martin v. Löwis <martin@v.loewis.de>
parents: 20418
diff changeset
244 import errno
31096
d35f8799cf12 Fix error in exception message.
Brett Cannon <bcannon@gmail.com>
parents: 27705
diff changeset
245 raise IOError(errno.EBADF, "read() on write-only GzipFile object")
23075
b7e87732f6ab Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 22867
diff changeset
246
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
247 if self.extrasize <= 0 and self.fileobj is None:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
248 return ''
7169
373d78118113 (This fix is really by Jeremy)
Guido van Rossum <guido@python.org>
parents: 7006
diff changeset
249
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
250 readsize = 1024
11692
0dd520f4dce8 Make read() and readlines() conform more to the file object interface:
Guido van Rossum <guido@python.org>
parents: 11255
diff changeset
251 if size < 0: # get the whole thing
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
252 try:
22867
e2f2187c3d2c Partial introduction of bools where appropriate.
Guido van Rossum <guido@python.org>
parents: 22417
diff changeset
253 while True:
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
254 self._read(readsize)
35050
e79793a8eeea [Bug #1074261, patch #1074381] Restrict the size of chunks read from the file in order to avoid overflow or huge memory consumption. Patch by Mark Eichin
Andrew M. Kuchling <amk@amk.ca>
parents: 34818
diff changeset
255 readsize = min(self.max_read_chunk, readsize * 2)
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
256 except EOFError:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
257 size = self.extrasize
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
258 else: # just get some more of it
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
259 try:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
260 while size > self.extrasize:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
261 self._read(readsize)
35050
e79793a8eeea [Bug #1074261, patch #1074381] Restrict the size of chunks read from the file in order to avoid overflow or huge memory consumption. Patch by Mark Eichin
Andrew M. Kuchling <amk@amk.ca>
parents: 34818
diff changeset
262 readsize = min(self.max_read_chunk, readsize * 2)
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
263 except EOFError:
8804
5d5a5bc8de5c Patch by Ron Klatchko: fix invariant in _unread(). Also fixed
Guido van Rossum <guido@python.org>
parents: 8324
diff changeset
264 if size > self.extrasize:
5d5a5bc8de5c Patch by Ron Klatchko: fix invariant in _unread(). Also fixed
Guido van Rossum <guido@python.org>
parents: 8324
diff changeset
265 size = self.extrasize
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
266
58137
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
267 offset = self.offset - self.extrastart
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
268 chunk = self.extrabuf[offset: offset + size]
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
269 self.extrasize = self.extrasize - size
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
270
19088
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
271 self.offset += size
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
272 return chunk
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
273
7169
373d78118113 (This fix is really by Jeremy)
Guido van Rossum <guido@python.org>
parents: 7006
diff changeset
274 def _unread(self, buf):
8804
5d5a5bc8de5c Patch by Ron Klatchko: fix invariant in _unread(). Also fixed
Guido van Rossum <guido@python.org>
parents: 8324
diff changeset
275 self.extrasize = len(buf) + self.extrasize
19088
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
276 self.offset -= len(buf)
7169
373d78118113 (This fix is really by Jeremy)
Guido van Rossum <guido@python.org>
parents: 7006
diff changeset
277
373d78118113 (This fix is really by Jeremy)
Guido van Rossum <guido@python.org>
parents: 7006
diff changeset
278 def _read(self, size=1024):
26019
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
279 if self.fileobj is None:
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
280 raise EOFError, "Reached EOF"
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
281
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
282 if self._new_member:
13743
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and
Andrew M. Kuchling <amk@amk.ca>
parents: 12619
diff changeset
283 # If the _new_member flag is set, we have to
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and
Andrew M. Kuchling <amk@amk.ca>
parents: 12619
diff changeset
284 # jump to the next member, if there is one.
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
285 #
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
286 # First, check if we're at the end of the file;
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
287 # if so, it's time to stop; no more members to read.
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
288 pos = self.fileobj.tell() # Save current position
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
289 self.fileobj.seek(0, 2) # Seek to end of file
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
290 if pos == self.fileobj.tell():
11255
427c3b50058e Fixed 'return EOFError' that should be 'raise EOFError', caught by
Andrew M. Kuchling <amk@amk.ca>
parents: 11184
diff changeset
291 raise EOFError, "Reached EOF"
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
292 else:
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
293 self.fileobj.seek( pos ) # Return to original position
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
294
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
295 self._init_read()
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
296 self._read_gzip_header()
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
297 self.decompress = zlib.decompressobj(-zlib.MAX_WBITS)
22867
e2f2187c3d2c Partial introduction of bools where appropriate.
Guido van Rossum <guido@python.org>
parents: 22417
diff changeset
298 self._new_member = False
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
299
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
300 # Read a chunk of data from the file
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
301 buf = self.fileobj.read(size)
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
302
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
303 # If the EOF has been reached, flush the decompression object
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
304 # and mark this object as finished.
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
305
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
306 if buf == "":
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
307 uncompress = self.decompress.flush()
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
308 self._read_eof()
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
309 self._add_read_data( uncompress )
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
310 raise EOFError, 'Reached EOF'
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
311
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
312 uncompress = self.decompress.decompress(buf)
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
313 self._add_read_data( uncompress )
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
314
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
315 if self.decompress.unused_data != "":
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
316 # Ending case: we've come to the end of a member in the file,
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
317 # so seek back to the start of the unused data, finish up
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
318 # this member, and read a new gzip header.
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
319 # (The number of bytes to seek back is the length of the unused
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
320 # data, minus 8 because _read_eof() will rewind a further 8 bytes)
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
321 self.fileobj.seek( -len(self.decompress.unused_data)+8, 1)
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
322
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
323 # Check the CRC and file size, and set the flag so we read
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
324 # a new member on the next call
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
325 self._read_eof()
22867
e2f2187c3d2c Partial introduction of bools where appropriate.
Guido van Rossum <guido@python.org>
parents: 22417
diff changeset
326 self._new_member = True
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
327
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
328 def _add_read_data(self, data):
46033
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to
Gregory P. Smith <greg@mad-scientist.com>
parents: 45933
diff changeset
329 self.crc = zlib.crc32(data, self.crc) & 0xffffffffL
58137
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
330 offset = self.offset - self.extrastart
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
331 self.extrabuf = self.extrabuf[offset:] + data
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
332 self.extrasize = self.extrasize + len(data)
58137
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
333 self.extrastart = self.offset
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
334 self.size = self.size + len(data)
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
335
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
336 def _read_eof(self):
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
337 # We've read to the end of the file, so we have to rewind in order
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
338 # to reread the 8 bytes containing the CRC and the file size.
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
339 # We check the that the computed CRC and size of the
26050
c54716aed26a Another round on SF patch 618135: gzip.py and files > 2G
Tim Peters <tim.peters@gmail.com>
parents: 26019
diff changeset
340 # uncompressed data matches the stored values. Note that the size
c54716aed26a Another round on SF patch 618135: gzip.py and files > 2G
Tim Peters <tim.peters@gmail.com>
parents: 26019
diff changeset
341 # stored is the true file size mod 2**32.
10470
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to
Andrew M. Kuchling <amk@amk.ca>
parents: 10450
diff changeset
342 self.fileobj.seek(-8, 1)
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
343 crc32 = read32(self.fileobj)
46033
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to
Gregory P. Smith <greg@mad-scientist.com>
parents: 45933
diff changeset
344 isize = read32(self.fileobj) # may exceed 2GB
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to
Gregory P. Smith <greg@mad-scientist.com>
parents: 45933
diff changeset
345 if crc32 != self.crc:
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to
Gregory P. Smith <greg@mad-scientist.com>
parents: 45933
diff changeset
346 raise IOError("CRC check failed %s != %s" % (hex(crc32),
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to
Gregory P. Smith <greg@mad-scientist.com>
parents: 45933
diff changeset
347 hex(self.crc)))
46036
f5b1fa1214f0 A bugfix for r61813, it would fail if the data size was >=2**32.
Gregory P. Smith <greg@mad-scientist.com>
parents: 46033
diff changeset
348 elif isize != (self.size & 0xffffffffL):
27705
0f6731d126de [Patch #654421 from Matthew Mueller]
Andrew M. Kuchling <amk@amk.ca>
parents: 26050
diff changeset
349 raise IOError, "Incorrect length of data produced"
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
350
58299
b21efc0cfe8b Issue #2846: Add support for gzip.GzipFile reading zero-padded files.
Antoine Pitrou <solipsis@pitrou.net>
parents: 58137
diff changeset
351 # Gzip files can be padded with zeroes and still have archives.
b21efc0cfe8b Issue #2846: Add support for gzip.GzipFile reading zero-padded files.
Antoine Pitrou <solipsis@pitrou.net>
parents: 58137
diff changeset
352 # Consume all zero bytes and set the file position to the first
b21efc0cfe8b Issue #2846: Add support for gzip.GzipFile reading zero-padded files.
Antoine Pitrou <solipsis@pitrou.net>
parents: 58137
diff changeset
353 # non-zero byte. See http://www.gzip.org/#faq8
b21efc0cfe8b Issue #2846: Add support for gzip.GzipFile reading zero-padded files.
Antoine Pitrou <solipsis@pitrou.net>
parents: 58137
diff changeset
354 c = "\x00"
b21efc0cfe8b Issue #2846: Add support for gzip.GzipFile reading zero-padded files.
Antoine Pitrou <solipsis@pitrou.net>
parents: 58137
diff changeset
355 while c == "\x00":
b21efc0cfe8b Issue #2846: Add support for gzip.GzipFile reading zero-padded files.
Antoine Pitrou <solipsis@pitrou.net>
parents: 58137
diff changeset
356 c = self.fileobj.read(1)
b21efc0cfe8b Issue #2846: Add support for gzip.GzipFile reading zero-padded files.
Antoine Pitrou <solipsis@pitrou.net>
parents: 58137
diff changeset
357 if c:
b21efc0cfe8b Issue #2846: Add support for gzip.GzipFile reading zero-padded files.
Antoine Pitrou <solipsis@pitrou.net>
parents: 58137
diff changeset
358 self.fileobj.seek(-1, 1)
b21efc0cfe8b Issue #2846: Add support for gzip.GzipFile reading zero-padded files.
Antoine Pitrou <solipsis@pitrou.net>
parents: 58137
diff changeset
359
58137
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
360 @property
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
361 def closed(self):
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
362 return self.fileobj is None
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
363
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
364 def close(self):
47256
d5591c16b997 #2959: allow multiple close() calls for GzipFile.
Georg Brandl <georg@python.org>
parents: 46037
diff changeset
365 if self.fileobj is None:
d5591c16b997 #2959: allow multiple close() calls for GzipFile.
Georg Brandl <georg@python.org>
parents: 46037
diff changeset
366 return
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
367 if self.mode == WRITE:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
368 self.fileobj.write(self.compress.flush())
46033
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to
Gregory P. Smith <greg@mad-scientist.com>
parents: 45933
diff changeset
369 write32u(self.fileobj, self.crc)
26050
c54716aed26a Another round on SF patch 618135: gzip.py and files > 2G
Tim Peters <tim.peters@gmail.com>
parents: 26019
diff changeset
370 # self.size may exceed 2GB, or even 4GB
46037
7caaab04a4d8 prevent a warning from the struct module when data size >= 2**32.
Gregory P. Smith <greg@mad-scientist.com>
parents: 46036
diff changeset
371 write32u(self.fileobj, self.size & 0xffffffffL)
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
372 self.fileobj = None
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
373 elif self.mode == READ:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
374 self.fileobj = None
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
375 if self.myfileobj:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
376 self.myfileobj.close()
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
377 self.myfileobj = None
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
378
34650
11fcdbb279de Patch #1110248: SYNC_FLUSH the zlib buffer for GZipFile.flush.
Martin v. Löwis <martin@v.loewis.de>
parents: 32835
diff changeset
379 def flush(self,zlib_mode=zlib.Z_SYNC_FLUSH):
65152
32dcccdc73f8 Merged revisions 85291 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61220
diff changeset
380 self._check_closed()
34650
11fcdbb279de Patch #1110248: SYNC_FLUSH the zlib buffer for GZipFile.flush.
Martin v. Löwis <martin@v.loewis.de>
parents: 32835
diff changeset
381 if self.mode == WRITE:
34818
4eaab38566c9 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 34650
diff changeset
382 # Ensure the compressor's buffer is flushed
4eaab38566c9 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 34650
diff changeset
383 self.fileobj.write(self.compress.flush(zlib_mode))
61221
adc2ebda7636 Merged revisions 80762 via svnmerge from
Mark Dickinson <dickinsm@gmail.com>
parents: 53903
diff changeset
384 self.fileobj.flush()
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
385
32834
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can
Tim Peters <tim.peters@gmail.com>
parents: 31515
diff changeset
386 def fileno(self):
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can
Tim Peters <tim.peters@gmail.com>
parents: 31515
diff changeset
387 """Invoke the underlying file object's fileno() method.
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can
Tim Peters <tim.peters@gmail.com>
parents: 31515
diff changeset
388
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can
Tim Peters <tim.peters@gmail.com>
parents: 31515
diff changeset
389 This will raise AttributeError if the underlying file object
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can
Tim Peters <tim.peters@gmail.com>
parents: 31515
diff changeset
390 doesn't support fileno().
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can
Tim Peters <tim.peters@gmail.com>
parents: 31515
diff changeset
391 """
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can
Tim Peters <tim.peters@gmail.com>
parents: 31515
diff changeset
392 return self.fileobj.fileno()
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can
Tim Peters <tim.peters@gmail.com>
parents: 31515
diff changeset
393
19088
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
394 def rewind(self):
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
395 '''Return the uncompressed stream file position indicator to the
19119
600a9a2b7a18 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 19088
diff changeset
396 beginning of the file'''
19088
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
397 if self.mode != READ:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
398 raise IOError("Can't rewind in write mode")
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
399 self.fileobj.seek(0)
22867
e2f2187c3d2c Partial introduction of bools where appropriate.
Guido van Rossum <guido@python.org>
parents: 22417
diff changeset
400 self._new_member = True
19088
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
401 self.extrabuf = ""
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
402 self.extrasize = 0
58137
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
403 self.extrastart = 0
19088
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
404 self.offset = 0
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
405
58137
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
406 def readable(self):
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
407 return self.mode == READ
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
408
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
409 def writable(self):
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
410 return self.mode == WRITE
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
411
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
412 def seekable(self):
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
413 return True
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
414
40334
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek.
Martin v. Löwis <martin@v.loewis.de>
parents: 39347
diff changeset
415 def seek(self, offset, whence=0):
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek.
Martin v. Löwis <martin@v.loewis.de>
parents: 39347
diff changeset
416 if whence:
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek.
Martin v. Löwis <martin@v.loewis.de>
parents: 39347
diff changeset
417 if whence == 1:
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek.
Martin v. Löwis <martin@v.loewis.de>
parents: 39347
diff changeset
418 offset = self.offset + offset
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek.
Martin v. Löwis <martin@v.loewis.de>
parents: 39347
diff changeset
419 else:
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek.
Martin v. Löwis <martin@v.loewis.de>
parents: 39347
diff changeset
420 raise ValueError('Seek from end not supported')
19088
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
421 if self.mode == WRITE:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
422 if offset < self.offset:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
423 raise IOError('Negative seek in write mode')
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
424 count = offset - self.offset
80328
f938d478359a Bug #16441: avoid excessive memory usage working with large gzip files
Chris Withers <chris@simplistix.co.uk>
parents: 79877
diff changeset
425 for i in xrange(count // 1024):
26019
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
426 self.write(1024 * '\0')
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
427 self.write((count % 1024) * '\0')
19088
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
428 elif self.mode == READ:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
429 if offset < self.offset:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
430 # for negative seek, rewind and do positive seek
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
431 self.rewind()
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
432 count = offset - self.offset
80328
f938d478359a Bug #16441: avoid excessive memory usage working with large gzip files
Chris Withers <chris@simplistix.co.uk>
parents: 79877
diff changeset
433 for i in xrange(count // 1024):
26019
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G.
Tim Peters <tim.peters@gmail.com>
parents: 24601
diff changeset
434 self.read(1024)
19088
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
435 self.read(count % 1024)
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile.
Martin v. Löwis <martin@v.loewis.de>
parents: 17083
diff changeset
436
58137
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
437 return self.offset
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
438
13743
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and
Andrew M. Kuchling <amk@amk.ca>
parents: 12619
diff changeset
439 def readline(self, size=-1):
38237
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
440 if size < 0:
58137
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
441 # Shortcut common case - newline found in buffer.
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
442 offset = self.offset - self.extrastart
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
443 i = self.extrabuf.find('\n', offset) + 1
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
444 if i > 0:
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
445 self.extrasize -= i - offset
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
446 self.offset += i - offset
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
447 return self.extrabuf[offset: i]
c857b56699b5 Issue #7471: Improve the performance of GzipFile's buffering mechanism,
Antoine Pitrou <solipsis@pitrou.net>
parents: 56960
diff changeset
448
38237
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
449 size = sys.maxint
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
450 readsize = self.min_readsize
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
451 else:
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
452 readsize = size
38235
fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found
Bob Ippolito <bob@redivi.com>
parents: 38234
diff changeset
453 bufs = []
38237
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
454 while size != 0:
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
455 c = self.read(readsize)
16496
1ff9d571bac0 String method conversion.
Eric S. Raymond <esr@thyrsus.com>
parents: 16160
diff changeset
456 i = c.find('\n')
38237
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
457
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
458 # We set i=size to break out of the loop under two
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
459 # conditions: 1) there's no newline, and the chunk is
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
460 # larger than size, or 2) there is a newline, but the
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
461 # resulting line would be longer than 'size'.
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
462 if (size <= i) or (i == -1 and len(c) > size):
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
463 i = size - 1
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
464
38235
fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found
Bob Ippolito <bob@redivi.com>
parents: 38234
diff changeset
465 if i >= 0 or c == '':
38237
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
466 bufs.append(c[:i + 1]) # Add portion of last chunk
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
467 self._unread(c[i + 1:]) # Push back rest of chunk
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
468 break
38234
27c80058173e GzipFile.readline performance improvement (~30-40%), patch #1281707
Bob Ippolito <bob@redivi.com>
parents: 35050
diff changeset
469
38235
fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found
Bob Ippolito <bob@redivi.com>
parents: 38234
diff changeset
470 # Append chunk to list, decrease 'size',
fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found
Bob Ippolito <bob@redivi.com>
parents: 38234
diff changeset
471 bufs.append(c)
fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found
Bob Ippolito <bob@redivi.com>
parents: 38234
diff changeset
472 size = size - len(c)
fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found
Bob Ippolito <bob@redivi.com>
parents: 38234
diff changeset
473 readsize = min(size, readsize * 2)
38237
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
474 if readsize > self.min_readsize:
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
475 self.min_readsize = min(readsize, self.min_readsize * 2, 512)
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707
Bob Ippolito <bob@redivi.com>
parents: 38235
diff changeset
476 return ''.join(bufs) # Return resulting line
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 13743
diff changeset
477
7006
9773d3941c96 When there's no filename, don't make one up.
Guido van Rossum <guido@python.org>
parents: 5872
diff changeset
478
9773d3941c96 When there's no filename, don't make one up.
Guido van Rossum <guido@python.org>
parents: 5872
diff changeset
479 def _test():
9773d3941c96 When there's no filename, don't make one up.
Guido van Rossum <guido@python.org>
parents: 5872
diff changeset
480 # Act like gzip; with -d, act like gunzip.
9773d3941c96 When there's no filename, don't make one up.
Guido van Rossum <guido@python.org>
parents: 5872
diff changeset
481 # The input file is not deleted, however, nor are any other gzip
9773d3941c96 When there's no filename, don't make one up.
Guido van Rossum <guido@python.org>
parents: 5872
diff changeset
482 # options or features supported.
9773d3941c96 When there's no filename, don't make one up.
Guido van Rossum <guido@python.org>
parents: 5872
diff changeset
483 args = sys.argv[1:]
9773d3941c96 When there's no filename, don't make one up.
Guido van Rossum <guido@python.org>
parents: 5872
diff changeset
484 decompress = args and args[0] == "-d"
9773d3941c96 When there's no filename, don't make one up.
Guido van Rossum <guido@python.org>
parents: 5872
diff changeset
485 if decompress:
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
486 args = args[1:]
7006
9773d3941c96 When there's no filename, don't make one up.
Guido van Rossum <guido@python.org>
parents: 5872
diff changeset
487 if not args:
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
488 args = ["-"]
7006
9773d3941c96 When there's no filename, don't make one up.
Guido van Rossum <guido@python.org>
parents: 5872
diff changeset
489 for arg in args:
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
490 if decompress:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
491 if arg == "-":
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
492 f = GzipFile(filename="", mode="rb", fileobj=sys.stdin)
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
493 g = sys.stdout
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
494 else:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
495 if arg[-3:] != ".gz":
31515
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 31096
diff changeset
496 print "filename doesn't end in .gz:", repr(arg)
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
497 continue
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
498 f = open(arg, "rb")
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
499 g = __builtin__.open(arg[:-3], "wb")
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
500 else:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
501 if arg == "-":
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
502 f = sys.stdin
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
503 g = GzipFile(filename="", mode="wb", fileobj=sys.stdout)
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
504 else:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
505 f = __builtin__.open(arg, "rb")
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
506 g = open(arg + ".gz", "wb")
22867
e2f2187c3d2c Partial introduction of bools where appropriate.
Guido van Rossum <guido@python.org>
parents: 22417
diff changeset
507 while True:
7755
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
508 chunk = f.read(1024)
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
509 if not chunk:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
510 break
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
511 g.write(chunk)
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
512 if g is not sys.stdout:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
513 g.close()
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
514 if f is not sys.stdin:
fb4b4d069cc7 Mass check-in after untabifying all files that need it.
Guido van Rossum <guido@python.org>
parents: 7169
diff changeset
515 f.close()
7006
9773d3941c96 When there's no filename, don't make one up.
Guido van Rossum <guido@python.org>
parents: 5872
diff changeset
516
9773d3941c96 When there's no filename, don't make one up.
Guido van Rossum <guido@python.org>
parents: 5872
diff changeset
517 if __name__ == '__main__':
9773d3941c96 When there's no filename, don't make one up.
Guido van Rossum <guido@python.org>
parents: 5872
diff changeset
518 _test()