cpython-lru_cache-weakref: Lib/gzip.py annotate

annotate Lib/gzip.py @ 58925:eecf1bef052d 2.6

Merged revisions 78207 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r78207 | ezio.melotti | 2010-02-17 01:26:09 +0200 (Wed, 17 Feb 2010) | 1 line #7930: fix stripid ........

author	Ezio Melotti <ezio.melotti@gmail.com>
date	Tue, 16 Feb 2010 23:29:44 +0000
parents	a36fa2ce61b1
children	adc2ebda7636

rev	line source
11708 4a23cc49e2b0 Actually, the previous batch's comment should have been different; Guido van Rossum <guido@python.org> parents: 11706 diff changeset	1 """Functions that read and write gzipped files.
4a23cc49e2b0 Actually, the previous batch's comment should have been different; Guido van Rossum <guido@python.org> parents: 11706 diff changeset	2
11706 b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee, Guido van Rossum <guido@python.org> parents: 11692 diff changeset	3 The user of the file doesn't have to worry about the compression,
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee, Guido van Rossum <guido@python.org> parents: 11692 diff changeset	4 but random access is not allowed."""
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee, Guido van Rossum <guido@python.org> parents: 11692 diff changeset	5
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee, Guido van Rossum <guido@python.org> parents: 11692 diff changeset	6 # based on Andrew Kuchling's minigzip.py distributed with the zlib module
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee, Guido van Rossum <guido@python.org> parents: 11692 diff changeset	7
32835 b58cf6b5fc91 Ack, removed useless import of os I just introduced. Tim Peters <tim.peters@gmail.com> parents: 32834 diff changeset	8 import struct, sys, time
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	9 import zlib
5872 dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates Guido van Rossum <guido@python.org> parents: 5386 diff changeset	10 import __builtin__
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	11
16160 46920b60cd33 added a few more __all__ lists Skip Montanaro <skip@pobox.com> parents: 15868 diff changeset	12 __all__ = ["GzipFile","open"]
46920b60cd33 added a few more __all__ lists Skip Montanaro <skip@pobox.com> parents: 15868 diff changeset	13
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	14 FTEXT, FHCRC, FEXTRA, FNAME, FCOMMENT = 1, 2, 4, 8, 16
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	15
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	16 READ, WRITE = 1, 2
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	17
10569 f58046339cbf Two different changes. Guido van Rossum <guido@python.org> parents: 10508 diff changeset	18 def write32u(output, value):
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	19 # The L format writes the bit pattern correctly whether signed
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	20 # or unsigned.
10569 f58046339cbf Two different changes. Guido van Rossum <guido@python.org> parents: 10508 diff changeset	21 output.write(struct.pack("<L", value))
f58046339cbf Two different changes. Guido van Rossum <guido@python.org> parents: 10508 diff changeset	22
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	23 def read32(input):
46033 467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	24 return struct.unpack("<I", input.read(4))[0]
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	25
10508 d79989a435d9 Oops, missed mode parameter to open(). Fred Drake <fdrake@acm.org> parents: 10507 diff changeset	26 def open(filename, mode="rb", compresslevel=9):
23565 1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	27 """Shorthand for GzipFile(filename, mode, compresslevel).
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	28
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	29 The filename argument is required; mode defaults to 'rb'
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	30 and compresslevel defaults to 9.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	31
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	32 """
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	33 return GzipFile(filename, mode, compresslevel)
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	34
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	35 class GzipFile:
23565 1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	36 """The GzipFile class simulates most of the methods of a file object with
24601 262ad16b4de0 Remove mention of deprecated xreadlines method. Guido van Rossum <guido@python.org> parents: 23565 diff changeset	37 the exception of the readinto() and truncate() methods.
23565 1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	38
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	39 """
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	40
5872 dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates Guido van Rossum <guido@python.org> parents: 5386 diff changeset	41 myfileobj = None
35050 e79793a8eeea [Bug #1074261, patch #1074381] Restrict the size of chunks read from the file in order to avoid overflow or huge memory consumption. Patch by Mark Eichin Andrew M. Kuchling <amk@amk.ca> parents: 34818 diff changeset	42 max_read_chunk = 10 * 1024 * 1024 # 10Mb
5872 dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates Guido van Rossum <guido@python.org> parents: 5386 diff changeset	43
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	44 def __init__(self, filename=None, mode=None,
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	45 compresslevel=9, fileobj=None):
23565 1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	46 """Constructor for the GzipFile class.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	47
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	48 At least one of fileobj and filename must be given a
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	49 non-trivial value.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	50
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	51 The new class instance is based on fileobj, which can be a regular
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	52 file, a StringIO object, or any other object which simulates a file.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	53 It defaults to None, in which case filename is opened to provide
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	54 a file object.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	55
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	56 When fileobj is not None, the filename argument is only used to be
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	57 included in the gzip file header, which may includes the original
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	58 filename of the uncompressed file. It defaults to the filename of
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	59 fileobj, if discernible; otherwise, it defaults to the empty string,
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	60 and in this case the original filename is not included in the header.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	61
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	62 The mode argument can be any of 'r', 'rb', 'a', 'ab', 'w', or 'wb',
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	63 depending on whether the file will be read or written. The default
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	64 is the mode of fileobj if discernible; otherwise, the default is 'rb'.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	65 Be aware that only the 'rb', 'ab', and 'wb' values should be used
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	66 for cross-platform portability.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	67
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	68 The compresslevel argument is an integer from 1 to 9 controlling the
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	69 level of compression; 1 is fastest and produces the least compression,
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	70 and 9 is slowest and produces the most compression. The default is 9.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	71
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	72 """
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	73
23518 c03e8899d4b1 force gzip module to open files using 'b'inary mode. Skip Montanaro <skip@pobox.com> parents: 23075 diff changeset	74 # guarantee the file is opened in binary mode on platforms
c03e8899d4b1 force gzip module to open files using 'b'inary mode. Skip Montanaro <skip@pobox.com> parents: 23075 diff changeset	75 # that care about that sort of thing
c03e8899d4b1 force gzip module to open files using 'b'inary mode. Skip Montanaro <skip@pobox.com> parents: 23075 diff changeset	76 if mode and 'b' not in mode:
c03e8899d4b1 force gzip module to open files using 'b'inary mode. Skip Montanaro <skip@pobox.com> parents: 23075 diff changeset	77 mode += 'b'
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	78 if fileobj is None:
10507 753f3f95c50f Made the default mode 'rb' instead of 'r', for better cross-platform Fred Drake <fdrake@acm.org> parents: 10470 diff changeset	79 fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
5872 dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates Guido van Rossum <guido@python.org> parents: 5386 diff changeset	80 if filename is None:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	81 if hasattr(fileobj, 'name'): filename = fileobj.name
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	82 else: filename = ''
5872 dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates Guido van Rossum <guido@python.org> parents: 5386 diff changeset	83 if mode is None:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	84 if hasattr(fileobj, 'mode'): mode = fileobj.mode
10507 753f3f95c50f Made the default mode 'rb' instead of 'r', for better cross-platform Fred Drake <fdrake@acm.org> parents: 10470 diff changeset	85 else: mode = 'rb'
5872 dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates Guido van Rossum <guido@python.org> parents: 5386 diff changeset	86
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	87 if mode[0:1] == 'r':
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	88 self.mode = READ
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	89 # Set flag indicating start of a new member
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	90 self._new_member = True
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	91 self.extrabuf = ""
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	92 self.extrasize = 0
40839 3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	93 self.name = filename
38237 d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	94 # Starts small, scales exponentially
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	95 self.min_readsize = 100
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	96
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	97 elif mode[0:1] == 'w' or mode[0:1] == 'a':
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	98 self.mode = WRITE
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	99 self._init_write(filename)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	100 self.compress = zlib.compressobj(compresslevel,
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	101 zlib.DEFLATED,
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	102 -zlib.MAX_WBITS,
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	103 zlib.DEF_MEM_LEVEL,
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	104 0)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	105 else:
22216 64c731d180e9 Patch #443899: Check modes on files before performing operations. Martin v. Löwis <martin@v.loewis.de> parents: 20418 diff changeset	106 raise IOError, "Mode " + mode + " not supported"
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	107
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	108 self.fileobj = fileobj
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	109 self.offset = 0
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	110
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	111 if self.mode == WRITE:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	112 self._write_gzip_header()
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	113
40839 3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	114 @property
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	115 def filename(self):
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	116 import warnings
53903 a36fa2ce61b1 Merged revisions 72458 via svnmerge from Philip Jenvey <pjenvey@underboss.org> parents: 47256 diff changeset	117 warnings.warn("use the name attribute", DeprecationWarning, 2)
40839 3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	118 if self.mode == WRITE and self.name[-3:] != ".gz":
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	119 return self.name + ".gz"
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	120 return self.name
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	121
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	122 def __repr__(self):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	123 s = repr(self.fileobj)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	124 return '<gzip ' + s[1:-1] + ' ' + hex(id(self)) + '>'
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	125
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	126 def _init_write(self, filename):
40839 3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	127 self.name = filename
46033 467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	128 self.crc = zlib.crc32("") & 0xffffffffL
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	129 self.size = 0
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	130 self.writebuf = []
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	131 self.bufsize = 0
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	132
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	133 def _write_gzip_header(self):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	134 self.fileobj.write('\037\213') # magic header
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	135 self.fileobj.write('\010') # compression method
40840 d4768bff643e Strip the '.gz' extension from the filename that is written to the Lars Gustäbel <lars@gustaebel.de> parents: 40839 diff changeset	136 fname = self.name
d4768bff643e Strip the '.gz' extension from the filename that is written to the Lars Gustäbel <lars@gustaebel.de> parents: 40839 diff changeset	137 if fname.endswith(".gz"):
d4768bff643e Strip the '.gz' extension from the filename that is written to the Lars Gustäbel <lars@gustaebel.de> parents: 40839 diff changeset	138 fname = fname[:-3]
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	139 flags = 0
40840 d4768bff643e Strip the '.gz' extension from the filename that is written to the Lars Gustäbel <lars@gustaebel.de> parents: 40839 diff changeset	140 if fname:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	141 flags = FNAME
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	142 self.fileobj.write(chr(flags))
10569 f58046339cbf Two different changes. Guido van Rossum <guido@python.org> parents: 10508 diff changeset	143 write32u(self.fileobj, long(time.time()))
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	144 self.fileobj.write('\002')
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	145 self.fileobj.write('\377')
40840 d4768bff643e Strip the '.gz' extension from the filename that is written to the Lars Gustäbel <lars@gustaebel.de> parents: 40839 diff changeset	146 if fname:
d4768bff643e Strip the '.gz' extension from the filename that is written to the Lars Gustäbel <lars@gustaebel.de> parents: 40839 diff changeset	147 self.fileobj.write(fname + '\000')
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	148
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	149 def _init_read(self):
46033 467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	150 self.crc = zlib.crc32("") & 0xffffffffL
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	151 self.size = 0
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	152
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	153 def _read_gzip_header(self):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	154 magic = self.fileobj.read(2)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	155 if magic != '\037\213':
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	156 raise IOError, 'Not a gzipped file'
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	157 method = ord( self.fileobj.read(1) )
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	158 if method != 8:
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	159 raise IOError, 'Unknown compression method'
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	160 flag = ord( self.fileobj.read(1) )
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	161 # modtime = self.fileobj.read(4)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	162 # extraflag = self.fileobj.read(1)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	163 # os = self.fileobj.read(1)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	164 self.fileobj.read(6)
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	165
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	166 if flag & FEXTRA:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	167 # Read & discard the extra field, if present
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	168 xlen = ord(self.fileobj.read(1))
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	169 xlen = xlen + 256*ord(self.fileobj.read(1))
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	170 self.fileobj.read(xlen)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	171 if flag & FNAME:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	172 # Read and discard a null-terminated string containing the filename
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	173 while True:
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	174 s = self.fileobj.read(1)
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	175 if not s or s=='\000':
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	176 break
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	177 if flag & FCOMMENT:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	178 # Read and discard a null-terminated string containing a comment
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	179 while True:
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	180 s = self.fileobj.read(1)
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	181 if not s or s=='\000':
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	182 break
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	183 if flag & FHCRC:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	184 self.fileobj.read(2) # Read & discard the 16-bit header CRC
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	185
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	186
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	187 def write(self,data):
22216 64c731d180e9 Patch #443899: Check modes on files before performing operations. Martin v. Löwis <martin@v.loewis.de> parents: 20418 diff changeset	188 if self.mode != WRITE:
64c731d180e9 Patch #443899: Check modes on files before performing operations. Martin v. Löwis <martin@v.loewis.de> parents: 20418 diff changeset	189 import errno
64c731d180e9 Patch #443899: Check modes on files before performing operations. Martin v. Löwis <martin@v.loewis.de> parents: 20418 diff changeset	190 raise IOError(errno.EBADF, "write() on read-only GzipFile object")
23075 b7e87732f6ab Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 22867 diff changeset	191
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	192 if self.fileobj is None:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	193 raise ValueError, "write() on closed GzipFile object"
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	194 if len(data) > 0:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	195 self.size = self.size + len(data)
46033 467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	196 self.crc = zlib.crc32(data, self.crc) & 0xffffffffL
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	197 self.fileobj.write( self.compress.compress(data) )
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	198 self.offset += len(data)
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	199
11692 0dd520f4dce8 Make read() and readlines() conform more to the file object interface: Guido van Rossum <guido@python.org> parents: 11255 diff changeset	200 def read(self, size=-1):
22216 64c731d180e9 Patch #443899: Check modes on files before performing operations. Martin v. Löwis <martin@v.loewis.de> parents: 20418 diff changeset	201 if self.mode != READ:
64c731d180e9 Patch #443899: Check modes on files before performing operations. Martin v. Löwis <martin@v.loewis.de> parents: 20418 diff changeset	202 import errno
31096 d35f8799cf12 Fix error in exception message. Brett Cannon <bcannon@gmail.com> parents: 27705 diff changeset	203 raise IOError(errno.EBADF, "read() on write-only GzipFile object")
23075 b7e87732f6ab Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 22867 diff changeset	204
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	205 if self.extrasize <= 0 and self.fileobj is None:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	206 return ''
7169 373d78118113 (This fix is really by Jeremy) Guido van Rossum <guido@python.org> parents: 7006 diff changeset	207
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	208 readsize = 1024
11692 0dd520f4dce8 Make read() and readlines() conform more to the file object interface: Guido van Rossum <guido@python.org> parents: 11255 diff changeset	209 if size < 0: # get the whole thing
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	210 try:
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	211 while True:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	212 self._read(readsize)
35050 e79793a8eeea [Bug #1074261, patch #1074381] Restrict the size of chunks read from the file in order to avoid overflow or huge memory consumption. Patch by Mark Eichin Andrew M. Kuchling <amk@amk.ca> parents: 34818 diff changeset	213 readsize = min(self.max_read_chunk, readsize * 2)
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	214 except EOFError:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	215 size = self.extrasize
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	216 else: # just get some more of it
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	217 try:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	218 while size > self.extrasize:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	219 self._read(readsize)
35050 e79793a8eeea [Bug #1074261, patch #1074381] Restrict the size of chunks read from the file in order to avoid overflow or huge memory consumption. Patch by Mark Eichin Andrew M. Kuchling <amk@amk.ca> parents: 34818 diff changeset	220 readsize = min(self.max_read_chunk, readsize * 2)
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	221 except EOFError:
8804 5d5a5bc8de5c Patch by Ron Klatchko: fix invariant in _unread(). Also fixed Guido van Rossum <guido@python.org> parents: 8324 diff changeset	222 if size > self.extrasize:
5d5a5bc8de5c Patch by Ron Klatchko: fix invariant in _unread(). Also fixed Guido van Rossum <guido@python.org> parents: 8324 diff changeset	223 size = self.extrasize
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	224
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	225 chunk = self.extrabuf[:size]
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	226 self.extrabuf = self.extrabuf[size:]
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	227 self.extrasize = self.extrasize - size
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	228
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	229 self.offset += size
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	230 return chunk
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	231
7169 373d78118113 (This fix is really by Jeremy) Guido van Rossum <guido@python.org> parents: 7006 diff changeset	232 def _unread(self, buf):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	233 self.extrabuf = buf + self.extrabuf
8804 5d5a5bc8de5c Patch by Ron Klatchko: fix invariant in _unread(). Also fixed Guido van Rossum <guido@python.org> parents: 8324 diff changeset	234 self.extrasize = len(buf) + self.extrasize
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	235 self.offset -= len(buf)
7169 373d78118113 (This fix is really by Jeremy) Guido van Rossum <guido@python.org> parents: 7006 diff changeset	236
373d78118113 (This fix is really by Jeremy) Guido van Rossum <guido@python.org> parents: 7006 diff changeset	237 def _read(self, size=1024):
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	238 if self.fileobj is None:
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	239 raise EOFError, "Reached EOF"
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	240
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	241 if self._new_member:
13743 c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	242 # If the _new_member flag is set, we have to
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	243 # jump to the next member, if there is one.
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	244 #
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	245 # First, check if we're at the end of the file;
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	246 # if so, it's time to stop; no more members to read.
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	247 pos = self.fileobj.tell() # Save current position
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	248 self.fileobj.seek(0, 2) # Seek to end of file
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	249 if pos == self.fileobj.tell():
11255 427c3b50058e Fixed 'return EOFError' that should be 'raise EOFError', caught by Andrew M. Kuchling <amk@amk.ca> parents: 11184 diff changeset	250 raise EOFError, "Reached EOF"
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	251 else:
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	252 self.fileobj.seek( pos ) # Return to original position
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	253
3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	254 self._init_read()
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	255 self._read_gzip_header()
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	256 self.decompress = zlib.decompressobj(-zlib.MAX_WBITS)
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	257 self._new_member = False
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	258
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	259 # Read a chunk of data from the file
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	260 buf = self.fileobj.read(size)
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	261
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	262 # If the EOF has been reached, flush the decompression object
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	263 # and mark this object as finished.
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	264
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	265 if buf == "":
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	266 uncompress = self.decompress.flush()
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	267 self._read_eof()
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	268 self._add_read_data( uncompress )
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	269 raise EOFError, 'Reached EOF'
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	270
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	271 uncompress = self.decompress.decompress(buf)
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	272 self._add_read_data( uncompress )
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	273
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	274 if self.decompress.unused_data != "":
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	275 # Ending case: we've come to the end of a member in the file,
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	276 # so seek back to the start of the unused data, finish up
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	277 # this member, and read a new gzip header.
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	278 # (The number of bytes to seek back is the length of the unused
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	279 # data, minus 8 because _read_eof() will rewind a further 8 bytes)
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	280 self.fileobj.seek( -len(self.decompress.unused_data)+8, 1)
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	281
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	282 # Check the CRC and file size, and set the flag so we read
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	283 # a new member on the next call
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	284 self._read_eof()
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	285 self._new_member = True
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	286
3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	287 def _add_read_data(self, data):
46033 467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	288 self.crc = zlib.crc32(data, self.crc) & 0xffffffffL
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	289 self.extrabuf = self.extrabuf + data
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	290 self.extrasize = self.extrasize + len(data)
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	291 self.size = self.size + len(data)
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	292
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	293 def _read_eof(self):
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	294 # We've read to the end of the file, so we have to rewind in order
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	295 # to reread the 8 bytes containing the CRC and the file size.
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	296 # We check the that the computed CRC and size of the
26050 c54716aed26a Another round on SF patch 618135: gzip.py and files > 2G Tim Peters <tim.peters@gmail.com> parents: 26019 diff changeset	297 # uncompressed data matches the stored values. Note that the size
c54716aed26a Another round on SF patch 618135: gzip.py and files > 2G Tim Peters <tim.peters@gmail.com> parents: 26019 diff changeset	298 # stored is the true file size mod 2**32.
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	299 self.fileobj.seek(-8, 1)
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	300 crc32 = read32(self.fileobj)
46033 467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	301 isize = read32(self.fileobj) # may exceed 2GB
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	302 if crc32 != self.crc:
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	303 raise IOError("CRC check failed %s != %s" % (hex(crc32),
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	304 hex(self.crc)))
46036 f5b1fa1214f0 A bugfix for r61813, it would fail if the data size was >=2*32. Gregory P. Smith <greg@mad-scientist.com>* parents: 46033 diff changeset	305 elif isize != (self.size & 0xffffffffL):
27705 0f6731d126de [Patch #654421 from Matthew Mueller] Andrew M. Kuchling <amk@amk.ca> parents: 26050 diff changeset	306 raise IOError, "Incorrect length of data produced"
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	307
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	308 def close(self):
47256 d5591c16b997 #2959: allow multiple close() calls for GzipFile. Georg Brandl <georg@python.org> parents: 46037 diff changeset	309 if self.fileobj is None:
d5591c16b997 #2959: allow multiple close() calls for GzipFile. Georg Brandl <georg@python.org> parents: 46037 diff changeset	310 return
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	311 if self.mode == WRITE:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	312 self.fileobj.write(self.compress.flush())
46033 467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	313 write32u(self.fileobj, self.crc)
26050 c54716aed26a Another round on SF patch 618135: gzip.py and files > 2G Tim Peters <tim.peters@gmail.com> parents: 26019 diff changeset	314 # self.size may exceed 2GB, or even 4GB
46037 7caaab04a4d8 prevent a warning from the struct module when data size >= 2*32. Gregory P. Smith <greg@mad-scientist.com>* parents: 46036 diff changeset	315 write32u(self.fileobj, self.size & 0xffffffffL)
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	316 self.fileobj = None
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	317 elif self.mode == READ:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	318 self.fileobj = None
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	319 if self.myfileobj:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	320 self.myfileobj.close()
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	321 self.myfileobj = None
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	322
11184 938b46fa6394 Added __del__ method to GzipFile class that will flush and close the Andrew M. Kuchling <amk@amk.ca> parents: 10569 diff changeset	323 def __del__(self):
12619 eb49141d6e10 if the GzipFile constructor fails, the __del__ method is still Jeremy Hylton <jeremy@alum.mit.edu> parents: 11708 diff changeset	324 try:
eb49141d6e10 if the GzipFile constructor fails, the __del__ method is still Jeremy Hylton <jeremy@alum.mit.edu> parents: 11708 diff changeset	325 if (self.myfileobj is None and
eb49141d6e10 if the GzipFile constructor fails, the __del__ method is still Jeremy Hylton <jeremy@alum.mit.edu> parents: 11708 diff changeset	326 self.fileobj is None):
eb49141d6e10 if the GzipFile constructor fails, the __del__ method is still Jeremy Hylton <jeremy@alum.mit.edu> parents: 11708 diff changeset	327 return
eb49141d6e10 if the GzipFile constructor fails, the __del__ method is still Jeremy Hylton <jeremy@alum.mit.edu> parents: 11708 diff changeset	328 except AttributeError:
eb49141d6e10 if the GzipFile constructor fails, the __del__ method is still Jeremy Hylton <jeremy@alum.mit.edu> parents: 11708 diff changeset	329 return
eb49141d6e10 if the GzipFile constructor fails, the __del__ method is still Jeremy Hylton <jeremy@alum.mit.edu> parents: 11708 diff changeset	330 self.close()
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	331
34650 11fcdbb279de Patch #1110248: SYNC_FLUSH the zlib buffer for GZipFile.flush. Martin v. Löwis <martin@v.loewis.de> parents: 32835 diff changeset	332 def flush(self,zlib_mode=zlib.Z_SYNC_FLUSH):
11fcdbb279de Patch #1110248: SYNC_FLUSH the zlib buffer for GZipFile.flush. Martin v. Löwis <martin@v.loewis.de> parents: 32835 diff changeset	333 if self.mode == WRITE:
34818 4eaab38566c9 Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 34650 diff changeset	334 # Ensure the compressor's buffer is flushed
4eaab38566c9 Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 34650 diff changeset	335 self.fileobj.write(self.compress.flush(zlib_mode))
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	336 self.fileobj.flush()
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	337
32834 718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	338 def fileno(self):
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	339 """Invoke the underlying file object's fileno() method.
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	340
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	341 This will raise AttributeError if the underlying file object
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	342 doesn't support fileno().
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	343 """
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	344 return self.fileobj.fileno()
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	345
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	346 def isatty(self):
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	347 return False
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	348
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	349 def tell(self):
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	350 return self.offset
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	351
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	352 def rewind(self):
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	353 '''Return the uncompressed stream file position indicator to the
19119 600a9a2b7a18 Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 19088 diff changeset	354 beginning of the file'''
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	355 if self.mode != READ:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	356 raise IOError("Can't rewind in write mode")
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	357 self.fileobj.seek(0)
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	358 self._new_member = True
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	359 self.extrabuf = ""
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	360 self.extrasize = 0
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	361 self.offset = 0
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	362
40334 9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek. Martin v. Löwis <martin@v.loewis.de> parents: 39347 diff changeset	363 def seek(self, offset, whence=0):
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek. Martin v. Löwis <martin@v.loewis.de> parents: 39347 diff changeset	364 if whence:
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek. Martin v. Löwis <martin@v.loewis.de> parents: 39347 diff changeset	365 if whence == 1:
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek. Martin v. Löwis <martin@v.loewis.de> parents: 39347 diff changeset	366 offset = self.offset + offset
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek. Martin v. Löwis <martin@v.loewis.de> parents: 39347 diff changeset	367 else:
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek. Martin v. Löwis <martin@v.loewis.de> parents: 39347 diff changeset	368 raise ValueError('Seek from end not supported')
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	369 if self.mode == WRITE:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	370 if offset < self.offset:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	371 raise IOError('Negative seek in write mode')
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	372 count = offset - self.offset
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	373 for i in range(count // 1024):
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	374 self.write(1024 * '\0')
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	375 self.write((count % 1024) * '\0')
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	376 elif self.mode == READ:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	377 if offset < self.offset:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	378 # for negative seek, rewind and do positive seek
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	379 self.rewind()
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	380 count = offset - self.offset
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	381 for i in range(count // 1024):
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	382 self.read(1024)
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	383 self.read(count % 1024)
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	384
13743 c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	385 def readline(self, size=-1):
38237 d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	386 if size < 0:
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	387 size = sys.maxint
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	388 readsize = self.min_readsize
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	389 else:
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	390 readsize = size
38235 fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found Bob Ippolito <bob@redivi.com> parents: 38234 diff changeset	391 bufs = []
38237 d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	392 while size != 0:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	393 c = self.read(readsize)
16496 1ff9d571bac0 String method conversion. Eric S. Raymond <esr@thyrsus.com> parents: 16160 diff changeset	394 i = c.find('\n')
38237 d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	395
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	396 # We set i=size to break out of the loop under two
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	397 # conditions: 1) there's no newline, and the chunk is
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	398 # larger than size, or 2) there is a newline, but the
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	399 # resulting line would be longer than 'size'.
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	400 if (size <= i) or (i == -1 and len(c) > size):
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	401 i = size - 1
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	402
38235 fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found Bob Ippolito <bob@redivi.com> parents: 38234 diff changeset	403 if i >= 0 or c == '':
38237 d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	404 bufs.append(c[:i + 1]) # Add portion of last chunk
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	405 self._unread(c[i + 1:]) # Push back rest of chunk
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	406 break
38234 27c80058173e GzipFile.readline performance improvement (~30-40%), patch #1281707 Bob Ippolito <bob@redivi.com> parents: 35050 diff changeset	407
38235 fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found Bob Ippolito <bob@redivi.com> parents: 38234 diff changeset	408 # Append chunk to list, decrease 'size',
fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found Bob Ippolito <bob@redivi.com> parents: 38234 diff changeset	409 bufs.append(c)
fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found Bob Ippolito <bob@redivi.com> parents: 38234 diff changeset	410 size = size - len(c)
fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found Bob Ippolito <bob@redivi.com> parents: 38234 diff changeset	411 readsize = min(size, readsize * 2)
38237 d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	412 if readsize > self.min_readsize:
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	413 self.min_readsize = min(readsize, self.min_readsize * 2, 512)
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	414 return ''.join(bufs) # Return resulting line
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	415
13743 c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	416 def readlines(self, sizehint=0):
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	417 # Negative numbers result in reading all the lines
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	418 if sizehint <= 0:
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	419 sizehint = sys.maxint
13743 c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	420 L = []
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	421 while sizehint > 0:
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	422 line = self.readline()
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	423 if line == "":
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	424 break
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	425 L.append(line)
13743 c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	426 sizehint = sizehint - len(line)
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	427
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	428 return L
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	429
5872 dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates Guido van Rossum <guido@python.org> parents: 5386 diff changeset	430 def writelines(self, L):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	431 for line in L:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	432 self.write(line)
7006 9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	433
22417 df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	434 def __iter__(self):
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	435 return self
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	436
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	437 def next(self):
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	438 line = self.readline()
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	439 if line:
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	440 return line
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	441 else:
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	442 raise StopIteration
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	443
7006 9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	444
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	445 def _test():
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	446 # Act like gzip; with -d, act like gunzip.
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	447 # The input file is not deleted, however, nor are any other gzip
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	448 # options or features supported.
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	449 args = sys.argv[1:]
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	450 decompress = args and args[0] == "-d"
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	451 if decompress:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	452 args = args[1:]
7006 9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	453 if not args:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	454 args = ["-"]
7006 9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	455 for arg in args:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	456 if decompress:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	457 if arg == "-":
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	458 f = GzipFile(filename="", mode="rb", fileobj=sys.stdin)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	459 g = sys.stdout
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	460 else:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	461 if arg[-3:] != ".gz":
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 31096 diff changeset	462 print "filename doesn't end in .gz:", repr(arg)
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	463 continue
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	464 f = open(arg, "rb")
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	465 g = __builtin__.open(arg[:-3], "wb")
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	466 else:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	467 if arg == "-":
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	468 f = sys.stdin
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	469 g = GzipFile(filename="", mode="wb", fileobj=sys.stdout)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	470 else:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	471 f = __builtin__.open(arg, "rb")
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	472 g = open(arg + ".gz", "wb")
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	473 while True:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	474 chunk = f.read(1024)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	475 if not chunk:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	476 break
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	477 g.write(chunk)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	478 if g is not sys.stdout:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	479 g.close()
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	480 if f is not sys.stdin:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	481 f.close()
7006 9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	482
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	483 if __name__ == '__main__':
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	484 _test()

Mercurial > cpython-lru_cache-weakref

annotate Lib/gzip.py @ 58925:eecf1bef052d 2.6