cpython-lru_cache-weakref: Lib/gzip.py annotate

annotate Lib/gzip.py @ 54294:dba767bc3155 legacy-trunk

remove mention of old ctypes version

author	Benjamin Peterson <benjamin@python.org>
date	Sat, 23 May 2009 20:59:09 +0000
parents	a015a298543d
children	b571c6685229

rev	line source
11708 4a23cc49e2b0 Actually, the previous batch's comment should have been different; Guido van Rossum <guido@python.org> parents: 11706 diff changeset	1 """Functions that read and write gzipped files.
4a23cc49e2b0 Actually, the previous batch's comment should have been different; Guido van Rossum <guido@python.org> parents: 11706 diff changeset	2
11706 b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee, Guido van Rossum <guido@python.org> parents: 11692 diff changeset	3 The user of the file doesn't have to worry about the compression,
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee, Guido van Rossum <guido@python.org> parents: 11692 diff changeset	4 but random access is not allowed."""
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee, Guido van Rossum <guido@python.org> parents: 11692 diff changeset	5
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee, Guido van Rossum <guido@python.org> parents: 11692 diff changeset	6 # based on Andrew Kuchling's minigzip.py distributed with the zlib module
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee, Guido van Rossum <guido@python.org> parents: 11692 diff changeset	7
32835 b58cf6b5fc91 Ack, removed useless import of os I just introduced. Tim Peters <tim.peters@gmail.com> parents: 32834 diff changeset	8 import struct, sys, time
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	9 import zlib
5872 dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates Guido van Rossum <guido@python.org> parents: 5386 diff changeset	10 import __builtin__
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	11
16160 46920b60cd33 added a few more __all__ lists Skip Montanaro <skip@pobox.com> parents: 15868 diff changeset	12 __all__ = ["GzipFile","open"]
46920b60cd33 added a few more __all__ lists Skip Montanaro <skip@pobox.com> parents: 15868 diff changeset	13
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	14 FTEXT, FHCRC, FEXTRA, FNAME, FCOMMENT = 1, 2, 4, 8, 16
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	15
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	16 READ, WRITE = 1, 2
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	17
10569 f58046339cbf Two different changes. Guido van Rossum <guido@python.org> parents: 10508 diff changeset	18 def write32u(output, value):
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	19 # The L format writes the bit pattern correctly whether signed
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	20 # or unsigned.
10569 f58046339cbf Two different changes. Guido van Rossum <guido@python.org> parents: 10508 diff changeset	21 output.write(struct.pack("<L", value))
f58046339cbf Two different changes. Guido van Rossum <guido@python.org> parents: 10508 diff changeset	22
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	23 def read32(input):
46033 467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	24 return struct.unpack("<I", input.read(4))[0]
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	25
10508 d79989a435d9 Oops, missed mode parameter to open(). Fred Drake <fdrake@acm.org> parents: 10507 diff changeset	26 def open(filename, mode="rb", compresslevel=9):
23565 1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	27 """Shorthand for GzipFile(filename, mode, compresslevel).
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	28
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	29 The filename argument is required; mode defaults to 'rb'
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	30 and compresslevel defaults to 9.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	31
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	32 """
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	33 return GzipFile(filename, mode, compresslevel)
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	34
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	35 class GzipFile:
23565 1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	36 """The GzipFile class simulates most of the methods of a file object with
24601 262ad16b4de0 Remove mention of deprecated xreadlines method. Guido van Rossum <guido@python.org> parents: 23565 diff changeset	37 the exception of the readinto() and truncate() methods.
23565 1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	38
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	39 """
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	40
5872 dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates Guido van Rossum <guido@python.org> parents: 5386 diff changeset	41 myfileobj = None
35050 e79793a8eeea [Bug #1074261, patch #1074381] Restrict the size of chunks read from the file in order to avoid overflow or huge memory consumption. Patch by Mark Eichin Andrew M. Kuchling <amk@amk.ca> parents: 34818 diff changeset	42 max_read_chunk = 10 * 1024 * 1024 # 10Mb
5872 dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates Guido van Rossum <guido@python.org> parents: 5386 diff changeset	43
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	44 def __init__(self, filename=None, mode=None,
50600 9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	45 compresslevel=9, fileobj=None, mtime=None):
23565 1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	46 """Constructor for the GzipFile class.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	47
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	48 At least one of fileobj and filename must be given a
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	49 non-trivial value.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	50
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	51 The new class instance is based on fileobj, which can be a regular
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	52 file, a StringIO object, or any other object which simulates a file.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	53 It defaults to None, in which case filename is opened to provide
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	54 a file object.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	55
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	56 When fileobj is not None, the filename argument is only used to be
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	57 included in the gzip file header, which may includes the original
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	58 filename of the uncompressed file. It defaults to the filename of
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	59 fileobj, if discernible; otherwise, it defaults to the empty string,
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	60 and in this case the original filename is not included in the header.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	61
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	62 The mode argument can be any of 'r', 'rb', 'a', 'ab', 'w', or 'wb',
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	63 depending on whether the file will be read or written. The default
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	64 is the mode of fileobj if discernible; otherwise, the default is 'rb'.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	65 Be aware that only the 'rb', 'ab', and 'wb' values should be used
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	66 for cross-platform portability.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	67
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	68 The compresslevel argument is an integer from 1 to 9 controlling the
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	69 level of compression; 1 is fastest and produces the least compression,
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	70 and 9 is slowest and produces the most compression. The default is 9.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	71
50600 9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	72 The mtime argument is an optional numeric timestamp to be written
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	73 to the stream when compressing. All gzip compressed streams
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	74 are required to contain a timestamp. If omitted or None, the
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	75 current time is used. This module ignores the timestamp when
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	76 decompressing; however, some programs, such as gunzip, make use
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	77 of it. The format of the timestamp is the same as that of the
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	78 return value of time.time() and of the st_mtime member of the
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	79 object returned by os.stat().
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	80
23565 1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	81 """
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 23518 diff changeset	82
23518 c03e8899d4b1 force gzip module to open files using 'b'inary mode. Skip Montanaro <skip@pobox.com> parents: 23075 diff changeset	83 # guarantee the file is opened in binary mode on platforms
c03e8899d4b1 force gzip module to open files using 'b'inary mode. Skip Montanaro <skip@pobox.com> parents: 23075 diff changeset	84 # that care about that sort of thing
c03e8899d4b1 force gzip module to open files using 'b'inary mode. Skip Montanaro <skip@pobox.com> parents: 23075 diff changeset	85 if mode and 'b' not in mode:
c03e8899d4b1 force gzip module to open files using 'b'inary mode. Skip Montanaro <skip@pobox.com> parents: 23075 diff changeset	86 mode += 'b'
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	87 if fileobj is None:
10507 753f3f95c50f Made the default mode 'rb' instead of 'r', for better cross-platform Fred Drake <fdrake@acm.org> parents: 10470 diff changeset	88 fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
5872 dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates Guido van Rossum <guido@python.org> parents: 5386 diff changeset	89 if filename is None:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	90 if hasattr(fileobj, 'name'): filename = fileobj.name
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	91 else: filename = ''
5872 dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates Guido van Rossum <guido@python.org> parents: 5386 diff changeset	92 if mode is None:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	93 if hasattr(fileobj, 'mode'): mode = fileobj.mode
10507 753f3f95c50f Made the default mode 'rb' instead of 'r', for better cross-platform Fred Drake <fdrake@acm.org> parents: 10470 diff changeset	94 else: mode = 'rb'
5872 dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates Guido van Rossum <guido@python.org> parents: 5386 diff changeset	95
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	96 if mode[0:1] == 'r':
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	97 self.mode = READ
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	98 # Set flag indicating start of a new member
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	99 self._new_member = True
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	100 self.extrabuf = ""
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	101 self.extrasize = 0
40839 3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	102 self.name = filename
38237 d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	103 # Starts small, scales exponentially
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	104 self.min_readsize = 100
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	105
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	106 elif mode[0:1] == 'w' or mode[0:1] == 'a':
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	107 self.mode = WRITE
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	108 self._init_write(filename)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	109 self.compress = zlib.compressobj(compresslevel,
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	110 zlib.DEFLATED,
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	111 -zlib.MAX_WBITS,
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	112 zlib.DEF_MEM_LEVEL,
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	113 0)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	114 else:
22216 64c731d180e9 Patch #443899: Check modes on files before performing operations. Martin v. Löwis <martin@v.loewis.de> parents: 20418 diff changeset	115 raise IOError, "Mode " + mode + " not supported"
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	116
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	117 self.fileobj = fileobj
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	118 self.offset = 0
50600 9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	119 self.mtime = mtime
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	120
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	121 if self.mode == WRITE:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	122 self._write_gzip_header()
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	123
40839 3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	124 @property
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	125 def filename(self):
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	126 import warnings
53901 a015a298543d #4351: more appropriate DeprecationWarning stacklevels Philip Jenvey <pjenvey@underboss.org> parents: 50698 diff changeset	127 warnings.warn("use the name attribute", DeprecationWarning, 2)
40839 3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	128 if self.mode == WRITE and self.name[-3:] != ".gz":
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	129 return self.name + ".gz"
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	130 return self.name
3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	131
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	132 def __repr__(self):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	133 s = repr(self.fileobj)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	134 return '<gzip ' + s[1:-1] + ' ' + hex(id(self)) + '>'
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	135
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	136 def _init_write(self, filename):
40839 3983ee0f6f96 Patch #1647484: Renamed GzipFile's filename attribute to name. The Lars Gustäbel <lars@gustaebel.de> parents: 40334 diff changeset	137 self.name = filename
46033 467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	138 self.crc = zlib.crc32("") & 0xffffffffL
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	139 self.size = 0
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	140 self.writebuf = []
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	141 self.bufsize = 0
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	142
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	143 def _write_gzip_header(self):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	144 self.fileobj.write('\037\213') # magic header
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	145 self.fileobj.write('\010') # compression method
40840 d4768bff643e Strip the '.gz' extension from the filename that is written to the Lars Gustäbel <lars@gustaebel.de> parents: 40839 diff changeset	146 fname = self.name
d4768bff643e Strip the '.gz' extension from the filename that is written to the Lars Gustäbel <lars@gustaebel.de> parents: 40839 diff changeset	147 if fname.endswith(".gz"):
d4768bff643e Strip the '.gz' extension from the filename that is written to the Lars Gustäbel <lars@gustaebel.de> parents: 40839 diff changeset	148 fname = fname[:-3]
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	149 flags = 0
40840 d4768bff643e Strip the '.gz' extension from the filename that is written to the Lars Gustäbel <lars@gustaebel.de> parents: 40839 diff changeset	150 if fname:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	151 flags = FNAME
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	152 self.fileobj.write(chr(flags))
50600 9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	153 mtime = self.mtime
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	154 if mtime is None:
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	155 mtime = time.time()
9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	156 write32u(self.fileobj, long(mtime))
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	157 self.fileobj.write('\002')
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	158 self.fileobj.write('\377')
40840 d4768bff643e Strip the '.gz' extension from the filename that is written to the Lars Gustäbel <lars@gustaebel.de> parents: 40839 diff changeset	159 if fname:
d4768bff643e Strip the '.gz' extension from the filename that is written to the Lars Gustäbel <lars@gustaebel.de> parents: 40839 diff changeset	160 self.fileobj.write(fname + '\000')
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	161
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	162 def _init_read(self):
46033 467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	163 self.crc = zlib.crc32("") & 0xffffffffL
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	164 self.size = 0
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	165
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	166 def _read_gzip_header(self):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	167 magic = self.fileobj.read(2)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	168 if magic != '\037\213':
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	169 raise IOError, 'Not a gzipped file'
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	170 method = ord( self.fileobj.read(1) )
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	171 if method != 8:
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	172 raise IOError, 'Unknown compression method'
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	173 flag = ord( self.fileobj.read(1) )
50600 9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	174 self.mtime = read32(self.fileobj)
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	175 # extraflag = self.fileobj.read(1)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	176 # os = self.fileobj.read(1)
50600 9de21e615577 Issue #4272: Add an optional argument to the GzipFile constructor to override the timestamp in the gzip stream. Antoine Pitrou <solipsis@pitrou.net> parents: 47256 diff changeset	177 self.fileobj.read(2)
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	178
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	179 if flag & FEXTRA:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	180 # Read & discard the extra field, if present
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	181 xlen = ord(self.fileobj.read(1))
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	182 xlen = xlen + 256*ord(self.fileobj.read(1))
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	183 self.fileobj.read(xlen)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	184 if flag & FNAME:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	185 # Read and discard a null-terminated string containing the filename
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	186 while True:
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	187 s = self.fileobj.read(1)
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	188 if not s or s=='\000':
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	189 break
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	190 if flag & FCOMMENT:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	191 # Read and discard a null-terminated string containing a comment
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	192 while True:
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	193 s = self.fileobj.read(1)
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	194 if not s or s=='\000':
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	195 break
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	196 if flag & FHCRC:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	197 self.fileobj.read(2) # Read & discard the 16-bit header CRC
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	198
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	199
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	200 def write(self,data):
22216 64c731d180e9 Patch #443899: Check modes on files before performing operations. Martin v. Löwis <martin@v.loewis.de> parents: 20418 diff changeset	201 if self.mode != WRITE:
64c731d180e9 Patch #443899: Check modes on files before performing operations. Martin v. Löwis <martin@v.loewis.de> parents: 20418 diff changeset	202 import errno
64c731d180e9 Patch #443899: Check modes on files before performing operations. Martin v. Löwis <martin@v.loewis.de> parents: 20418 diff changeset	203 raise IOError(errno.EBADF, "write() on read-only GzipFile object")
23075 b7e87732f6ab Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 22867 diff changeset	204
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	205 if self.fileobj is None:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	206 raise ValueError, "write() on closed GzipFile object"
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	207 if len(data) > 0:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	208 self.size = self.size + len(data)
46033 467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	209 self.crc = zlib.crc32(data, self.crc) & 0xffffffffL
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	210 self.fileobj.write( self.compress.compress(data) )
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	211 self.offset += len(data)
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	212
11692 0dd520f4dce8 Make read() and readlines() conform more to the file object interface: Guido van Rossum <guido@python.org> parents: 11255 diff changeset	213 def read(self, size=-1):
22216 64c731d180e9 Patch #443899: Check modes on files before performing operations. Martin v. Löwis <martin@v.loewis.de> parents: 20418 diff changeset	214 if self.mode != READ:
64c731d180e9 Patch #443899: Check modes on files before performing operations. Martin v. Löwis <martin@v.loewis.de> parents: 20418 diff changeset	215 import errno
31096 d35f8799cf12 Fix error in exception message. Brett Cannon <bcannon@gmail.com> parents: 27705 diff changeset	216 raise IOError(errno.EBADF, "read() on write-only GzipFile object")
23075 b7e87732f6ab Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 22867 diff changeset	217
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	218 if self.extrasize <= 0 and self.fileobj is None:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	219 return ''
7169 373d78118113 (This fix is really by Jeremy) Guido van Rossum <guido@python.org> parents: 7006 diff changeset	220
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	221 readsize = 1024
11692 0dd520f4dce8 Make read() and readlines() conform more to the file object interface: Guido van Rossum <guido@python.org> parents: 11255 diff changeset	222 if size < 0: # get the whole thing
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	223 try:
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	224 while True:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	225 self._read(readsize)
35050 e79793a8eeea [Bug #1074261, patch #1074381] Restrict the size of chunks read from the file in order to avoid overflow or huge memory consumption. Patch by Mark Eichin Andrew M. Kuchling <amk@amk.ca> parents: 34818 diff changeset	226 readsize = min(self.max_read_chunk, readsize * 2)
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	227 except EOFError:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	228 size = self.extrasize
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	229 else: # just get some more of it
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	230 try:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	231 while size > self.extrasize:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	232 self._read(readsize)
35050 e79793a8eeea [Bug #1074261, patch #1074381] Restrict the size of chunks read from the file in order to avoid overflow or huge memory consumption. Patch by Mark Eichin Andrew M. Kuchling <amk@amk.ca> parents: 34818 diff changeset	233 readsize = min(self.max_read_chunk, readsize * 2)
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	234 except EOFError:
8804 5d5a5bc8de5c Patch by Ron Klatchko: fix invariant in _unread(). Also fixed Guido van Rossum <guido@python.org> parents: 8324 diff changeset	235 if size > self.extrasize:
5d5a5bc8de5c Patch by Ron Klatchko: fix invariant in _unread(). Also fixed Guido van Rossum <guido@python.org> parents: 8324 diff changeset	236 size = self.extrasize
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	237
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	238 chunk = self.extrabuf[:size]
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	239 self.extrabuf = self.extrabuf[size:]
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	240 self.extrasize = self.extrasize - size
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	241
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	242 self.offset += size
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	243 return chunk
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	244
7169 373d78118113 (This fix is really by Jeremy) Guido van Rossum <guido@python.org> parents: 7006 diff changeset	245 def _unread(self, buf):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	246 self.extrabuf = buf + self.extrabuf
8804 5d5a5bc8de5c Patch by Ron Klatchko: fix invariant in _unread(). Also fixed Guido van Rossum <guido@python.org> parents: 8324 diff changeset	247 self.extrasize = len(buf) + self.extrasize
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	248 self.offset -= len(buf)
7169 373d78118113 (This fix is really by Jeremy) Guido van Rossum <guido@python.org> parents: 7006 diff changeset	249
373d78118113 (This fix is really by Jeremy) Guido van Rossum <guido@python.org> parents: 7006 diff changeset	250 def _read(self, size=1024):
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	251 if self.fileobj is None:
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	252 raise EOFError, "Reached EOF"
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	253
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	254 if self._new_member:
13743 c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	255 # If the _new_member flag is set, we have to
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	256 # jump to the next member, if there is one.
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	257 #
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	258 # First, check if we're at the end of the file;
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	259 # if so, it's time to stop; no more members to read.
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	260 pos = self.fileobj.tell() # Save current position
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	261 self.fileobj.seek(0, 2) # Seek to end of file
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	262 if pos == self.fileobj.tell():
11255 427c3b50058e Fixed 'return EOFError' that should be 'raise EOFError', caught by Andrew M. Kuchling <amk@amk.ca> parents: 11184 diff changeset	263 raise EOFError, "Reached EOF"
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	264 else:
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	265 self.fileobj.seek( pos ) # Return to original position
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	266
3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	267 self._init_read()
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	268 self._read_gzip_header()
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	269 self.decompress = zlib.decompressobj(-zlib.MAX_WBITS)
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	270 self._new_member = False
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	271
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	272 # Read a chunk of data from the file
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	273 buf = self.fileobj.read(size)
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	274
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	275 # If the EOF has been reached, flush the decompression object
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	276 # and mark this object as finished.
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	277
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	278 if buf == "":
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	279 uncompress = self.decompress.flush()
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	280 self._read_eof()
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	281 self._add_read_data( uncompress )
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	282 raise EOFError, 'Reached EOF'
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	283
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	284 uncompress = self.decompress.decompress(buf)
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	285 self._add_read_data( uncompress )
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	286
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	287 if self.decompress.unused_data != "":
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	288 # Ending case: we've come to the end of a member in the file,
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	289 # so seek back to the start of the unused data, finish up
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	290 # this member, and read a new gzip header.
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	291 # (The number of bytes to seek back is the length of the unused
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	292 # data, minus 8 because _read_eof() will rewind a further 8 bytes)
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	293 self.fileobj.seek( -len(self.decompress.unused_data)+8, 1)
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	294
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	295 # Check the CRC and file size, and set the flag so we read
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	296 # a new member on the next call
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	297 self._read_eof()
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	298 self._new_member = True
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	299
3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	300 def _add_read_data(self, data):
46033 467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	301 self.crc = zlib.crc32(data, self.crc) & 0xffffffffL
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	302 self.extrabuf = self.extrabuf + data
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	303 self.extrasize = self.extrasize + len(data)
022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	304 self.size = self.size + len(data)
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	305
43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	306 def _read_eof(self):
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	307 # We've read to the end of the file, so we have to rewind in order
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	308 # to reread the 8 bytes containing the CRC and the file size.
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	309 # We check the that the computed CRC and size of the
26050 c54716aed26a Another round on SF patch 618135: gzip.py and files > 2G Tim Peters <tim.peters@gmail.com> parents: 26019 diff changeset	310 # uncompressed data matches the stored values. Note that the size
c54716aed26a Another round on SF patch 618135: gzip.py and files > 2G Tim Peters <tim.peters@gmail.com> parents: 26019 diff changeset	311 # stored is the true file size mod 2**32.
10470 022b178fd0ed Based on a suggestion from bruce@hams.com, make a trivial change to Andrew M. Kuchling <amk@amk.ca> parents: 10450 diff changeset	312 self.fileobj.seek(-8, 1)
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	313 crc32 = read32(self.fileobj)
46033 467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	314 isize = read32(self.fileobj) # may exceed 2GB
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	315 if crc32 != self.crc:
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	316 raise IOError("CRC check failed %s != %s" % (hex(crc32),
467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	317 hex(self.crc)))
46036 f5b1fa1214f0 A bugfix for r61813, it would fail if the data size was >=2*32. Gregory P. Smith <greg@mad-scientist.com>* parents: 46033 diff changeset	318 elif isize != (self.size & 0xffffffffL):
27705 0f6731d126de [Patch #654421 from Matthew Mueller] Andrew M. Kuchling <amk@amk.ca> parents: 26050 diff changeset	319 raise IOError, "Incorrect length of data produced"
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	320
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	321 def close(self):
47256 d5591c16b997 #2959: allow multiple close() calls for GzipFile. Georg Brandl <georg@python.org> parents: 46037 diff changeset	322 if self.fileobj is None:
d5591c16b997 #2959: allow multiple close() calls for GzipFile. Georg Brandl <georg@python.org> parents: 46037 diff changeset	323 return
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	324 if self.mode == WRITE:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	325 self.fileobj.write(self.compress.flush())
46033 467c16a50392 Fix gzip to deal with CRC's being signed values in Python 2.x properly and to Gregory P. Smith <greg@mad-scientist.com> parents: 45933 diff changeset	326 write32u(self.fileobj, self.crc)
26050 c54716aed26a Another round on SF patch 618135: gzip.py and files > 2G Tim Peters <tim.peters@gmail.com> parents: 26019 diff changeset	327 # self.size may exceed 2GB, or even 4GB
46037 7caaab04a4d8 prevent a warning from the struct module when data size >= 2*32. Gregory P. Smith <greg@mad-scientist.com>* parents: 46036 diff changeset	328 write32u(self.fileobj, self.size & 0xffffffffL)
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	329 self.fileobj = None
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	330 elif self.mode == READ:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	331 self.fileobj = None
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	332 if self.myfileobj:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	333 self.myfileobj.close()
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	334 self.myfileobj = None
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	335
11184 938b46fa6394 Added __del__ method to GzipFile class that will flush and close the Andrew M. Kuchling <amk@amk.ca> parents: 10569 diff changeset	336 def __del__(self):
12619 eb49141d6e10 if the GzipFile constructor fails, the __del__ method is still Jeremy Hylton <jeremy@alum.mit.edu> parents: 11708 diff changeset	337 try:
eb49141d6e10 if the GzipFile constructor fails, the __del__ method is still Jeremy Hylton <jeremy@alum.mit.edu> parents: 11708 diff changeset	338 if (self.myfileobj is None and
eb49141d6e10 if the GzipFile constructor fails, the __del__ method is still Jeremy Hylton <jeremy@alum.mit.edu> parents: 11708 diff changeset	339 self.fileobj is None):
eb49141d6e10 if the GzipFile constructor fails, the __del__ method is still Jeremy Hylton <jeremy@alum.mit.edu> parents: 11708 diff changeset	340 return
eb49141d6e10 if the GzipFile constructor fails, the __del__ method is still Jeremy Hylton <jeremy@alum.mit.edu> parents: 11708 diff changeset	341 except AttributeError:
eb49141d6e10 if the GzipFile constructor fails, the __del__ method is still Jeremy Hylton <jeremy@alum.mit.edu> parents: 11708 diff changeset	342 return
eb49141d6e10 if the GzipFile constructor fails, the __del__ method is still Jeremy Hylton <jeremy@alum.mit.edu> parents: 11708 diff changeset	343 self.close()
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	344
34650 11fcdbb279de Patch #1110248: SYNC_FLUSH the zlib buffer for GZipFile.flush. Martin v. Löwis <martin@v.loewis.de> parents: 32835 diff changeset	345 def flush(self,zlib_mode=zlib.Z_SYNC_FLUSH):
11fcdbb279de Patch #1110248: SYNC_FLUSH the zlib buffer for GZipFile.flush. Martin v. Löwis <martin@v.loewis.de> parents: 32835 diff changeset	346 if self.mode == WRITE:
34818 4eaab38566c9 Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 34650 diff changeset	347 # Ensure the compressor's buffer is flushed
4eaab38566c9 Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 34650 diff changeset	348 self.fileobj.write(self.compress.flush(zlib_mode))
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	349 self.fileobj.flush()
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	350
32834 718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	351 def fileno(self):
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	352 """Invoke the underlying file object's fileno() method.
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	353
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	354 This will raise AttributeError if the underlying file object
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	355 doesn't support fileno().
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	356 """
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	357 return self.fileobj.fileno()
718305112f9f Added a new fileno() method. ZODB's repozo.py wants this so it can Tim Peters <tim.peters@gmail.com> parents: 31515 diff changeset	358
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	359 def isatty(self):
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	360 return False
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	361
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	362 def tell(self):
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	363 return self.offset
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	364
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	365 def rewind(self):
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	366 '''Return the uncompressed stream file position indicator to the
19119 600a9a2b7a18 Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 19088 diff changeset	367 beginning of the file'''
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	368 if self.mode != READ:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	369 raise IOError("Can't rewind in write mode")
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	370 self.fileobj.seek(0)
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	371 self._new_member = True
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	372 self.extrabuf = ""
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	373 self.extrasize = 0
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	374 self.offset = 0
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	375
40334 9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek. Martin v. Löwis <martin@v.loewis.de> parents: 39347 diff changeset	376 def seek(self, offset, whence=0):
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek. Martin v. Löwis <martin@v.loewis.de> parents: 39347 diff changeset	377 if whence:
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek. Martin v. Löwis <martin@v.loewis.de> parents: 39347 diff changeset	378 if whence == 1:
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek. Martin v. Löwis <martin@v.loewis.de> parents: 39347 diff changeset	379 offset = self.offset + offset
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek. Martin v. Löwis <martin@v.loewis.de> parents: 39347 diff changeset	380 else:
9f7fc68a20ae Patch #1355023: support whence argument for GzipFile.seek. Martin v. Löwis <martin@v.loewis.de> parents: 39347 diff changeset	381 raise ValueError('Seek from end not supported')
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	382 if self.mode == WRITE:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	383 if offset < self.offset:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	384 raise IOError('Negative seek in write mode')
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	385 count = offset - self.offset
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	386 for i in range(count // 1024):
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	387 self.write(1024 * '\0')
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	388 self.write((count % 1024) * '\0')
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	389 elif self.mode == READ:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	390 if offset < self.offset:
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	391 # for negative seek, rewind and do positive seek
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	392 self.rewind()
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	393 count = offset - self.offset
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	394 for i in range(count // 1024):
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	395 self.read(1024)
19088 1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	396 self.read(count % 1024)
1960ceec1a00 Patch #448474: Add support for tell() and seek() to gzip.GzipFile. Martin v. Löwis <martin@v.loewis.de> parents: 17083 diff changeset	397
13743 c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	398 def readline(self, size=-1):
38237 d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	399 if size < 0:
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	400 size = sys.maxint
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	401 readsize = self.min_readsize
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	402 else:
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	403 readsize = size
38235 fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found Bob Ippolito <bob@redivi.com> parents: 38234 diff changeset	404 bufs = []
38237 d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	405 while size != 0:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	406 c = self.read(readsize)
16496 1ff9d571bac0 String method conversion. Eric S. Raymond <esr@thyrsus.com> parents: 16160 diff changeset	407 i = c.find('\n')
38237 d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	408
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	409 # We set i=size to break out of the loop under two
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	410 # conditions: 1) there's no newline, and the chunk is
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	411 # larger than size, or 2) there is a newline, but the
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	412 # resulting line would be longer than 'size'.
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	413 if (size <= i) or (i == -1 and len(c) > size):
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	414 i = size - 1
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	415
38235 fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found Bob Ippolito <bob@redivi.com> parents: 38234 diff changeset	416 if i >= 0 or c == '':
38237 d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	417 bufs.append(c[:i + 1]) # Add portion of last chunk
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	418 self._unread(c[i + 1:]) # Push back rest of chunk
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	419 break
38234 27c80058173e GzipFile.readline performance improvement (~30-40%), patch #1281707 Bob Ippolito <bob@redivi.com> parents: 35050 diff changeset	420
38235 fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found Bob Ippolito <bob@redivi.com> parents: 38234 diff changeset	421 # Append chunk to list, decrease 'size',
fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found Bob Ippolito <bob@redivi.com> parents: 38234 diff changeset	422 bufs.append(c)
fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found Bob Ippolito <bob@redivi.com> parents: 38234 diff changeset	423 size = size - len(c)
fb4f0fd7eba4 Revert gzip readline performance patch #1281707 until a more generic performance improvement can be found Bob Ippolito <bob@redivi.com> parents: 38234 diff changeset	424 readsize = min(size, readsize * 2)
38237 d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	425 if readsize > self.min_readsize:
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	426 self.min_readsize = min(readsize, self.min_readsize * 2, 512)
d710c2dba3e8 Apply revised patch for GzipFile.readline performance #1281707 Bob Ippolito <bob@redivi.com> parents: 38235 diff changeset	427 return ''.join(bufs) # Return resulting line
15868 3060cbfa01be Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 13743 diff changeset	428
13743 c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	429 def readlines(self, sizehint=0):
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	430 # Negative numbers result in reading all the lines
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	431 if sizehint <= 0:
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	432 sizehint = sys.maxint
13743 c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	433 L = []
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	434 while sizehint > 0:
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	435 line = self.readline()
26019 6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	436 if line == "":
6fa5c95408eb Related to SF patch 618135: gzip.py and files > 2G. Tim Peters <tim.peters@gmail.com> parents: 24601 diff changeset	437 break
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	438 L.append(line)
13743 c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	439 sizehint = sizehint - len(line)
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	440
c76e74abb65b SF patch #100740: Add optional size arguments to .readline() and Andrew M. Kuchling <amk@amk.ca> parents: 12619 diff changeset	441 return L
5386 43c5e1f9154e Adding Jeremy Hylton's gzip module. Guido van Rossum <guido@python.org> parents: diff changeset	442
5872 dd0e9b646616 Add the option to pass an open file object to GzipFile. This obviates Guido van Rossum <guido@python.org> parents: 5386 diff changeset	443 def writelines(self, L):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	444 for line in L:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	445 self.write(line)
7006 9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	446
22417 df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	447 def __iter__(self):
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	448 return self
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	449
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	450 def next(self):
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	451 line = self.readline()
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	452 if line:
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	453 return line
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	454 else:
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	455 raise StopIteration
df35656e7d99 Make GzipFile an iterator. Closes bug #532621. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 22216 diff changeset	456
50698 5d5d9074f4ca Issue #3860: GzipFile and BZ2File now support the context manager protocol. Antoine Pitrou <solipsis@pitrou.net> parents: 50600 diff changeset	457 def __enter__(self):
5d5d9074f4ca Issue #3860: GzipFile and BZ2File now support the context manager protocol. Antoine Pitrou <solipsis@pitrou.net> parents: 50600 diff changeset	458 if self.fileobj is None:
5d5d9074f4ca Issue #3860: GzipFile and BZ2File now support the context manager protocol. Antoine Pitrou <solipsis@pitrou.net> parents: 50600 diff changeset	459 raise ValueError("I/O operation on closed GzipFile object")
5d5d9074f4ca Issue #3860: GzipFile and BZ2File now support the context manager protocol. Antoine Pitrou <solipsis@pitrou.net> parents: 50600 diff changeset	460 return self
5d5d9074f4ca Issue #3860: GzipFile and BZ2File now support the context manager protocol. Antoine Pitrou <solipsis@pitrou.net> parents: 50600 diff changeset	461
5d5d9074f4ca Issue #3860: GzipFile and BZ2File now support the context manager protocol. Antoine Pitrou <solipsis@pitrou.net> parents: 50600 diff changeset	462 def __exit__(self, *args):
5d5d9074f4ca Issue #3860: GzipFile and BZ2File now support the context manager protocol. Antoine Pitrou <solipsis@pitrou.net> parents: 50600 diff changeset	463 self.close()
5d5d9074f4ca Issue #3860: GzipFile and BZ2File now support the context manager protocol. Antoine Pitrou <solipsis@pitrou.net> parents: 50600 diff changeset	464
7006 9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	465
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	466 def _test():
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	467 # Act like gzip; with -d, act like gunzip.
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	468 # The input file is not deleted, however, nor are any other gzip
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	469 # options or features supported.
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	470 args = sys.argv[1:]
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	471 decompress = args and args[0] == "-d"
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	472 if decompress:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	473 args = args[1:]
7006 9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	474 if not args:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	475 args = ["-"]
7006 9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	476 for arg in args:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	477 if decompress:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	478 if arg == "-":
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	479 f = GzipFile(filename="", mode="rb", fileobj=sys.stdin)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	480 g = sys.stdout
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	481 else:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	482 if arg[-3:] != ".gz":
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 31096 diff changeset	483 print "filename doesn't end in .gz:", repr(arg)
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	484 continue
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	485 f = open(arg, "rb")
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	486 g = __builtin__.open(arg[:-3], "wb")
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	487 else:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	488 if arg == "-":
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	489 f = sys.stdin
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	490 g = GzipFile(filename="", mode="wb", fileobj=sys.stdout)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	491 else:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	492 f = __builtin__.open(arg, "rb")
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	493 g = open(arg + ".gz", "wb")
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 22417 diff changeset	494 while True:
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	495 chunk = f.read(1024)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	496 if not chunk:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	497 break
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	498 g.write(chunk)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	499 if g is not sys.stdout:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	500 g.close()
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	501 if f is not sys.stdin:
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 7169 diff changeset	502 f.close()
7006 9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	503
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	504 if __name__ == '__main__':
9773d3941c96 When there's no filename, don't make one up. Guido van Rossum <guido@python.org> parents: 5872 diff changeset	505 _test()

Mercurial > cpython-lru_cache-weakref

annotate Lib/gzip.py @ 54294:dba767bc3155 legacy-trunk