Mercurial > cpython-lru_cache-weakref

annotate Lib/gzip.py @ 5817:48b94de85bf3 legacy-trunk

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Extensive changes by AMK.
author Guido van Rossum <guido@python.org>
date Thu, 17 Jul 1997 16:29:42 +0000
parents 43c5e1f9154e
children dd0e9b646616
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
5386
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
1 import time
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
2 import string
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
3 import zlib
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
4 import StringIO
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
5
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
6 # implements a python function that reads and writes a gzipped file
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
7 # the user of the file doesn't have to worry about the compression,
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
8 # but sequential access is not allowed
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
9
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
10 # based on Andrew Kuchling's minigzip.py distributed with the zlib module
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
11
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
12 FTEXT, FHCRC, FEXTRA, FNAME, FCOMMENT = 1, 2, 4, 8, 16
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
13
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
14 READ, WRITE = 1, 2
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
15
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
16 def write32(output, value):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
17 t = divmod(value, 256)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
18 b1 = chr(t[1])
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
19
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
20 t = divmod(t[0], 256)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
21 b2 = chr(t[1])
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
22
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
23 t = divmod(t[0], 256)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
24 b3 = chr(t[1])
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
25
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
26 t = divmod(t[0], 256)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
27 b4 = chr(t[1])
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
28
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
29 buf = b1 + b2 + b3 + b4
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
30 output.write(buf)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
31
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
32
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
33 def read32(input):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
34 buf = input.read(4)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
35 v = ord(buf[0])
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
36 v = v + (ord(buf[1]) << 8)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
37 v = v + (ord(buf[2]) << 16)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
38 v = v + (ord(buf[3]) << 24)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
39 return v
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
40
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
41 written = []
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
42
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
43 _py_open = open
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
44
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
45 def open(filename, mode, compresslevel=9):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
46 return GzipFile(filename, mode, compresslevel)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
47
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
48 class GzipFile:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
49
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
50 def __init__(self, filename, mode='r', compresslevel=9):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
51 if mode == 'r' or mode == 'rb':
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
52 self.mode = READ
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
53 self._init_read()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
54 self.filename = filename
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
55 self.decompress = zlib.decompressobj(-zlib.MAX_WBITS)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
56
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
57 elif mode == 'w' or mode == 'wb':
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
58 self.mode = WRITE
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
59 self._init_write(filename)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
60 self.compress = zlib.compressobj(compresslevel,
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
61 zlib.DEFLATED,
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
62 -zlib.MAX_WBITS,
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
63 zlib.DEF_MEM_LEVEL,
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
64 0)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
65 else:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
66 raise ValueError, "Mode " + mode + " not supported"
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
67
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
68 self.fileobj = _py_open(self.filename,mode)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
69
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
70 if self.mode == WRITE:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
71 self._write_gzip_header()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
72 elif self.mode == READ:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
73 self._read_gzip_header()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
74
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
75
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
76 def __repr__(self):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
77 s = repr(self.fileobj)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
78 return '<gzip ' + s[1:-1] + ' ' + hex(id(self)) + '>'
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
79
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
80 def _init_write(self, filename):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
81 if filename[-3:] != '.gz':
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
82 filename = filename + '.gz'
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
83 self.filename = filename
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
84 self.crc = zlib.crc32("")
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
85 self.size = 0
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
86 self.writebuf = []
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
87 self.bufsize = 0
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
88
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
89 def _write_gzip_header(self):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
90 self.fileobj.write('\037\213') # magic header
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
91 self.fileobj.write('\010') # compression method
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
92 self.fileobj.write(chr(FNAME))
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
93 write32(self.fileobj, int(time.time()))
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
94 self.fileobj.write('\002')
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
95 self.fileobj.write('\377')
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
96 self.fileobj.write(self.filename[:-3] + '\000')
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
97
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
98 def _init_read(self):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
99 self.crc = zlib.crc32("")
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
100 self.size = 0
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
101 self.extrabuf = ""
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
102 self.extrasize = 0
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
103
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
104 def _read_gzip_header(self):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
105 magic = self.fileobj.read(2)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
106 if magic != '\037\213':
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
107 raise RuntimeError, 'Not a gzipped file'
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
108 method = ord( self.fileobj.read(1) )
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
109 if method != 8:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
110 raise RuntimeError, 'Unknown compression method'
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
111 flag = ord( self.fileobj.read(1) )
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
112 # modtime = self.fileobj.read(4)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
113 # extraflag = self.fileobj.read(1)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
114 # os = self.fileobj.read(1)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
115 self.fileobj.read(6)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
116
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
117 if flag & FEXTRA:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
118 # Read & discard the extra field, if present
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
119 xlen=ord(self.fileobj.read(1))
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
120 xlen=xlen+256*ord(self.fileobj.read(1))
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
121 self.fileobj.read(xlen)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
122 if flag & FNAME:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
123 # Read and discard a null-terminated string containing the filename
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
124 while (1):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
125 s=self.fileobj.read(1)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
126 if s=='\000': break
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
127 if flag & FCOMMENT:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
128 # Read and discard a null-terminated string containing a comment
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
129 while (1):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
130 s=self.fileobj.read(1)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
131 if s=='\000': break
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
132 if flag & FHCRC:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
133 self.fileobj.read(2) # Read & discard the 16-bit header CRC
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
134
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
135
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
136 def write(self,data):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
137 if len(data) > 0:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
138 self.size = self.size + len(data)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
139 self.crc = zlib.crc32(data, self.crc)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
140 self.fileobj.write( self.compress.compress(data) )
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
141
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
142 def writelines(self,lines):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
143 self.write(string.join(lines))
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
144
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
145 def read(self,size=None):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
146 if self.extrasize <= 0 and self.fileobj.closed:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
147 return ''
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
148
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
149 if not size:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
150 # get the whole thing
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
151 try:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
152 while 1:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
153 self._read()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
154 except EOFError:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
155 size = self.extrasize
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
156 else:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
157 # just get some more of it
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
158 try:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
159 while size > self.extrasize:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
160 self._read()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
161 except EOFError:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
162 pass
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
163
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
164 chunk = self.extrabuf[:size]
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
165 self.extrabuf = self.extrabuf[size:]
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
166 self.extrasize = self.extrasize - size
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
167
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
168 return chunk
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
169
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
170 def _read(self):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
171 buf = self.fileobj.read(1024)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
172 if buf == "":
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
173 uncompress = self.decompress.flush()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
174 if uncompress == "":
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
175 self._read_eof()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
176 self.fileobj.close()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
177 raise EOFError, 'Reached EOF'
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
178 else:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
179 uncompress = self.decompress.decompress(buf)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
180 self.crc = zlib.crc32(uncompress, self.crc)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
181 self.extrabuf = self.extrabuf + uncompress
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
182 self.extrasize = self.extrasize + len(uncompress)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
183 self.size = self.size + len(uncompress)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
184
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
185 def _read_eof(self):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
186 # Andrew writes:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
187 ## We've read to the end of the file, so we have to rewind in order
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
188 ## to reread the 8 bytes containing the CRC and the file size. The
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
189 ## decompressor is smart and knows when to stop, so feeding it
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
190 ## extra data is harmless.
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
191 self.fileobj.seek(-8, 2)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
192 crc32 = read32(self.fileobj)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
193 isize = read32(self.fileobj)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
194 if crc32 != self.crc:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
195 self.error = "CRC check failed"
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
196 elif isize != self.size:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
197 self.error = "Incorrect length of data produced"
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
198
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
199 def close(self):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
200 if self.mode == WRITE:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
201 self.fileobj.write(self.compress.flush())
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
202 write32(self.fileobj, self.crc)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
203 write32(self.fileobj, self.size)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
204 self.fileobj.close()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
205 elif self.mode == READ:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
206 self.fileobj.close()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
207
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
208 def flush(self):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
209 self.fileobj.flush()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
210
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
211 def seek(self):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
212 raise IOError, 'Random access not allowed in gzip files'
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
213
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
214 def tell(self):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
215 raise IOError, 'I won\'t tell() you for gzip files'
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
216
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
217 def isatty(self):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
218 return 0
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
219
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
220 def readline(self):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
221 # should I bother with this
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
222 raise RuntimeError, "not implemented"
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
223
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
224 def readlines(self):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
225 # should I bother with this
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
226 raise RuntimeError, "not implemented"
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
227
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
228
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
229 class StringIOgz(GzipFile):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
230
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
231 """A StringIO substitute that reads/writes gzipped buffers."""
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
232
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
233 def __init__(self, buf=None, filename="StringIOgz"):
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
234 """Read/write mode depends on first argument.
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
235
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
236 If __init__ is passed a buffer, it will treat that as the
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
237 gzipped data and set up the StringIO for reading. Without the
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
238 initial argument, it will assume a new file for writing.
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
239
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
240 The filename argument is written in the header of buffers
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
241 opened for writing. Not sure that this is useful, but the
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
242 GzipFile code expects *some* filename."""
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
243
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
244 if buf:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
245 self.mode = READ
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
246 self._init_read()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
247 self.filename = filename
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
248 self.decompress = zlib.decompressobj(-zlib.MAX_WBITS)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
249 self.fileobj = StringIO.StringIO(buf)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
250 else:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
251 self.mode = WRITE
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
252 self._init_write(filename)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
253 self.compress = zlib.compressobj(compresslevel,
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
254 zlib.DEFLATED,
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
255 -zlib.MAX_WBITS,
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
256 zlib.DEF_MEM_LEVEL,
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
257 0)
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
258 self.fileobj = StringIO.StringIO()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
259
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
260 if self.mode == WRITE:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
261 self._write_gzip_header()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
262 elif self.mode == READ:
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
263 self._read_gzip_header()
43c5e1f9154e Adding Jeremy Hylton's gzip module.
Guido van Rossum <guido@python.org>
parents:
diff changeset
264