cpython-lru_cache-weakref: Lib/tarfile.py annotate

annotate Lib/tarfile.py @ 78069:a8aa77458e12 3.2

Add c_size_t and c_ssize_t to the ctypes summary table.

author	Antoine Pitrou <solipsis@pitrou.net>
date	Thu, 12 Jul 2012 20:31:50 +0200
parents	0adf4fd8df83
children	b7bdc0b3c2fe 38df99776901

rev	line source
59522 fcfe324663a6 convert shebang lines: python -> python3 Benjamin Peterson <benjamin@python.org> parents: 59304 diff changeset	1 #!/usr/bin/env python3
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2 #-------------------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	3 # tarfile.py
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	4 #-------------------------------------------------------------------
43760 e90065e17212 Disabled test_sys_path_with_unicode for now Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	5 # Copyright (C) 2002 Lars Gustaebel <lars@gustaebel.de>
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	6 # All rights reserved.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	7 #
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	8 # Permission is hereby granted, free of charge, to any person
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	9 # obtaining a copy of this software and associated documentation
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	10 # files (the "Software"), to deal in the Software without
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	11 # restriction, including without limitation the rights to use,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	12 # copy, modify, merge, publish, distribute, sublicense, and/or sell
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	13 # copies of the Software, and to permit persons to whom the
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	14 # Software is furnished to do so, subject to the following
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	15 # conditions:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	16 #
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	17 # The above copyright notice and this permission notice shall be
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	18 # included in all copies or substantial portions of the Software.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	19 #
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	20 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	21 # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	22 # OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	23 # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	24 # HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	25 # WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	26 # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	27 # OTHER DEALINGS IN THE SOFTWARE.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	28 #
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	29 """Read from and write to tar format archives.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	30 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	31
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	32 __version__ = "$Revision$"
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	33
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	34 version = "0.9.0"
43777 0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch. Guido van Rossum <guido@python.org> parents: 43760 diff changeset	35 __author__ = "Lars Gust\u00e4bel (lars@gustaebel.de)"
71540 3e26c9033306 Fix closes Issue11439 Remove the SVN keywords from the code as it is no longer applicable in hg. Patch Contributed by Neil Muller. Senthil Kumaran <senthil@uthcode.com> parents: 68257 diff changeset	36 __date__ = "$Date: 2011-02-25 17:42:01 +0200 (Fri, 25 Feb 2011) $"
3e26c9033306 Fix closes Issue11439 Remove the SVN keywords from the code as it is no longer applicable in hg. Patch Contributed by Neil Muller. Senthil Kumaran <senthil@uthcode.com> parents: 68257 diff changeset	37 __cvsid__ = "$Id: tarfile.py 88586 2011-02-25 15:42:01Z marc-andre.lemburg $"
43777 0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch. Guido van Rossum <guido@python.org> parents: 43760 diff changeset	38 __credits__ = "Gustavo Niemeyer, Niels Gust\u00e4bel, Richard Townsend."
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	39
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	40 #---------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	41 # Imports
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	42 #---------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	43 import sys
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	44 import os
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	45 import shutil
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	46 import stat
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	47 import errno
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	48 import time
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	49 import struct
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	50 import copy
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	51 import re
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	52
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	53 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	54 import grp, pwd
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	55 except ImportError:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	56 grp = pwd = None
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	57
62898 5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	58 # os.symlink on Windows prior to 6.0 raises NotImplementedError
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	59 symlink_exception = (AttributeError, NotImplementedError)
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	60 try:
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	61 # WindowsError (1314) will be raised if the caller does not hold the
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	62 # SeCreateSymbolicLinkPrivilege privilege
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	63 symlink_exception += (WindowsError,)
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	64 except NameError:
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	65 pass
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	66
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	67 # from tarfile import *
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	68 __all__ = ["TarFile", "TarInfo", "is_tarfile", "TarError"]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	69
44098 0889daf6d0d9 #1535: rename __builtin__ module to builtins. Georg Brandl <georg@python.org> parents: 43947 diff changeset	70 from builtins import open as _open # Since 'open' is TarFile.open
39649 7496f94754ff Fix fallout from Anna's file -> open changes. Guido van Rossum <guido@python.org> parents: 39647 diff changeset	71
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	72 #---------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	73 # tar constants
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	74 #---------------------------------------------------------
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	75 NUL = b"\0" # the null character
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	76 BLOCKSIZE = 512 # length of processing blocks
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	77 RECORDSIZE = BLOCKSIZE * 20 # length of records
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	78 GNU_MAGIC = b"ustar \0" # magic gnu tar string
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	79 POSIX_MAGIC = b"ustar\x0000" # magic posix tar string
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	80
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	81 LENGTH_NAME = 100 # maximum length of a filename
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	82 LENGTH_LINK = 100 # maximum length of a linkname
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	83 LENGTH_PREFIX = 155 # maximum length of the prefix field
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	84
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	85 REGTYPE = b"0" # regular file
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	86 AREGTYPE = b"\0" # regular file
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	87 LNKTYPE = b"1" # link (inside tarfile)
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	88 SYMTYPE = b"2" # symbolic link
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	89 CHRTYPE = b"3" # character special device
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	90 BLKTYPE = b"4" # block special device
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	91 DIRTYPE = b"5" # directory
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	92 FIFOTYPE = b"6" # fifo special device
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	93 CONTTYPE = b"7" # contiguous file
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	94
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	95 GNUTYPE_LONGNAME = b"L" # GNU tar longname
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	96 GNUTYPE_LONGLINK = b"K" # GNU tar longlink
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	97 GNUTYPE_SPARSE = b"S" # GNU tar sparse file
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	98
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	99 XHDTYPE = b"x" # POSIX.1-2001 extended header
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	100 XGLTYPE = b"g" # POSIX.1-2001 global header
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	101 SOLARIS_XHDTYPE = b"X" # Solaris extended header
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	102
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	103 USTAR_FORMAT = 0 # POSIX.1-1988 (ustar) format
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	104 GNU_FORMAT = 1 # GNU tar format
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	105 PAX_FORMAT = 2 # POSIX.1-2001 (pax) format
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	106 DEFAULT_FORMAT = GNU_FORMAT
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	107
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	108 #---------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	109 # tarfile constants
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	110 #---------------------------------------------------------
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	111 # File types that tarfile supports:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	112 SUPPORTED_TYPES = (REGTYPE, AREGTYPE, LNKTYPE,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	113 SYMTYPE, DIRTYPE, FIFOTYPE,
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	114 CONTTYPE, CHRTYPE, BLKTYPE,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	115 GNUTYPE_LONGNAME, GNUTYPE_LONGLINK,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	116 GNUTYPE_SPARSE)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	117
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	118 # File types that will be treated as a regular file.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	119 REGULAR_TYPES = (REGTYPE, AREGTYPE,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	120 CONTTYPE, GNUTYPE_SPARSE)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	121
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	122 # File types that are part of the GNU tar format.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	123 GNU_TYPES = (GNUTYPE_LONGNAME, GNUTYPE_LONGLINK,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	124 GNUTYPE_SPARSE)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	125
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	126 # Fields from a pax header that override a TarInfo attribute.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	127 PAX_FIELDS = ("path", "linkpath", "size", "mtime",
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	128 "uid", "gid", "uname", "gname")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	129
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	130 # Fields from a pax header that are affected by hdrcharset.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	131 PAX_NAME_FIELDS = {"path", "linkpath", "uname", "gname"}
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	132
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	133 # Fields in a pax header that are numbers, all other fields
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	134 # are treated as strings.
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	135 PAX_NUMBER_FIELDS = {
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	136 "atime": float,
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	137 "ctime": float,
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	138 "mtime": float,
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	139 "uid": int,
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	140 "gid": int,
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	141 "size": int
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	142 }
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	143
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	144 #---------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	145 # Bits used in the mode field, values in octal.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	146 #---------------------------------------------------------
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	147 S_IFLNK = 0o120000 # symbolic link
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	148 S_IFREG = 0o100000 # regular file
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	149 S_IFBLK = 0o060000 # block device
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	150 S_IFDIR = 0o040000 # directory
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	151 S_IFCHR = 0o020000 # character device
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	152 S_IFIFO = 0o010000 # fifo
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	153
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	154 TSUID = 0o4000 # set UID on execution
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	155 TSGID = 0o2000 # set GID on execution
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	156 TSVTX = 0o1000 # reserved
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	157
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	158 TUREAD = 0o400 # read by owner
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	159 TUWRITE = 0o200 # write by owner
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	160 TUEXEC = 0o100 # execute/search by owner
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	161 TGREAD = 0o040 # read by group
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	162 TGWRITE = 0o020 # write by group
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	163 TGEXEC = 0o010 # execute/search by group
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	164 TOREAD = 0o004 # read by other
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	165 TOWRITE = 0o002 # write by other
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	166 TOEXEC = 0o001 # execute/search by other
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	167
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	168 #---------------------------------------------------------
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	169 # initialization
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	170 #---------------------------------------------------------
62274 6eaa79be0449 Issue #8784: Set tarfile default encoding to 'utf-8' on Windows. Victor Stinner <victor.stinner@haypocalc.com> parents: 62047 diff changeset	171 if os.name in ("nt", "ce"):
6eaa79be0449 Issue #8784: Set tarfile default encoding to 'utf-8' on Windows. Victor Stinner <victor.stinner@haypocalc.com> parents: 62047 diff changeset	172 ENCODING = "utf-8"
6eaa79be0449 Issue #8784: Set tarfile default encoding to 'utf-8' on Windows. Victor Stinner <victor.stinner@haypocalc.com> parents: 62047 diff changeset	173 else:
6eaa79be0449 Issue #8784: Set tarfile default encoding to 'utf-8' on Windows. Victor Stinner <victor.stinner@haypocalc.com> parents: 62047 diff changeset	174 ENCODING = sys.getfilesystemencoding()
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	175
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	176 #---------------------------------------------------------
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	177 # Some useful functions
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	178 #---------------------------------------------------------
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	179
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	180 def stn(s, length, encoding, errors):
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	181 """Convert a string to a null-terminated bytes object.
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	182 """
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	183 s = s.encode(encoding, errors)
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	184 return s[:length] + (length - len(s)) * NUL
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	185
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	186 def nts(s, encoding, errors):
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	187 """Convert a null-terminated bytes object to a string.
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	188 """
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	189 p = s.find(b"\0")
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	190 if p != -1:
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	191 s = s[:p]
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	192 return s.decode(encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	193
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	194 def nti(s):
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	195 """Convert a number field to a python number.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	196 """
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	197 # There are two possible encodings for a number field, see
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	198 # itn() below.
72923 341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	199 if s[0] in (0o200, 0o377):
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	200 n = 0
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	201 for i in range(len(s) - 1):
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	202 n <<= 8
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	203 n += s[i + 1]
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	204 if s[0] == 0o377:
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	205 n = -(256 ** (len(s) - 1) - n)
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	206 else:
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	207 try:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	208 n = int(nts(s, "ascii", "strict") or "0", 8)
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	209 except ValueError:
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	210 raise InvalidHeaderError("invalid header")
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	211 return n
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	212
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	213 def itn(n, digits=8, format=DEFAULT_FORMAT):
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	214 """Convert a python number to a number field.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	215 """
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	216 # POSIX 1003.1-1988 requires numbers to be encoded as a string of
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	217 # octal digits followed by a null-byte, this allows values up to
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	218 # (8**(digits-1))-1. GNU tar allows storing numbers greater than
72923 341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	219 # that if necessary. A leading 0o200 or 0o377 byte indicate this
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	220 # particular encoding, the following digits-1 bytes are a big-endian
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	221 # base-256 representation. This allows values up to (256**(digits-1))-1.
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	222 # A 0o200 byte indicates a positive number, a 0o377 byte a negative
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	223 # number.
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	224 if 0 <= n < 8 ** (digits - 1):
42941 171b9f15c223 Conform to strict str/bytes distinction. Lars Gustäbel <lars@gustaebel.de> parents: 42691 diff changeset	225 s = bytes("%0*o" % (digits - 1, n), "ascii") + NUL
72923 341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	226 elif format == GNU_FORMAT and -256 (digits - 1) <= n < 256 (digits - 1):
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	227 if n >= 0:
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	228 s = bytearray([0o200])
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	229 else:
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	230 s = bytearray([0o377])
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	231 n = 256 ** digits + n
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	232
72923 341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	233 for i in range(digits - 1):
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	234 s.insert(1, n & 0o377)
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	235 n >>= 8
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	236 else:
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	237 raise ValueError("overflow in number field")
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	238
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	239 return s
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	240
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	241 def calc_chksums(buf):
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	242 """Calculate the checksum for a member's header by summing up all
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	243 characters except for the chksum field which is treated as if
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	244 it was filled with spaces. According to the GNU tar sources,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	245 some tars (Sun and NeXT) calculate chksum with signed char,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	246 which will be different if there are chars in the buffer with
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	247 the high bit set. So we calculate two checksums, unsigned and
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	248 signed.
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	249 """
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	250 unsigned_chksum = 256 + sum(struct.unpack("148B", buf[:148]) + struct.unpack("356B", buf[156:512]))
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	251 signed_chksum = 256 + sum(struct.unpack("148b", buf[:148]) + struct.unpack("356b", buf[156:512]))
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	252 return unsigned_chksum, signed_chksum
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	253
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	254 def copyfileobj(src, dst, length=None):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	255 """Copy length bytes from fileobj src to fileobj dst.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	256 If length is None, copy the entire content.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	257 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	258 if length == 0:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	259 return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	260 if length is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	261 shutil.copyfileobj(src, dst)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	262 return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	263
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	264 BUFSIZE = 16 * 1024
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	265 blocks, remainder = divmod(length, BUFSIZE)
41642 a6eb6acfe04a Merged revisions 55007-55179 via svnmerge from Guido van Rossum <guido@python.org> parents: 41575 diff changeset	266 for b in range(blocks):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	267 buf = src.read(BUFSIZE)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	268 if len(buf) < BUFSIZE:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	269 raise IOError("end of file reached")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	270 dst.write(buf)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	271
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	272 if remainder != 0:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	273 buf = src.read(remainder)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	274 if len(buf) < remainder:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	275 raise IOError("end of file reached")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	276 dst.write(buf)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	277 return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	278
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	279 filemode_table = (
33810 3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	280 ((S_IFLNK, "l"),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	281 (S_IFREG, "-"),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	282 (S_IFBLK, "b"),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	283 (S_IFDIR, "d"),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	284 (S_IFCHR, "c"),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	285 (S_IFIFO, "p")),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	286
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	287 ((TUREAD, "r"),),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	288 ((TUWRITE, "w"),),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	289 ((TUEXEC\|TSUID, "s"),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	290 (TSUID, "S"),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	291 (TUEXEC, "x")),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	292
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	293 ((TGREAD, "r"),),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	294 ((TGWRITE, "w"),),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	295 ((TGEXEC\|TSGID, "s"),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	296 (TSGID, "S"),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	297 (TGEXEC, "x")),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	298
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	299 ((TOREAD, "r"),),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	300 ((TOWRITE, "w"),),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	301 ((TOEXEC\|TSVTX, "t"),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	302 (TSVTX, "T"),
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	303 (TOEXEC, "x"))
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	304 )
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	305
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	306 def filemode(mode):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	307 """Convert a file's mode to a string of the form
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	308 -rwxrwxrwx.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	309 Used by TarFile.list()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	310 """
33810 3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	311 perm = []
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	312 for table in filemode_table:
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	313 for bit, char in table:
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	314 if mode & bit == bit:
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	315 perm.append(char)
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	316 break
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	317 else:
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	318 perm.append("-")
3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	319 return "".join(perm)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	320
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	321 class TarError(Exception):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	322 """Base exception."""
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	323 pass
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	324 class ExtractError(TarError):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	325 """General exception for extract errors."""
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	326 pass
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	327 class ReadError(TarError):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	328 """Exception for unreadble tar archives."""
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	329 pass
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	330 class CompressionError(TarError):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	331 """Exception for unavailable compression methods."""
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	332 pass
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	333 class StreamError(TarError):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	334 """Exception for unsupported operations on stream-like TarFiles."""
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	335 pass
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	336 class HeaderError(TarError):
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	337 """Base exception for header errors."""
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	338 pass
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	339 class EmptyHeaderError(HeaderError):
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	340 """Exception for empty headers."""
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	341 pass
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	342 class TruncatedHeaderError(HeaderError):
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	343 """Exception for truncated headers."""
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	344 pass
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	345 class EOFHeaderError(HeaderError):
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	346 """Exception for end of file headers."""
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	347 pass
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	348 class InvalidHeaderError(HeaderError):
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	349 """Exception for invalid headers."""
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	350 pass
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	351 class SubsequentHeaderError(HeaderError):
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	352 """Exception for missing and invalid extended headers."""
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	353 pass
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	354
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	355 #---------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	356 # internal stream interface
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	357 #---------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	358 class _LowLevelFile:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	359 """Low-level file object. Supports reading and writing.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	360 It is used instead of a regular file object for streaming
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	361 access.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	362 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	363
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	364 def __init__(self, name, mode):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	365 mode = {
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	366 "r": os.O_RDONLY,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	367 "w": os.O_WRONLY \| os.O_CREAT \| os.O_TRUNC,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	368 }[mode]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	369 if hasattr(os, "O_BINARY"):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	370 mode \|= os.O_BINARY
61092 405237b11104 Merged revisions 80618 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57736 diff changeset	371 self.fd = os.open(name, mode, 0o666)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	372
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	373 def close(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	374 os.close(self.fd)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	375
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	376 def read(self, size):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	377 return os.read(self.fd, size)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	378
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	379 def write(self, s):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	380 os.write(self.fd, s)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	381
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	382 class _Stream:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	383 """Class that serves as an adapter between TarFile and
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	384 a stream-like object. The stream-like object only
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	385 needs to have a read() or write() method and is accessed
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	386 blockwise. Use of gzip or bzip2 compression is possible.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	387 A stream-like object could be for example: sys.stdin,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	388 sys.stdout, a socket, a tape device etc.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	389
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	390 _Stream is intended to be used only internally.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	391 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	392
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	393 def __init__(self, name, mode, comptype, fileobj, bufsize):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	394 """Construct a _Stream object.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	395 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	396 self._extfileobj = True
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	397 if fileobj is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	398 fileobj = _LowLevelFile(name, mode)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	399 self._extfileobj = False
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	400
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	401 if comptype == '*':
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	402 # Enable transparent compression detection for the
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	403 # stream interface
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	404 fileobj = _StreamProxy(fileobj)
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	405 comptype = fileobj.getcomptype()
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	406
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	407 self.name = name or ""
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	408 self.mode = mode
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	409 self.comptype = comptype
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	410 self.fileobj = fileobj
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	411 self.bufsize = bufsize
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	412 self.buf = b""
40644 1c26505c1de6 Rip out 'long' and 'L'-suffixed integer literals. Guido van Rossum <guido@python.org> parents: 40587 diff changeset	413 self.pos = 0
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	414 self.closed = False
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	415
64915 0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	416 try:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	417 if comptype == "gz":
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	418 try:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	419 import zlib
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	420 except ImportError:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	421 raise CompressionError("zlib module is not available")
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	422 self.zlib = zlib
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	423 self.crc = zlib.crc32(b"")
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	424 if mode == "r":
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	425 self._init_read_gz()
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	426 else:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	427 self._init_write_gz()
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	428
64915 0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	429 if comptype == "bz2":
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	430 try:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	431 import bz2
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	432 except ImportError:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	433 raise CompressionError("bz2 module is not available")
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	434 if mode == "r":
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	435 self.dbuf = b""
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	436 self.cmp = bz2.BZ2Decompressor()
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	437 else:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	438 self.cmp = bz2.BZ2Compressor()
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	439 except:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	440 if not self._extfileobj:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	441 self.fileobj.close()
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	442 self.closed = True
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	443 raise
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	444
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	445 def __del__(self):
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	446 if hasattr(self, "closed") and not self.closed:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	447 self.close()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	448
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	449 def _init_write_gz(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	450 """Initialize for writing with gzip compression.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	451 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	452 self.cmp = self.zlib.compressobj(9, self.zlib.DEFLATED,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	453 -self.zlib.MAX_WBITS,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	454 self.zlib.DEF_MEM_LEVEL,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	455 0)
40644 1c26505c1de6 Rip out 'long' and 'L'-suffixed integer literals. Guido van Rossum <guido@python.org> parents: 40587 diff changeset	456 timestamp = struct.pack("<L", int(time.time()))
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	457 self.__write(b"\037\213\010\010" + timestamp + b"\002\377")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	458 if self.name.endswith(".gz"):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	459 self.name = self.name[:-3]
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	460 # RFC1952 says we must use ISO-8859-1 for the FNAME field.
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	461 self.__write(self.name.encode("iso-8859-1", "replace") + NUL)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	462
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	463 def write(self, s):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	464 """Write string s to the stream.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	465 """
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	466 if self.comptype == "gz":
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	467 self.crc = self.zlib.crc32(s, self.crc)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	468 self.pos += len(s)
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	469 if self.comptype != "tar":
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	470 s = self.cmp.compress(s)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	471 self.__write(s)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	472
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	473 def __write(self, s):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	474 """Write string s to the stream if a whole new block
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	475 is ready to be written.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	476 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	477 self.buf += s
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	478 while len(self.buf) > self.bufsize:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	479 self.fileobj.write(self.buf[:self.bufsize])
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	480 self.buf = self.buf[self.bufsize:]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	481
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	482 def close(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	483 """Close the _Stream object. No operation should be
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	484 done on it afterwards.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	485 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	486 if self.closed:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	487 return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	488
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	489 if self.mode == "w" and self.comptype != "tar":
33234 ab34980ffa16 Flush bz2 data even if nothing had been written so far. Fixes #1013882. Martin v. Löwis <martin@v.loewis.de> parents: 33208 diff changeset	490 self.buf += self.cmp.flush()
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	491
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	492 if self.mode == "w" and self.buf:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	493 self.fileobj.write(self.buf)
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	494 self.buf = b""
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	495 if self.comptype == "gz":
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	496 # The native zlib crc is an unsigned 32-bit integer, but
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	497 # the Python wrapper implicitly casts that to a signed C
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	498 # long. So, on a 32-bit box self.crc may "look negative",
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	499 # while the same crc on a 64-bit box may "look positive".
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	500 # To avoid irksome warnings from the `struct` module, force
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	501 # it to look positive on all boxes.
40644 1c26505c1de6 Rip out 'long' and 'L'-suffixed integer literals. Guido van Rossum <guido@python.org> parents: 40587 diff changeset	502 self.fileobj.write(struct.pack("<L", self.crc & 0xffffffff))
1c26505c1de6 Rip out 'long' and 'L'-suffixed integer literals. Guido van Rossum <guido@python.org> parents: 40587 diff changeset	503 self.fileobj.write(struct.pack("<L", self.pos & 0xffffFFFF))
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	504
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	505 if not self._extfileobj:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	506 self.fileobj.close()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	507
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	508 self.closed = True
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	509
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	510 def _init_read_gz(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	511 """Initialize for reading a gzip compressed fileobj.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	512 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	513 self.cmp = self.zlib.decompressobj(-self.zlib.MAX_WBITS)
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	514 self.dbuf = b""
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	515
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	516 # taken from gzip.GzipFile with some alterations
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	517 if self.__read(2) != b"\037\213":
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	518 raise ReadError("not a gzip file")
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	519 if self.__read(1) != b"\010":
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	520 raise CompressionError("unsupported compression method")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	521
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	522 flag = ord(self.__read(1))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	523 self.__read(6)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	524
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	525 if flag & 4:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	526 xlen = ord(self.__read(1)) + 256 * ord(self.__read(1))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	527 self.read(xlen)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	528 if flag & 8:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	529 while True:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	530 s = self.__read(1)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	531 if not s or s == NUL:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	532 break
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	533 if flag & 16:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	534 while True:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	535 s = self.__read(1)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	536 if not s or s == NUL:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	537 break
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	538 if flag & 2:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	539 self.__read(2)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	540
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	541 def tell(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	542 """Return the stream's file pointer position.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	543 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	544 return self.pos
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	545
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	546 def seek(self, pos=0):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	547 """Set the stream's file pointer to pos. Negative seeking
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	548 is forbidden.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	549 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	550 if pos - self.pos >= 0:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	551 blocks, remainder = divmod(pos - self.pos, self.bufsize)
41642 a6eb6acfe04a Merged revisions 55007-55179 via svnmerge from Guido van Rossum <guido@python.org> parents: 41575 diff changeset	552 for i in range(blocks):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	553 self.read(self.bufsize)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	554 self.read(remainder)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	555 else:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	556 raise StreamError("seeking backwards is not allowed")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	557 return self.pos
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	558
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	559 def read(self, size=None):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	560 """Return the next size number of bytes from the stream.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	561 If size is not defined, return all bytes of the stream
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	562 up to EOF.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	563 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	564 if size is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	565 t = []
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	566 while True:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	567 buf = self._read(self.bufsize)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	568 if not buf:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	569 break
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	570 t.append(buf)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	571 buf = "".join(t)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	572 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	573 buf = self._read(size)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	574 self.pos += len(buf)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	575 return buf
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	576
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	577 def _read(self, size):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	578 """Return size bytes from the stream.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	579 """
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	580 if self.comptype == "tar":
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	581 return self.__read(size)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	582
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	583 c = len(self.dbuf)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	584 while c < size:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	585 buf = self.__read(self.bufsize)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	586 if not buf:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	587 break
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	588 try:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	589 buf = self.cmp.decompress(buf)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	590 except IOError:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	591 raise ReadError("invalid compressed data")
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	592 self.dbuf += buf
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	593 c += len(buf)
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	594 buf = self.dbuf[:size]
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	595 self.dbuf = self.dbuf[size:]
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	596 return buf
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	597
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	598 def __read(self, size):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	599 """Return size bytes from stream. If internal buffer is empty,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	600 read another block from the stream.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	601 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	602 c = len(self.buf)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	603 while c < size:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	604 buf = self.fileobj.read(self.bufsize)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	605 if not buf:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	606 break
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	607 self.buf += buf
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	608 c += len(buf)
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	609 buf = self.buf[:size]
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	610 self.buf = self.buf[size:]
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	611 return buf
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	612 # class _Stream
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	613
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	614 class _StreamProxy(object):
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	615 """Small proxy class that enables transparent compression
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	616 detection for the Stream interface (mode 'r\|*').
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	617 """
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	618
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	619 def __init__(self, fileobj):
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	620 self.fileobj = fileobj
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	621 self.buf = self.fileobj.read(BLOCKSIZE)
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	622
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	623 def read(self, size):
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	624 self.read = self.fileobj.read
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	625 return self.buf
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	626
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	627 def getcomptype(self):
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	628 if self.buf.startswith(b"\037\213\010"):
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	629 return "gz"
73868 80876df8adce Correctly detect bzip2 compressed streams with blocksizes other than 900k. Lars Gustäbel <lars@gustaebel.de> parents: 72923 diff changeset	630 if self.buf[0:3] == b"BZh" and self.buf[4:10] == b"1AY&SY":
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	631 return "bz2"
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	632 return "tar"
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	633
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	634 def close(self):
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	635 self.fileobj.close()
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	636 # class StreamProxy
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	637
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	638 class _BZ2Proxy(object):
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	639 """Small proxy class that enables external file object
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	640 support for "r:bz2" and "w:bz2" modes. This is actually
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	641 a workaround for a limitation in bz2 module's BZ2File
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	642 class which (unlike gzip.GzipFile) has no support for
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	643 a file object argument.
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	644 """
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	645
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	646 blocksize = 16 * 1024
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	647
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	648 def __init__(self, fileobj, mode):
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	649 self.fileobj = fileobj
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	650 self.mode = mode
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	651 self.name = getattr(self.fileobj, "name", None)
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	652 self.init()
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	653
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	654 def init(self):
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	655 import bz2
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	656 self.pos = 0
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	657 if self.mode == "r":
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	658 self.bz2obj = bz2.BZ2Decompressor()
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	659 self.fileobj.seek(0)
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	660 self.buf = b""
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	661 else:
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	662 self.bz2obj = bz2.BZ2Compressor()
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	663
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	664 def read(self, size):
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	665 x = len(self.buf)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	666 while x < size:
52269 095d874a3054 Merged revisions 70523 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 51776 diff changeset	667 raw = self.fileobj.read(self.blocksize)
095d874a3054 Merged revisions 70523 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 51776 diff changeset	668 if not raw:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	669 break
52269 095d874a3054 Merged revisions 70523 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 51776 diff changeset	670 data = self.bz2obj.decompress(raw)
095d874a3054 Merged revisions 70523 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 51776 diff changeset	671 self.buf += data
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	672 x += len(data)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	673
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	674 buf = self.buf[:size]
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	675 self.buf = self.buf[size:]
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	676 self.pos += len(buf)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	677 return buf
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	678
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	679 def seek(self, pos):
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	680 if pos < self.pos:
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	681 self.init()
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	682 self.read(pos - self.pos)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	683
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	684 def tell(self):
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	685 return self.pos
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	686
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	687 def write(self, data):
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	688 self.pos += len(data)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	689 raw = self.bz2obj.compress(data)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	690 self.fileobj.write(raw)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	691
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	692 def close(self):
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	693 if self.mode == "w":
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	694 raw = self.bz2obj.flush()
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	695 self.fileobj.write(raw)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	696 # class _BZ2Proxy
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	697
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	698 #------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	699 # Extraction file object
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	700 #------------------------
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	701 class _FileInFile(object):
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	702 """A thin wrapper around an existing file object that
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	703 provides a part of its data as an individual file
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	704 object.
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	705 """
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	706
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	707 def __init__(self, fileobj, offset, size, blockinfo=None):
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	708 self.fileobj = fileobj
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	709 self.offset = offset
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	710 self.size = size
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	711 self.position = 0
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	712
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	713 if blockinfo is None:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	714 blockinfo = [(0, size)]
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	715
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	716 # Construct a map with data and zero blocks.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	717 self.map_index = 0
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	718 self.map = []
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	719 lastpos = 0
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	720 realpos = self.offset
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	721 for offset, size in blockinfo:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	722 if offset > lastpos:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	723 self.map.append((False, lastpos, offset, None))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	724 self.map.append((True, offset, offset + size, realpos))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	725 realpos += size
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	726 lastpos = offset + size
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	727 if lastpos < self.size:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	728 self.map.append((False, lastpos, self.size, None))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	729
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	730 def seekable(self):
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	731 if not hasattr(self.fileobj, "seekable"):
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	732 # XXX gzip.GzipFile and bz2.BZ2File
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	733 return True
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	734 return self.fileobj.seekable()
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	735
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	736 def tell(self):
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	737 """Return the current file position.
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	738 """
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	739 return self.position
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	740
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	741 def seek(self, position):
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	742 """Seek to a position in the file.
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	743 """
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	744 self.position = position
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	745
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	746 def read(self, size=None):
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	747 """Read data from the file.
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	748 """
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	749 if size is None:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	750 size = self.size - self.position
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	751 else:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	752 size = min(size, self.size - self.position)
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	753
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	754 buf = b""
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	755 while size > 0:
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	756 while True:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	757 data, start, stop, offset = self.map[self.map_index]
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	758 if start <= self.position < stop:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	759 break
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	760 else:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	761 self.map_index += 1
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	762 if self.map_index == len(self.map):
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	763 self.map_index = 0
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	764 length = min(size, stop - self.position)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	765 if data:
68117 2325fe36aa34 Issue #11224: Improved sparse file read support (r85916) introduced a Lars Gustäbel <lars@gustaebel.de> parents: 67771 diff changeset	766 self.fileobj.seek(offset + (self.position - start))
2325fe36aa34 Issue #11224: Improved sparse file read support (r85916) introduced a Lars Gustäbel <lars@gustaebel.de> parents: 67771 diff changeset	767 buf += self.fileobj.read(length)
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	768 else:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	769 buf += NUL * length
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	770 size -= length
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	771 self.position += length
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	772 return buf
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	773 #class _FileInFile
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	774
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	775
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	776 class ExFileObject(object):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	777 """File-like object for reading an archive member.
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	778 Is returned by TarFile.extractfile().
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	779 """
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	780 blocksize = 1024
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	781
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	782 def __init__(self, tarfile, tarinfo):
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	783 self.fileobj = _FileInFile(tarfile.fileobj,
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	784 tarinfo.offset_data,
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	785 tarinfo.size,
46441 72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	786 tarinfo.sparse)
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	787 self.name = tarinfo.name
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	788 self.mode = "r"
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	789 self.closed = False
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	790 self.size = tarinfo.size
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	791
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	792 self.position = 0
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	793 self.buffer = b""
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	794
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	795 def readable(self):
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	796 return True
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	797
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	798 def writable(self):
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	799 return False
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	800
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	801 def seekable(self):
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	802 return self.fileobj.seekable()
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	803
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	804 def read(self, size=None):
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	805 """Read at most size bytes from the file. If size is not
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	806 present or None, read all data until EOF is reached.
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	807 """
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	808 if self.closed:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	809 raise ValueError("I/O operation on closed file")
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	810
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	811 buf = b""
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	812 if self.buffer:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	813 if size is None:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	814 buf = self.buffer
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	815 self.buffer = b""
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	816 else:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	817 buf = self.buffer[:size]
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	818 self.buffer = self.buffer[size:]
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	819
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	820 if size is None:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	821 buf += self.fileobj.read()
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	822 else:
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	823 buf += self.fileobj.read(size - len(buf))
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	824
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	825 self.position += len(buf)
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	826 return buf
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	827
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	828 # XXX TextIOWrapper uses the read1() method.
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	829 read1 = read
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	830
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	831 def readline(self, size=-1):
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	832 """Read one entire line from the file. If size is present
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	833 and non-negative, return a string with at most that
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	834 size, which may be an incomplete line.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	835 """
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	836 if self.closed:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	837 raise ValueError("I/O operation on closed file")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	838
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	839 pos = self.buffer.find(b"\n") + 1
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	840 if pos == 0:
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	841 # no newline found.
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	842 while True:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	843 buf = self.fileobj.read(self.blocksize)
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	844 self.buffer += buf
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	845 if not buf or b"\n" in buf:
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	846 pos = self.buffer.find(b"\n") + 1
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	847 if pos == 0:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	848 # no newline found.
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	849 pos = len(self.buffer)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	850 break
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	851
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	852 if size != -1:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	853 pos = min(size, pos)
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	854
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	855 buf = self.buffer[:pos]
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	856 self.buffer = self.buffer[pos:]
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	857 self.position += len(buf)
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	858 return buf
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	859
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	860 def readlines(self):
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	861 """Return a list with all remaining lines.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	862 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	863 result = []
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	864 while True:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	865 line = self.readline()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	866 if not line: break
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	867 result.append(line)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	868 return result
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	869
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	870 def tell(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	871 """Return the current file position.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	872 """
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	873 if self.closed:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	874 raise ValueError("I/O operation on closed file")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	875
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	876 return self.position
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	877
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	878 def seek(self, pos, whence=os.SEEK_SET):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	879 """Seek to a position in the file.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	880 """
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	881 if self.closed:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	882 raise ValueError("I/O operation on closed file")
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	883
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	884 if whence == os.SEEK_SET:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	885 self.position = min(max(pos, 0), self.size)
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	886 elif whence == os.SEEK_CUR:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	887 if pos < 0:
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	888 self.position = max(self.position + pos, 0)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	889 else:
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	890 self.position = min(self.position + pos, self.size)
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	891 elif whence == os.SEEK_END:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	892 self.position = max(min(self.size + pos, self.size), 0)
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	893 else:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	894 raise ValueError("Invalid argument")
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	895
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	896 self.buffer = b""
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	897 self.fileobj.seek(self.position)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	898
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	899 def close(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	900 """Close the file object.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	901 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	902 self.closed = True
34649 13a950a3624b Patch #1107973: tarfile.ExFileObject iterators. Martin v. Löwis <martin@v.loewis.de> parents: 34421 diff changeset	903
13a950a3624b Patch #1107973: tarfile.ExFileObject iterators. Martin v. Löwis <martin@v.loewis.de> parents: 34421 diff changeset	904 def __iter__(self):
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	905 """Get an iterator over the file's lines.
34649 13a950a3624b Patch #1107973: tarfile.ExFileObject iterators. Martin v. Löwis <martin@v.loewis.de> parents: 34421 diff changeset	906 """
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	907 while True:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	908 line = self.readline()
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	909 if not line:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	910 break
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	911 yield line
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	912 #class ExFileObject
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	913
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	914 #------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	915 # Exported Classes
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	916 #------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	917 class TarInfo(object):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	918 """Informational class which holds the details about an
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	919 archive member given by a tar header block.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	920 TarInfo objects are returned by TarFile.getmember(),
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	921 TarFile.getmembers() and TarFile.gettarinfo() and are
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	922 usually created internally.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	923 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	924
46441 72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	925 __slots__ = ("name", "mode", "uid", "gid", "size", "mtime",
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	926 "chksum", "type", "linkname", "uname", "gname",
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	927 "devmajor", "devminor",
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	928 "offset", "offset_data", "pax_headers", "sparse",
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	929 "tarfile", "_sparse_structs", "_link_target")
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	930
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	931 def __init__(self, name=""):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	932 """Construct a TarInfo object. name is the optional name
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	933 of the member.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	934 """
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	935 self.name = name # member name
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	936 self.mode = 0o644 # file permissions
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	937 self.uid = 0 # user id
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	938 self.gid = 0 # group id
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	939 self.size = 0 # file size
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	940 self.mtime = 0 # modification time
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	941 self.chksum = 0 # header checksum
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	942 self.type = REGTYPE # member type
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	943 self.linkname = "" # link name
65085 6af7b4102d77 Merged revisions 85211 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62043 diff changeset	944 self.uname = "" # user name
6af7b4102d77 Merged revisions 85211 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62043 diff changeset	945 self.gname = "" # group name
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	946 self.devmajor = 0 # device major number
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	947 self.devminor = 0 # device minor number
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	948
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	949 self.offset = 0 # the tar header starts here
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	950 self.offset_data = 0 # the file's data starts here
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	951
46441 72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	952 self.sparse = None # sparse member information
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	953 self.pax_headers = {} # pax header information
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	954
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	955 # In pax headers the "name" and "linkname" field are called
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	956 # "path" and "linkpath".
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	957 def _getpath(self):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	958 return self.name
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	959 def _setpath(self, name):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	960 self.name = name
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	961 path = property(_getpath, _setpath)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	962
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	963 def _getlinkpath(self):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	964 return self.linkname
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	965 def _setlinkpath(self, linkname):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	966 self.linkname = linkname
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	967 linkpath = property(_getlinkpath, _setlinkpath)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	968
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	969 def __repr__(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	970 return "<%s %r at %#x>" % (self.__class__.__name__,self.name,id(self))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	971
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	972 def get_info(self):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	973 """Return the TarInfo's attributes as a dictionary.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	974 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	975 info = {
55748 9a78e3d90617 Merged revisions 74571 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	976 "name": self.name,
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	977 "mode": self.mode & 0o7777,
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	978 "uid": self.uid,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	979 "gid": self.gid,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	980 "size": self.size,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	981 "mtime": self.mtime,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	982 "chksum": self.chksum,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	983 "type": self.type,
55748 9a78e3d90617 Merged revisions 74571 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	984 "linkname": self.linkname,
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	985 "uname": self.uname,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	986 "gname": self.gname,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	987 "devmajor": self.devmajor,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	988 "devminor": self.devminor
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	989 }
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	990
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	991 if info["type"] == DIRTYPE and not info["name"].endswith("/"):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	992 info["name"] += "/"
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	993
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	994 return info
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	995
61278 88466ef1f0cd Issue #8390: tarfile uses surrogateespace as the default error handler Victor Stinner <victor.stinner@haypocalc.com> parents: 61259 diff changeset	996 def tobuf(self, format=DEFAULT_FORMAT, encoding=ENCODING, errors="surrogateescape"):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	997 """Return a tar header as a string of 512 byte blocks.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	998 """
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	999 info = self.get_info()
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1000
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1001 if format == USTAR_FORMAT:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1002 return self.create_ustar_header(info, encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1003 elif format == GNU_FORMAT:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1004 return self.create_gnu_header(info, encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1005 elif format == PAX_FORMAT:
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1006 return self.create_pax_header(info, encoding)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1007 else:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1008 raise ValueError("invalid format")
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1009
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1010 def create_ustar_header(self, info, encoding, errors):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1011 """Return the object as a ustar header block.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1012 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1013 info["magic"] = POSIX_MAGIC
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1014
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1015 if len(info["linkname"]) > LENGTH_LINK:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1016 raise ValueError("linkname is too long")
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1017
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1018 if len(info["name"]) > LENGTH_NAME:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1019 info["prefix"], info["name"] = self._posix_split_name(info["name"])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1020
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1021 return self._create_header(info, USTAR_FORMAT, encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1022
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1023 def create_gnu_header(self, info, encoding, errors):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1024 """Return the object as a GNU header block sequence.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1025 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1026 info["magic"] = GNU_MAGIC
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1027
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1028 buf = b""
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1029 if len(info["linkname"]) > LENGTH_LINK:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1030 buf += self._create_gnu_long_header(info["linkname"], GNUTYPE_LONGLINK, encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1031
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1032 if len(info["name"]) > LENGTH_NAME:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1033 buf += self._create_gnu_long_header(info["name"], GNUTYPE_LONGNAME, encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1034
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1035 return buf + self._create_header(info, GNU_FORMAT, encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1036
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1037 def create_pax_header(self, info, encoding):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1038 """Return the object as a ustar header block. If it cannot be
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1039 represented this way, prepend a pax extended header sequence
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1040 with supplement information.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1041 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1042 info["magic"] = POSIX_MAGIC
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1043 pax_headers = self.pax_headers.copy()
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1044
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1045 # Test string fields for values that exceed the field length or cannot
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1046 # be represented in ASCII encoding.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1047 for name, hname, length in (
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1048 ("name", "path", LENGTH_NAME), ("linkname", "linkpath", LENGTH_LINK),
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1049 ("uname", "uname", 32), ("gname", "gname", 32)):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1050
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1051 if hname in pax_headers:
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1052 # The pax header has priority.
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1053 continue
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1054
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1055 # Try to encode the string as ASCII.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1056 try:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1057 info[name].encode("ascii", "strict")
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1058 except UnicodeEncodeError:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1059 pax_headers[hname] = info[name]
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1060 continue
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1061
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1062 if len(info[name]) > length:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1063 pax_headers[hname] = info[name]
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1064
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1065 # Test number fields for values that exceed the field limit or values
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1066 # that like to be stored as float.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1067 for name, digits in (("uid", 8), ("gid", 8), ("size", 12), ("mtime", 12)):
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1068 if name in pax_headers:
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1069 # The pax header has priority. Avoid overflow.
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1070 info[name] = 0
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1071 continue
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1072
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1073 val = info[name]
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1074 if not 0 <= val < 8 ** (digits - 1) or isinstance(val, float):
41575 beacf40b5960 Rip out all the u"..." literals and calls to unicode(). Guido van Rossum <guido@python.org> parents: 41554 diff changeset	1075 pax_headers[name] = str(val)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1076 info[name] = 0
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1077
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1078 # Create a pax extended header if necessary.
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1079 if pax_headers:
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1080 buf = self._create_pax_generic_header(pax_headers, XHDTYPE, encoding)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1081 else:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1082 buf = b""
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1083
42691 5bf68fc73923 Fall back to 'ascii' encoding if sys.getfilesystemencoding() returns Lars Gustäbel <lars@gustaebel.de> parents: 42475 diff changeset	1084 return buf + self._create_header(info, USTAR_FORMAT, "ascii", "replace")
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1085
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1086 @classmethod
42691 5bf68fc73923 Fall back to 'ascii' encoding if sys.getfilesystemencoding() returns Lars Gustäbel <lars@gustaebel.de> parents: 42475 diff changeset	1087 def create_pax_global_header(cls, pax_headers):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1088 """Return the object as a pax global header block sequence.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1089 """
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1090 return cls._create_pax_generic_header(pax_headers, XGLTYPE, "utf8")
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1091
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1092 def _posix_split_name(self, name):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1093 """Split a name longer than 100 chars into a prefix
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1094 and a name part.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1095 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1096 prefix = name[:LENGTH_PREFIX + 1]
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1097 while prefix and prefix[-1] != "/":
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1098 prefix = prefix[:-1]
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1099
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1100 name = name[len(prefix):]
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1101 prefix = prefix[:-1]
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1102
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1103 if not prefix or len(name) > LENGTH_NAME:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1104 raise ValueError("name is too long")
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1105 return prefix, name
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1106
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1107 @staticmethod
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1108 def _create_header(info, format, encoding, errors):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1109 """Return a header block. info is a dictionary with file
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1110 information, format must be one of the *_FORMAT constants.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1111 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1112 parts = [
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1113 stn(info.get("name", ""), 100, encoding, errors),
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	1114 itn(info.get("mode", 0) & 0o7777, 8, format),
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1115 itn(info.get("uid", 0), 8, format),
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1116 itn(info.get("gid", 0), 8, format),
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1117 itn(info.get("size", 0), 12, format),
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1118 itn(info.get("mtime", 0), 12, format),
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1119 b" ", # checksum field
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1120 info.get("type", REGTYPE),
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1121 stn(info.get("linkname", ""), 100, encoding, errors),
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1122 info.get("magic", POSIX_MAGIC),
65085 6af7b4102d77 Merged revisions 85211 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62043 diff changeset	1123 stn(info.get("uname", ""), 32, encoding, errors),
6af7b4102d77 Merged revisions 85211 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62043 diff changeset	1124 stn(info.get("gname", ""), 32, encoding, errors),
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1125 itn(info.get("devmajor", 0), 8, format),
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1126 itn(info.get("devminor", 0), 8, format),
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1127 stn(info.get("prefix", ""), 155, encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1128 ]
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1129
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1130 buf = struct.pack("%ds" % BLOCKSIZE, b"".join(parts))
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1131 chksum = calc_chksums(buf[-BLOCKSIZE:])[0]
42941 171b9f15c223 Conform to strict str/bytes distinction. Lars Gustäbel <lars@gustaebel.de> parents: 42691 diff changeset	1132 buf = buf[:-364] + bytes("%06o\0" % chksum, "ascii") + buf[-357:]
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1133 return buf
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1134
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1135 @staticmethod
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1136 def _create_payload(payload):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1137 """Return the string payload filled with zero bytes
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1138 up to the next 512 byte border.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1139 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1140 blocks, remainder = divmod(len(payload), BLOCKSIZE)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1141 if remainder > 0:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1142 payload += (BLOCKSIZE - remainder) * NUL
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1143 return payload
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1144
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1145 @classmethod
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1146 def _create_gnu_long_header(cls, name, type, encoding, errors):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1147 """Return a GNUTYPE_LONGNAME or GNUTYPE_LONGLINK sequence
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1148 for name.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1149 """
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1150 name = name.encode(encoding, errors) + NUL
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1151
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1152 info = {}
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1153 info["name"] = "././@LongLink"
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1154 info["type"] = type
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1155 info["size"] = len(name)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1156 info["magic"] = GNU_MAGIC
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1157
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1158 # create extended header + name blocks.
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1159 return cls._create_header(info, USTAR_FORMAT, encoding, errors) + \
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1160 cls._create_payload(name)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1161
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1162 @classmethod
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1163 def _create_pax_generic_header(cls, pax_headers, type, encoding):
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1164 """Return a POSIX.1-2008 extended or global header sequence
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1165 that contains a list of keyword, value pairs. The values
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1166 must be strings.
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1167 """
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1168 # Check if one of the fields contains surrogate characters and thereby
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1169 # forces hdrcharset=BINARY, see _proc_pax() for more information.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1170 binary = False
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1171 for keyword, value in pax_headers.items():
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1172 try:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1173 value.encode("utf8", "strict")
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1174 except UnicodeEncodeError:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1175 binary = True
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1176 break
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1177
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1178 records = b""
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1179 if binary:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1180 # Put the hdrcharset field at the beginning of the header.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1181 records += b"21 hdrcharset=BINARY\n"
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1182
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1183 for keyword, value in pax_headers.items():
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1184 keyword = keyword.encode("utf8")
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1185 if binary:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1186 # Try to restore the original byte representation of `value'.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1187 # Needless to say, that the encoding must match the string.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1188 value = value.encode(encoding, "surrogateescape")
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1189 else:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1190 value = value.encode("utf8")
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1191
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1192 l = len(keyword) + len(value) + 3 # ' ' + '=' + '\n'
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1193 n = p = 0
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1194 while True:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1195 n = l + len(str(p))
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1196 if n == p:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1197 break
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1198 p = n
42941 171b9f15c223 Conform to strict str/bytes distinction. Lars Gustäbel <lars@gustaebel.de> parents: 42691 diff changeset	1199 records += bytes(str(p), "ascii") + b" " + keyword + b"=" + value + b"\n"
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1200
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1201 # We use a hardcoded "././@PaxHeader" name like star does
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1202 # instead of the one that POSIX recommends.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1203 info = {}
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1204 info["name"] = "././@PaxHeader"
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1205 info["type"] = type
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1206 info["size"] = len(records)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1207 info["magic"] = POSIX_MAGIC
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1208
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1209 # Create pax header + record blocks.
42691 5bf68fc73923 Fall back to 'ascii' encoding if sys.getfilesystemencoding() returns Lars Gustäbel <lars@gustaebel.de> parents: 42475 diff changeset	1210 return cls._create_header(info, USTAR_FORMAT, "ascii", "replace") + \
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1211 cls._create_payload(records)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1212
34421 3316aa540e68 Use decorators. Guido van Rossum <guido@python.org> parents: 34293 diff changeset	1213 @classmethod
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1214 def frombuf(cls, buf, encoding, errors):
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1215 """Construct a TarInfo object from a 512 byte bytes object.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1216 """
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1217 if len(buf) == 0:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1218 raise EmptyHeaderError("empty header")
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1219 if len(buf) != BLOCKSIZE:
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1220 raise TruncatedHeaderError("truncated header")
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1221 if buf.count(NUL) == BLOCKSIZE:
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1222 raise EOFHeaderError("end of file header")
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1223
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1224 chksum = nti(buf[148:156])
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1225 if chksum not in calc_chksums(buf):
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1226 raise InvalidHeaderError("bad checksum")
32770 a3b1d6c99e46 SF #918101, allow files >= 8 GB using GNU extension Neal Norwitz <nnorwitz@gmail.com> parents: 32769 diff changeset	1227
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1228 obj = cls()
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1229 obj.name = nts(buf[0:100], encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1230 obj.mode = nti(buf[100:108])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1231 obj.uid = nti(buf[108:116])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1232 obj.gid = nti(buf[116:124])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1233 obj.size = nti(buf[124:136])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1234 obj.mtime = nti(buf[136:148])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1235 obj.chksum = chksum
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1236 obj.type = buf[156:157]
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1237 obj.linkname = nts(buf[157:257], encoding, errors)
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1238 obj.uname = nts(buf[265:297], encoding, errors)
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1239 obj.gname = nts(buf[297:329], encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1240 obj.devmajor = nti(buf[329:337])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1241 obj.devminor = nti(buf[337:345])
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1242 prefix = nts(buf[345:500], encoding, errors)
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1243
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1244 # Old V7 tar format represents a directory as a regular
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1245 # file with a trailing slash.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1246 if obj.type == AREGTYPE and obj.name.endswith("/"):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1247 obj.type = DIRTYPE
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1248
46441 72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1249 # The old GNU sparse format occupies some of the unused
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1250 # space in the buffer for up to 4 sparse structures.
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1251 # Save the them for later processing in _proc_sparse().
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1252 if obj.type == GNUTYPE_SPARSE:
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1253 pos = 386
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1254 structs = []
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1255 for i in range(4):
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1256 try:
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1257 offset = nti(buf[pos:pos + 12])
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1258 numbytes = nti(buf[pos + 12:pos + 24])
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1259 except ValueError:
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1260 break
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1261 structs.append((offset, numbytes))
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1262 pos += 24
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1263 isextended = bool(buf[482])
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1264 origsize = nti(buf[483:495])
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1265 obj._sparse_structs = (structs, isextended, origsize)
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1266
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1267 # Remove redundant slashes from directories.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1268 if obj.isdir():
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1269 obj.name = obj.name.rstrip("/")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1270
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1271 # Reconstruct a ustar longname.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1272 if prefix and obj.type not in GNU_TYPES:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1273 obj.name = prefix + "/" + obj.name
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1274 return obj
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1275
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1276 @classmethod
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1277 def fromtarfile(cls, tarfile):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1278 """Return the next TarInfo object from TarFile object
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1279 tarfile.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1280 """
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1281 buf = tarfile.fileobj.read(BLOCKSIZE)
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1282 obj = cls.frombuf(buf, tarfile.encoding, tarfile.errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1283 obj.offset = tarfile.fileobj.tell() - BLOCKSIZE
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1284 return obj._proc_member(tarfile)
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1285
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1286 #--------------------------------------------------------------------------
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1287 # The following are methods that are called depending on the type of a
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1288 # member. The entry point is _proc_member() which can be overridden in a
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1289 # subclass to add custom _proc_() methods. A _proc_() method MUST
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1290 # implement the following
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1291 # operations:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1292 # 1. Set self.offset_data to the position where the data blocks begin,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1293 # if there is data that follows.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1294 # 2. Set tarfile.offset to the position where the next member's header will
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1295 # begin.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1296 # 3. Return self or another valid TarInfo object.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1297 def _proc_member(self, tarfile):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1298 """Choose the right processing method depending on
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1299 the type and call it.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1300 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1301 if self.type in (GNUTYPE_LONGNAME, GNUTYPE_LONGLINK):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1302 return self._proc_gnulong(tarfile)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1303 elif self.type == GNUTYPE_SPARSE:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1304 return self._proc_sparse(tarfile)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1305 elif self.type in (XHDTYPE, XGLTYPE, SOLARIS_XHDTYPE):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1306 return self._proc_pax(tarfile)
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1307 else:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1308 return self._proc_builtin(tarfile)
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1309
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1310 def _proc_builtin(self, tarfile):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1311 """Process a builtin type or an unknown type which
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1312 will be treated as a regular file.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1313 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1314 self.offset_data = tarfile.fileobj.tell()
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1315 offset = self.offset_data
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1316 if self.isreg() or self.type not in SUPPORTED_TYPES:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1317 # Skip the following data blocks.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1318 offset += self._block(self.size)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1319 tarfile.offset = offset
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1320
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1321 # Patch the TarInfo object with saved global
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1322 # header information.
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1323 self._apply_pax_info(tarfile.pax_headers, tarfile.encoding, tarfile.errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1324
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1325 return self
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1326
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1327 def _proc_gnulong(self, tarfile):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1328 """Process the blocks that hold a GNU longname
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1329 or longlink member.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1330 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1331 buf = tarfile.fileobj.read(self._block(self.size))
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1332
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1333 # Fetch the next header and process it.
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1334 try:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1335 next = self.fromtarfile(tarfile)
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1336 except HeaderError:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1337 raise SubsequentHeaderError("missing or bad subsequent header")
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1338
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1339 # Patch the TarInfo object from the next header with
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1340 # the longname information.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1341 next.offset = self.offset
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1342 if self.type == GNUTYPE_LONGNAME:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1343 next.name = nts(buf, tarfile.encoding, tarfile.errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1344 elif self.type == GNUTYPE_LONGLINK:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1345 next.linkname = nts(buf, tarfile.encoding, tarfile.errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1346
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1347 return next
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1348
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1349 def _proc_sparse(self, tarfile):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1350 """Process a GNU sparse header plus extra headers.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1351 """
46441 72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1352 # We already collected some sparse structures in frombuf().
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1353 structs, isextended, origsize = self._sparse_structs
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1354 del self._sparse_structs
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1355
46441 72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1356 # Collect sparse structures from extended header blocks.
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1357 while isextended:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1358 buf = tarfile.fileobj.read(BLOCKSIZE)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1359 pos = 0
41642 a6eb6acfe04a Merged revisions 55007-55179 via svnmerge from Guido van Rossum <guido@python.org> parents: 41575 diff changeset	1360 for i in range(21):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1361 try:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1362 offset = nti(buf[pos:pos + 12])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1363 numbytes = nti(buf[pos + 12:pos + 24])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1364 except ValueError:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1365 break
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1366 if offset and numbytes:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1367 structs.append((offset, numbytes))
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1368 pos += 24
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1369 isextended = bool(buf[504])
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1370 self.sparse = structs
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1371
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1372 self.offset_data = tarfile.fileobj.tell()
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1373 tarfile.offset = self.offset_data + self._block(self.size)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1374 self.size = origsize
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1375 return self
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1376
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1377 def _proc_pax(self, tarfile):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1378 """Process an extended or global header as described in
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1379 POSIX.1-2008.
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1380 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1381 # Read the header information.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1382 buf = tarfile.fileobj.read(self._block(self.size))
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1383
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1384 # A pax header stores supplemental information for either
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1385 # the following file (extended) or all following files
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1386 # (global).
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1387 if self.type == XGLTYPE:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1388 pax_headers = tarfile.pax_headers
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1389 else:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1390 pax_headers = tarfile.pax_headers.copy()
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1391
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1392 # Check if the pax header contains a hdrcharset field. This tells us
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1393 # the encoding of the path, linkpath, uname and gname fields. Normally,
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1394 # these fields are UTF-8 encoded but since POSIX.1-2008 tar
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1395 # implementations are allowed to store them as raw binary strings if
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1396 # the translation to UTF-8 fails.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1397 match = re.search(br"\d+ hdrcharset=([^\n]+)\n", buf)
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1398 if match is not None:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1399 pax_headers["hdrcharset"] = match.group(1).decode("utf8")
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1400
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1401 # For the time being, we don't care about anything other than "BINARY".
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1402 # The only other value that is currently allowed by the standard is
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1403 # "ISO-IR 10646 2000 UTF-8" in other words UTF-8.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1404 hdrcharset = pax_headers.get("hdrcharset")
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1405 if hdrcharset == "BINARY":
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1406 encoding = tarfile.encoding
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1407 else:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1408 encoding = "utf8"
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1409
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1410 # Parse pax header information. A record looks like that:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1411 # "%d %s=%s\n" % (length, keyword, value). length is the size
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1412 # of the complete record including the length field itself and
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1413 # the newline. keyword and value are both UTF-8 encoded strings.
48769 83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48644 diff changeset	1414 regex = re.compile(br"(\d+) ([^=]+)=")
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1415 pos = 0
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1416 while True:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1417 match = regex.match(buf, pos)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1418 if not match:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1419 break
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1420
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1421 length, keyword = match.groups()
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1422 length = int(length)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1423 value = buf[match.end(2) + 1:match.start(1) + length - 1]
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1424
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1425 # Normally, we could just use "utf8" as the encoding and "strict"
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1426 # as the error handler, but we better not take the risk. For
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1427 # example, GNU tar <= 1.23 is known to store filenames it cannot
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1428 # translate to UTF-8 as raw strings (unfortunately without a
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1429 # hdrcharset=BINARY header).
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1430 # We first try the strict standard encoding, and if that fails we
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1431 # fall back on the user's encoding and error handler.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1432 keyword = self._decode_pax_field(keyword, "utf8", "utf8",
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1433 tarfile.errors)
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1434 if keyword in PAX_NAME_FIELDS:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1435 value = self._decode_pax_field(value, encoding, tarfile.encoding,
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1436 tarfile.errors)
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1437 else:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1438 value = self._decode_pax_field(value, "utf8", "utf8",
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1439 tarfile.errors)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1440
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1441 pax_headers[keyword] = value
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1442 pos += length
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1443
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1444 # Fetch the next header.
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1445 try:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1446 next = self.fromtarfile(tarfile)
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1447 except HeaderError:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1448 raise SubsequentHeaderError("missing or bad subsequent header")
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1449
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1450 # Process GNU sparse information.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1451 if "GNU.sparse.map" in pax_headers:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1452 # GNU extended sparse format version 0.1.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1453 self._proc_gnusparse_01(next, pax_headers)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1454
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1455 elif "GNU.sparse.size" in pax_headers:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1456 # GNU extended sparse format version 0.0.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1457 self._proc_gnusparse_00(next, pax_headers, buf)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1458
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1459 elif pax_headers.get("GNU.sparse.major") == "1" and pax_headers.get("GNU.sparse.minor") == "0":
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1460 # GNU extended sparse format version 1.0.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1461 self._proc_gnusparse_10(next, pax_headers, tarfile)
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1462
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1463 if self.type in (XHDTYPE, SOLARIS_XHDTYPE):
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1464 # Patch the TarInfo object with the extended header info.
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1465 next._apply_pax_info(pax_headers, tarfile.encoding, tarfile.errors)
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1466 next.offset = self.offset
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1467
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1468 if "size" in pax_headers:
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1469 # If the extended header replaces the size field,
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1470 # we need to recalculate the offset where the next
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1471 # header starts.
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1472 offset = next.offset_data
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1473 if next.isreg() or next.type not in SUPPORTED_TYPES:
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1474 offset += next._block(next.size)
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1475 tarfile.offset = offset
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1476
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1477 return next
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1478
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1479 def _proc_gnusparse_00(self, next, pax_headers, buf):
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1480 """Process a GNU tar extended sparse header, version 0.0.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1481 """
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1482 offsets = []
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1483 for match in re.finditer(br"\d+ GNU.sparse.offset=(\d+)\n", buf):
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1484 offsets.append(int(match.group(1)))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1485 numbytes = []
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1486 for match in re.finditer(br"\d+ GNU.sparse.numbytes=(\d+)\n", buf):
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1487 numbytes.append(int(match.group(1)))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1488 next.sparse = list(zip(offsets, numbytes))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1489
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1490 def _proc_gnusparse_01(self, next, pax_headers):
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1491 """Process a GNU tar extended sparse header, version 0.1.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1492 """
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1493 sparse = [int(x) for x in pax_headers["GNU.sparse.map"].split(",")]
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1494 next.sparse = list(zip(sparse[::2], sparse[1::2]))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1495
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1496 def _proc_gnusparse_10(self, next, pax_headers, tarfile):
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1497 """Process a GNU tar extended sparse header, version 1.0.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1498 """
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1499 fields = None
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1500 sparse = []
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1501 buf = tarfile.fileobj.read(BLOCKSIZE)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1502 fields, buf = buf.split(b"\n", 1)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1503 fields = int(fields)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1504 while len(sparse) < fields * 2:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1505 if b"\n" not in buf:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1506 buf += tarfile.fileobj.read(BLOCKSIZE)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1507 number, buf = buf.split(b"\n", 1)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1508 sparse.append(int(number))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1509 next.offset_data = tarfile.fileobj.tell()
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1510 next.sparse = list(zip(sparse[::2], sparse[1::2]))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1511
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1512 def _apply_pax_info(self, pax_headers, encoding, errors):
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1513 """Replace fields with supplemental information from a previous
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1514 pax extended or global header.
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1515 """
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1516 for keyword, value in pax_headers.items():
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1517 if keyword == "GNU.sparse.name":
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1518 setattr(self, "path", value)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1519 elif keyword == "GNU.sparse.size":
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1520 setattr(self, "size", int(value))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1521 elif keyword == "GNU.sparse.realsize":
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1522 setattr(self, "size", int(value))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1523 elif keyword in PAX_FIELDS:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1524 if keyword in PAX_NUMBER_FIELDS:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1525 try:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1526 value = PAX_NUMBER_FIELDS[keyword](value)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1527 except ValueError:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1528 value = 0
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1529 if keyword == "path":
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1530 value = value.rstrip("/")
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1531 setattr(self, keyword, value)
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1532
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1533 self.pax_headers = pax_headers.copy()
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1534
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1535 def _decode_pax_field(self, value, encoding, fallback_encoding, fallback_errors):
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1536 """Decode a single field from a pax record.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1537 """
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1538 try:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1539 return value.decode(encoding, "strict")
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1540 except UnicodeDecodeError:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1541 return value.decode(fallback_encoding, fallback_errors)
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1542
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1543 def _block(self, count):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1544 """Round up a byte count by BLOCKSIZE and return it,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1545 e.g. _block(834) => 1024.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1546 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1547 blocks, remainder = divmod(count, BLOCKSIZE)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1548 if remainder:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1549 blocks += 1
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1550 return blocks * BLOCKSIZE
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1551
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1552 def isreg(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1553 return self.type in REGULAR_TYPES
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1554 def isfile(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1555 return self.isreg()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1556 def isdir(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1557 return self.type == DIRTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1558 def issym(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1559 return self.type == SYMTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1560 def islnk(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1561 return self.type == LNKTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1562 def ischr(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1563 return self.type == CHRTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1564 def isblk(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1565 return self.type == BLKTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1566 def isfifo(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1567 return self.type == FIFOTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1568 def issparse(self):
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1569 return self.sparse is not None
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1570 def isdev(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1571 return self.type in (CHRTYPE, BLKTYPE, FIFOTYPE)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1572 # class TarInfo
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1573
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1574 class TarFile(object):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1575 """The TarFile Class provides an interface to tar archives.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1576 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1577
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1578 debug = 0 # May be set from 0 (no msgs) to 3 (all msgs)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1579
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1580 dereference = False # If true, add content of linked file to the
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1581 # tar file, else the link.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1582
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1583 ignore_zeros = False # If true, skips empty or invalid blocks and
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1584 # continues processing.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1585
57681 46c229742c64 Merged revisions 76780 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57394 diff changeset	1586 errorlevel = 1 # If 0, fatal errors only appear in debug
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1587 # messages (if debug >= 0). If > 0, errors
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1588 # are passed to the caller as exceptions.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1589
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1590 format = DEFAULT_FORMAT # The format to use when creating an archive.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1591
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1592 encoding = ENCODING # Encoding for 8-bit character strings.
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1593
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1594 errors = None # Error handler for unicode conversion.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1595
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1596 tarinfo = TarInfo # The default TarInfo class to use.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1597
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1598 fileobject = ExFileObject # The default ExFileObject class to use.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1599
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1600 def __init__(self, name=None, mode="r", fileobj=None, format=None,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1601 tarinfo=None, dereference=None, ignore_zeros=None, encoding=None,
61278 88466ef1f0cd Issue #8390: tarfile uses surrogateespace as the default error handler Victor Stinner <victor.stinner@haypocalc.com> parents: 61259 diff changeset	1602 errors="surrogateescape", pax_headers=None, debug=None, errorlevel=None):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1603 """Open an (uncompressed) tar archive `name'. `mode' is either 'r' to
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1604 read from an existing archive, 'a' to append data to an existing
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1605 file or 'w' to create a new file overwriting an existing one. `mode'
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1606 defaults to 'r'.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1607 If `fileobj' is given, it is used for reading or writing data. If it
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1608 can be determined, `mode' is overridden by `fileobj's mode.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1609 `fileobj' is not closed, when TarFile is closed.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1610 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1611 if len(mode) > 1 or mode not in "raw":
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1612 raise ValueError("mode must be 'r', 'a' or 'w'")
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1613 self.mode = mode
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1614 self._mode = {"r": "rb", "a": "r+b", "w": "wb"}[mode]
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1615
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1616 if not fileobj:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1617 if self.mode == "a" and not os.path.exists(name):
40893 32c4897b5d46 Merged revisions 53623-53858 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40775 diff changeset	1618 # Create nonexistent files in append mode.
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1619 self.mode = "w"
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1620 self._mode = "wb"
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1621 fileobj = bltn_open(name, self._mode)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1622 self._extfileobj = False
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1623 else:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1624 if name is None and hasattr(fileobj, "name"):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1625 name = fileobj.name
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1626 if hasattr(fileobj, "mode"):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1627 self._mode = fileobj.mode
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1628 self._extfileobj = True
42954 27f0289b6835 Merge the trunk changes in. Breaks socket.ssl for now. Thomas Wouters <thomas@python.org> parents: 42941 diff changeset	1629 self.name = os.path.abspath(name) if name else None
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1630 self.fileobj = fileobj
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1631
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1632 # Init attributes.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1633 if format is not None:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1634 self.format = format
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1635 if tarinfo is not None:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1636 self.tarinfo = tarinfo
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1637 if dereference is not None:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1638 self.dereference = dereference
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1639 if ignore_zeros is not None:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1640 self.ignore_zeros = ignore_zeros
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1641 if encoding is not None:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1642 self.encoding = encoding
61278 88466ef1f0cd Issue #8390: tarfile uses surrogateespace as the default error handler Victor Stinner <victor.stinner@haypocalc.com> parents: 61259 diff changeset	1643 self.errors = errors
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1644
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1645 if pax_headers is not None and self.format == PAX_FORMAT:
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1646 self.pax_headers = pax_headers
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1647 else:
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1648 self.pax_headers = {}
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1649
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1650 if debug is not None:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1651 self.debug = debug
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1652 if errorlevel is not None:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1653 self.errorlevel = errorlevel
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1654
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1655 # Init datastructures.
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1656 self.closed = False
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1657 self.members = [] # list of members as TarInfo objects
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1658 self._loaded = False # flag if all members have been read
44104 ff1766eb2fce Merged revisions 59259-59274 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44098 diff changeset	1659 self.offset = self.fileobj.tell()
ff1766eb2fce Merged revisions 59259-59274 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44098 diff changeset	1660 # current position in the archive file
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1661 self.inodes = {} # dictionary caching the inodes of
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1662 # archive members already added
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1663
57338 d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1664 try:
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1665 if self.mode == "r":
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1666 self.firstmember = None
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1667 self.firstmember = self.next()
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1668
57338 d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1669 if self.mode == "a":
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1670 # Move to the end of the archive,
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1671 # before the first empty block.
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1672 while True:
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1673 self.fileobj.seek(self.offset)
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1674 try:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1675 tarinfo = self.tarinfo.fromtarfile(self)
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1676 self.members.append(tarinfo)
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1677 except EOFHeaderError:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1678 self.fileobj.seek(self.offset)
57338 d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1679 break
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1680 except HeaderError as e:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1681 raise ReadError(str(e))
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1682
57338 d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1683 if self.mode in "aw":
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1684 self._loaded = True
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1685
57338 d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1686 if self.pax_headers:
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1687 buf = self.tarinfo.create_pax_global_header(self.pax_headers.copy())
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1688 self.fileobj.write(buf)
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1689 self.offset += len(buf)
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1690 except:
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1691 if not self._extfileobj:
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1692 self.fileobj.close()
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1693 self.closed = True
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1694 raise
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1695
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1696 #--------------------------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1697 # Below are the classmethods which act as alternate constructors to the
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1698 # TarFile class. The open() method is the only one that is needed for
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1699 # public use; it is the "super"-constructor and is able to select an
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1700 # adequate "sub"-constructor for a particular compression using the mapping
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1701 # from OPEN_METH.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1702 #
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1703 # This concept allows one to subclass TarFile without losing the comfort of
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1704 # the super-constructor. A sub-constructor is registered and made available
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1705 # by adding it to the mapping in OPEN_METH.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1706
34421 3316aa540e68 Use decorators. Guido van Rossum <guido@python.org> parents: 34293 diff changeset	1707 @classmethod
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1708 def open(cls, name=None, mode="r", fileobj=None, bufsize=RECORDSIZE, **kwargs):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1709 """Open a tar archive for reading, writing or appending. Return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1710 an appropriate TarFile class.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1711
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1712 mode:
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1713 'r' or 'r:*' open for reading with transparent compression
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1714 'r:' open for reading exclusively uncompressed
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1715 'r:gz' open for reading with gzip compression
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1716 'r:bz2' open for reading with bzip2 compression
40893 32c4897b5d46 Merged revisions 53623-53858 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40775 diff changeset	1717 'a' or 'a:' open for appending, creating the file if necessary
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1718 'w' or 'w:' open for writing without compression
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1719 'w:gz' open for writing with gzip compression
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1720 'w:bz2' open for writing with bzip2 compression
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1721
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1722 'r\|*' open a stream of tar blocks with transparent compression
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1723 'r\|' open an uncompressed stream of tar blocks for reading
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1724 'r\|gz' open a gzip compressed stream of tar blocks
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1725 'r\|bz2' open a bzip2 compressed stream of tar blocks
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1726 'w\|' open an uncompressed stream for writing
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1727 'w\|gz' open a gzip compressed stream for writing
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1728 'w\|bz2' open a bzip2 compressed stream for writing
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1729 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1730
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1731 if not name and not fileobj:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1732 raise ValueError("nothing to open")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1733
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1734 if mode in ("r", "r:*"):
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1735 # Find out which *open() is appropriate for opening the file.
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1736 for comptype in cls.OPEN_METH:
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1737 func = getattr(cls, cls.OPEN_METH[comptype])
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1738 if fileobj is not None:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1739 saved_pos = fileobj.tell()
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1740 try:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1741 return func(name, "r", fileobj, **kwargs)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1742 except (ReadError, CompressionError) as e:
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1743 if fileobj is not None:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1744 fileobj.seek(saved_pos)
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1745 continue
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1746 raise ReadError("file could not be opened successfully")
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1747
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1748 elif ":" in mode:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1749 filemode, comptype = mode.split(":", 1)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1750 filemode = filemode or "r"
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1751 comptype = comptype or "tar"
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1752
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1753 # Select the *open() function according to
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1754 # given compression.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1755 if comptype in cls.OPEN_METH:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1756 func = getattr(cls, cls.OPEN_METH[comptype])
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1757 else:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1758 raise CompressionError("unknown compression type %r" % comptype)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1759 return func(name, filemode, fileobj, **kwargs)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1760
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1761 elif "\|" in mode:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1762 filemode, comptype = mode.split("\|", 1)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1763 filemode = filemode or "r"
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1764 comptype = comptype or "tar"
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1765
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1766 if filemode not in "rw":
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1767 raise ValueError("mode must be 'r' or 'w'")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1768
64915 0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	1769 stream = _Stream(name, filemode, comptype, fileobj, bufsize)
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	1770 try:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	1771 t = cls(name, filemode, stream, **kwargs)
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	1772 except:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	1773 stream.close()
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	1774 raise
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1775 t._extfileobj = False
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1776 return t
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1777
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1778 elif mode in "aw":
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1779 return cls.taropen(name, mode, fileobj, **kwargs)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1780
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1781 raise ValueError("undiscernible mode")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1782
34421 3316aa540e68 Use decorators. Guido van Rossum <guido@python.org> parents: 34293 diff changeset	1783 @classmethod
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1784 def taropen(cls, name, mode="r", fileobj=None, **kwargs):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1785 """Open uncompressed tar archive name for reading or writing.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1786 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1787 if len(mode) > 1 or mode not in "raw":
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1788 raise ValueError("mode must be 'r', 'a' or 'w'")
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1789 return cls(name, mode, fileobj, **kwargs)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1790
34421 3316aa540e68 Use decorators. Guido van Rossum <guido@python.org> parents: 34293 diff changeset	1791 @classmethod
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1792 def gzopen(cls, name, mode="r", fileobj=None, compresslevel=9, **kwargs):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1793 """Open gzip compressed tar archive name for reading or writing.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1794 Appending is not allowed.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1795 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1796 if len(mode) > 1 or mode not in "rw":
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1797 raise ValueError("mode must be 'r' or 'w'")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1798
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1799 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1800 import gzip
28576 1bd50c28203f Fix test_tarfile failure when gzip is not available Neal Norwitz <nnorwitz@gmail.com> parents: 28222 diff changeset	1801 gzip.GzipFile
1bd50c28203f Fix test_tarfile failure when gzip is not available Neal Norwitz <nnorwitz@gmail.com> parents: 28222 diff changeset	1802 except (ImportError, AttributeError):
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1803 raise CompressionError("gzip module is not available")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1804
65742 64ef61d9ffdc Issue #10233: Close file objects in a timely manner in the tarfile module Antoine Pitrou <solipsis@pitrou.net> parents: 65705 diff changeset	1805 extfileobj = fileobj is not None
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1806 try:
65742 64ef61d9ffdc Issue #10233: Close file objects in a timely manner in the tarfile module Antoine Pitrou <solipsis@pitrou.net> parents: 65705 diff changeset	1807 fileobj = gzip.GzipFile(name, mode + "b", compresslevel, fileobj)
64ef61d9ffdc Issue #10233: Close file objects in a timely manner in the tarfile module Antoine Pitrou <solipsis@pitrou.net> parents: 65705 diff changeset	1808 t = cls.taropen(name, mode, fileobj, **kwargs)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1809 except IOError:
71855 843cd43206b4 Fix #11513: wrong exception handling for the case that GzipFile itself raises an IOError. Georg Brandl <georg@python.org> parents: 71540 diff changeset	1810 if not extfileobj and fileobj is not None:
64908 d560eece0857 Try to fix test_tarfile issues on Windows buildbots by closing file Antoine Pitrou <solipsis@pitrou.net> parents: 62898 diff changeset	1811 fileobj.close()
71855 843cd43206b4 Fix #11513: wrong exception handling for the case that GzipFile itself raises an IOError. Georg Brandl <georg@python.org> parents: 71540 diff changeset	1812 if fileobj is None:
843cd43206b4 Fix #11513: wrong exception handling for the case that GzipFile itself raises an IOError. Georg Brandl <georg@python.org> parents: 71540 diff changeset	1813 raise
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1814 raise ReadError("not a gzip file")
65742 64ef61d9ffdc Issue #10233: Close file objects in a timely manner in the tarfile module Antoine Pitrou <solipsis@pitrou.net> parents: 65705 diff changeset	1815 except:
71855 843cd43206b4 Fix #11513: wrong exception handling for the case that GzipFile itself raises an IOError. Georg Brandl <georg@python.org> parents: 71540 diff changeset	1816 if not extfileobj and fileobj is not None:
65742 64ef61d9ffdc Issue #10233: Close file objects in a timely manner in the tarfile module Antoine Pitrou <solipsis@pitrou.net> parents: 65705 diff changeset	1817 fileobj.close()
64ef61d9ffdc Issue #10233: Close file objects in a timely manner in the tarfile module Antoine Pitrou <solipsis@pitrou.net> parents: 65705 diff changeset	1818 raise
64908 d560eece0857 Try to fix test_tarfile issues on Windows buildbots by closing file Antoine Pitrou <solipsis@pitrou.net> parents: 62898 diff changeset	1819 t._extfileobj = extfileobj
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1820 return t
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1821
34421 3316aa540e68 Use decorators. Guido van Rossum <guido@python.org> parents: 34293 diff changeset	1822 @classmethod
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1823 def bz2open(cls, name, mode="r", fileobj=None, compresslevel=9, **kwargs):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1824 """Open bzip2 compressed tar archive name for reading or writing.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1825 Appending is not allowed.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1826 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1827 if len(mode) > 1 or mode not in "rw":
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1828 raise ValueError("mode must be 'r' or 'w'.")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1829
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1830 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1831 import bz2
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1832 except ImportError:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1833 raise CompressionError("bz2 module is not available")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1834
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1835 if fileobj is not None:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1836 fileobj = _BZ2Proxy(fileobj, mode)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1837 else:
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1838 fileobj = bz2.BZ2File(name, mode, compresslevel=compresslevel)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1839
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1840 try:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1841 t = cls.taropen(name, mode, fileobj, **kwargs)
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1842 except (IOError, EOFError):
64908 d560eece0857 Try to fix test_tarfile issues on Windows buildbots by closing file Antoine Pitrou <solipsis@pitrou.net> parents: 62898 diff changeset	1843 fileobj.close()
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1844 raise ReadError("not a bzip2 file")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1845 t._extfileobj = False
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1846 return t
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1847
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1848 # All *open() methods are registered here.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1849 OPEN_METH = {
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1850 "tar": "taropen", # uncompressed tar
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1851 "gz": "gzopen", # gzip compressed tar
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1852 "bz2": "bz2open" # bzip2 compressed tar
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1853 }
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1854
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1855 #--------------------------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1856 # The public methods which TarFile provides:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1857
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1858 def close(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1859 """Close the TarFile. In write-mode, two finishing zero blocks are
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1860 appended to the archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1861 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1862 if self.closed:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1863 return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1864
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1865 if self.mode in "aw":
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1866 self.fileobj.write(NUL * (BLOCKSIZE * 2))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1867 self.offset += (BLOCKSIZE * 2)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1868 # fill up the end with zero-blocks
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1869 # (like option -b20 for tar does)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1870 blocks, remainder = divmod(self.offset, RECORDSIZE)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1871 if remainder > 0:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1872 self.fileobj.write(NUL * (RECORDSIZE - remainder))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1873
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1874 if not self._extfileobj:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1875 self.fileobj.close()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1876 self.closed = True
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1877
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1878 def getmember(self, name):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1879 """Return a TarInfo object for member `name'. If `name' can not be
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1880 found in the archive, KeyError is raised. If a member occurs more
51776 d06a3ebf306f Merged revisions 69846 via svnmerge from Mark Dickinson <dickinsm@gmail.com> parents: 50132 diff changeset	1881 than once in the archive, its last occurrence is assumed to be the
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1882 most up-to-date version.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1883 """
33507 725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	1884 tarinfo = self._getmember(name)
725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	1885 if tarinfo is None:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1886 raise KeyError("filename %r not found" % name)
33507 725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	1887 return tarinfo
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1888
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1889 def getmembers(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1890 """Return the members of the archive as a list of TarInfo objects. The
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1891 list has the same order as the members in the archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1892 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1893 self._check()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1894 if not self._loaded: # if we want to obtain a list of
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1895 self._load() # all members, we first have to
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1896 # scan the whole archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1897 return self.members
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1898
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1899 def getnames(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1900 """Return the members of the archive as a list of their names. It has
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1901 the same order as the list returned by getmembers().
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1902 """
33507 725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	1903 return [tarinfo.name for tarinfo in self.getmembers()]
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1904
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1905 def gettarinfo(self, name=None, arcname=None, fileobj=None):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1906 """Create a TarInfo object for either the file `name' or the file
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1907 object `fileobj' (using os.fstat on its file descriptor). You can
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1908 modify some of the TarInfo's attributes before you add it using
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1909 addfile(). If given, `arcname' specifies an alternative name for the
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1910 file in the archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1911 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1912 self._check("aw")
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1913
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1914 # When fileobj is given, replace name by
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1915 # fileobj's real name.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1916 if fileobj is not None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1917 name = fileobj.name
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1918
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1919 # Building the name of the member in the archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1920 # Backward slashes are converted to forward slashes,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1921 # Absolute paths are turned to relative paths.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1922 if arcname is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1923 arcname = name
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1924 drv, arcname = os.path.splitdrive(arcname)
55748 9a78e3d90617 Merged revisions 74571 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1925 arcname = arcname.replace(os.sep, "/")
9a78e3d90617 Merged revisions 74571 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1926 arcname = arcname.lstrip("/")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1927
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1928 # Now, fill the TarInfo object with
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1929 # information specific for the file.
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1930 tarinfo = self.tarinfo()
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1931 tarinfo.tarfile = self
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1932
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1933 # Use os.stat or os.lstat, depending on platform
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1934 # and if symlinks shall be resolved.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1935 if fileobj is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1936 if hasattr(os, "lstat") and not self.dereference:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1937 statres = os.lstat(name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1938 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1939 statres = os.stat(name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1940 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1941 statres = os.fstat(fileobj.fileno())
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1942 linkname = ""
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1943
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1944 stmd = statres.st_mode
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1945 if stat.S_ISREG(stmd):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1946 inode = (statres.st_ino, statres.st_dev)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1947 if not self.dereference and statres.st_nlink > 1 and \
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1948 inode in self.inodes and arcname != self.inodes[inode]:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1949 # Is it a hardlink to an already
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1950 # archived file?
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1951 type = LNKTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1952 linkname = self.inodes[inode]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1953 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1954 # The inode is added only if its valid.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1955 # For win32 it is always 0.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1956 type = REGTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1957 if inode[0]:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1958 self.inodes[inode] = arcname
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1959 elif stat.S_ISDIR(stmd):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1960 type = DIRTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1961 elif stat.S_ISFIFO(stmd):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1962 type = FIFOTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1963 elif stat.S_ISLNK(stmd):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1964 type = SYMTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1965 linkname = os.readlink(name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1966 elif stat.S_ISCHR(stmd):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1967 type = CHRTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1968 elif stat.S_ISBLK(stmd):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1969 type = BLKTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1970 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1971 return None
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1972
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1973 # Fill the TarInfo object with all
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1974 # information we can get.
35466 85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1975 tarinfo.name = arcname
85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1976 tarinfo.mode = stmd
85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1977 tarinfo.uid = statres.st_uid
85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1978 tarinfo.gid = statres.st_gid
62043 cea5ef682bdd Merged revisions 81665 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 61092 diff changeset	1979 if type == REGTYPE:
35466 85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1980 tarinfo.size = statres.st_size
33208 04864c39b006 Replace tricky and/or with straight-forward if:else: Martin v. Löwis <martin@v.loewis.de> parents: 33179 diff changeset	1981 else:
40644 1c26505c1de6 Rip out 'long' and 'L'-suffixed integer literals. Guido van Rossum <guido@python.org> parents: 40587 diff changeset	1982 tarinfo.size = 0
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1983 tarinfo.mtime = statres.st_mtime
35466 85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1984 tarinfo.type = type
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1985 tarinfo.linkname = linkname
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1986 if pwd:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1987 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1988 tarinfo.uname = pwd.getpwuid(tarinfo.uid)[0]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1989 except KeyError:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1990 pass
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1991 if grp:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1992 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1993 tarinfo.gname = grp.getgrgid(tarinfo.gid)[0]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1994 except KeyError:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1995 pass
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1996
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1997 if type in (CHRTYPE, BLKTYPE):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1998 if hasattr(os, "major") and hasattr(os, "minor"):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1999 tarinfo.devmajor = os.major(statres.st_rdev)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2000 tarinfo.devminor = os.minor(statres.st_rdev)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2001 return tarinfo
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2002
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2003 def list(self, verbose=True):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2004 """Print a table of contents to sys.stdout. If `verbose' is False, only
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2005 the names of the members are printed. If it is True, an `ls -l'-like
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2006 output is produced.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2007 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2008 self._check()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2009
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2010 for tarinfo in self:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2011 if verbose:
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	2012 print(filemode(tarinfo.mode), end=' ')
4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	2013 print("%s/%s" % (tarinfo.uname or tarinfo.uid,
4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	2014 tarinfo.gname or tarinfo.gid), end=' ')
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2015 if tarinfo.ischr() or tarinfo.isblk():
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	2016 print("%10s" % ("%d,%d" \
4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	2017 % (tarinfo.devmajor, tarinfo.devminor)), end=' ')
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2018 else:
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	2019 print("%10d" % tarinfo.size, end=' ')
4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	2020 print("%d-%02d-%02d %02d:%02d:%02d" \
4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	2021 % time.localtime(tarinfo.mtime)[:6], end=' ')
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2022
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2023 print(tarinfo.name + ("/" if tarinfo.isdir() else ""), end=' ')
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2024
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2025 if verbose:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2026 if tarinfo.issym():
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	2027 print("->", tarinfo.linkname, end=' ')
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2028 if tarinfo.islnk():
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	2029 print("link to", tarinfo.linkname, end=' ')
4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	2030 print()
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2031
67771 1f215fda2383 Issue #11014: Make 'filter' argument in tarfile.Tarfile.add() into a Raymond Hettinger <python@rcn.com> parents: 65879 diff changeset	2032 def add(self, name, arcname=None, recursive=True, exclude=None, *, filter=None):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2033 """Add the file `name' to the archive. `name' may be any type of file
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2034 (directory, fifo, symbolic link, etc.). If given, `arcname'
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2035 specifies an alternative name for the file in the archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2036 Directories are added recursively by default. This can be avoided by
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 42084 diff changeset	2037 setting `recursive' to False. `exclude' is a function that should
55917 51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2038 return True for each filename to be excluded. `filter' is a function
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2039 that expects a TarInfo object argument and returns the changed
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2040 TarInfo object, if it returns None the TarInfo object will be
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2041 excluded from the archive.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2042 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2043 self._check("aw")
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2044
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2045 if arcname is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2046 arcname = name
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2047
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 42084 diff changeset	2048 # Exclude pathnames.
55917 51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2049 if exclude is not None:
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2050 import warnings
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2051 warnings.warn("use the filter argument instead",
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2052 DeprecationWarning, 2)
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2053 if exclude(name):
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2054 self._dbg(2, "tarfile: Excluded %r" % name)
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2055 return
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 42084 diff changeset	2056
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2057 # Skip if somebody tries to archive the archive...
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	2058 if self.name is not None and os.path.abspath(name) == self.name:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2059 self._dbg(2, "tarfile: Skipped %r" % name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2060 return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2061
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2062 self._dbg(1, name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2063
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2064 # Create a TarInfo object from the file.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2065 tarinfo = self.gettarinfo(name, arcname)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2066
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2067 if tarinfo is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2068 self._dbg(1, "tarfile: Unsupported type %r" % name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2069 return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2070
55917 51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2071 # Change or exclude the TarInfo object.
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2072 if filter is not None:
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2073 tarinfo = filter(tarinfo)
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2074 if tarinfo is None:
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2075 self._dbg(2, "tarfile: Excluded %r" % name)
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2076 return
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2077
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2078 # Append the tar header and data to the archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2079 if tarinfo.isreg():
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	2080 f = bltn_open(name, "rb")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2081 self.addfile(tarinfo, f)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2082 f.close()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2083
35466 85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	2084 elif tarinfo.isdir():
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2085 self.addfile(tarinfo)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2086 if recursive:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2087 for f in os.listdir(name):
55917 51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	2088 self.add(os.path.join(name, f), os.path.join(arcname, f),
67771 1f215fda2383 Issue #11014: Make 'filter' argument in tarfile.Tarfile.add() into a Raymond Hettinger <python@rcn.com> parents: 65879 diff changeset	2089 recursive, exclude, filter=filter)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2090
35466 85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	2091 else:
85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	2092 self.addfile(tarinfo)
85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	2093
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2094 def addfile(self, tarinfo, fileobj=None):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2095 """Add the TarInfo object `tarinfo' to the archive. If `fileobj' is
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2096 given, tarinfo.size bytes are read from it and added to the archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2097 You can create TarInfo objects using gettarinfo().
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2098 On Windows platforms, `fileobj' should always be opened with mode
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2099 'rb' to avoid irritation about the file size.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2100 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2101 self._check("aw")
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2102
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	2103 tarinfo = copy.copy(tarinfo)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2104
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	2105 buf = tarinfo.tobuf(self.format, self.encoding, self.errors)
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	2106 self.fileobj.write(buf)
cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	2107 self.offset += len(buf)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2108
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2109 # If there's data to follow, append it.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2110 if fileobj is not None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2111 copyfileobj(fileobj, self.fileobj, tarinfo.size)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2112 blocks, remainder = divmod(tarinfo.size, BLOCKSIZE)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2113 if remainder > 0:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2114 self.fileobj.write(NUL * (BLOCKSIZE - remainder))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2115 blocks += 1
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2116 self.offset += blocks * BLOCKSIZE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2117
33507 725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	2118 self.members.append(tarinfo)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2119
34688 1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2120 def extractall(self, path=".", members=None):
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2121 """Extract all members from the archive to the current working
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2122 directory and set owner, modification time and permissions on
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2123 directories afterwards. `path' specifies a different directory
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2124 to extract to. `members' is optional and must be a subset of the
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2125 list returned by getmembers().
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2126 """
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2127 directories = []
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2128
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2129 if members is None:
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2130 members = self
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2131
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2132 for tarinfo in members:
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2133 if tarinfo.isdir():
45192 142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	2134 # Extract directories with a safe mode.
34688 1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2135 directories.append(tarinfo)
45192 142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	2136 tarinfo = copy.copy(tarinfo)
142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	2137 tarinfo.mode = 0o700
65879 be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2138 # Do not set_attrs directories, as we will do that further down
be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2139 self.extract(tarinfo, path, set_attrs=not tarinfo.isdir())
34688 1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2140
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2141 # Reverse sort directories.
45049 90e86ee3684b Convert some custom sort comparison functions to equivalent key functions. Raymond Hettinger <python@rcn.com> parents: 44535 diff changeset	2142 directories.sort(key=lambda a: a.name)
34688 1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2143 directories.reverse()
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2144
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2145 # Set correct owner, mtime and filemode on directories.
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2146 for tarinfo in directories:
44535 420fe6d7d5f4 Merged revisions 59703-59773 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44104 diff changeset	2147 dirpath = os.path.join(path, tarinfo.name)
34688 1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2148 try:
44535 420fe6d7d5f4 Merged revisions 59703-59773 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44104 diff changeset	2149 self.chown(tarinfo, dirpath)
420fe6d7d5f4 Merged revisions 59703-59773 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44104 diff changeset	2150 self.utime(tarinfo, dirpath)
420fe6d7d5f4 Merged revisions 59703-59773 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44104 diff changeset	2151 self.chmod(tarinfo, dirpath)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40582 diff changeset	2152 except ExtractError as e:
34688 1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2153 if self.errorlevel > 1:
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2154 raise
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2155 else:
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2156 self._dbg(1, "tarfile: %s" % e)
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2157
65879 be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2158 def extract(self, member, path="", set_attrs=True):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2159 """Extract a member from the archive to the current working directory,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2160 using its full name. Its file information is extracted as accurately
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2161 as possible. `member' may be a filename or a TarInfo object. You can
65879 be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2162 specify a different directory using `path'. File attributes (owner,
be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2163 mtime, mode) are set unless `set_attrs' is False.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2164 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2165 self._check("r")
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2166
43577 ace45b361e87 Patch# 1258 by Christian Heimes: kill basestring. Guido van Rossum <guido@python.org> parents: 43020 diff changeset	2167 if isinstance(member, str):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2168 tarinfo = self.getmember(member)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2169 else:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2170 tarinfo = member
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2171
32769 983a91b50f6e SF #857297 and 916874, improve handling of hard links when extracting Neal Norwitz <nnorwitz@gmail.com> parents: 32768 diff changeset	2172 # Prepare the link target for makelink().
983a91b50f6e SF #857297 and 916874, improve handling of hard links when extracting Neal Norwitz <nnorwitz@gmail.com> parents: 32768 diff changeset	2173 if tarinfo.islnk():
983a91b50f6e SF #857297 and 916874, improve handling of hard links when extracting Neal Norwitz <nnorwitz@gmail.com> parents: 32768 diff changeset	2174 tarinfo._link_target = os.path.join(path, tarinfo.linkname)
983a91b50f6e SF #857297 and 916874, improve handling of hard links when extracting Neal Norwitz <nnorwitz@gmail.com> parents: 32768 diff changeset	2175
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2176 try:
65879 be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2177 self._extract_member(tarinfo, os.path.join(path, tarinfo.name),
be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2178 set_attrs=set_attrs)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40582 diff changeset	2179 except EnvironmentError as e:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2180 if self.errorlevel > 0:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2181 raise
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2182 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2183 if e.filename is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2184 self._dbg(1, "tarfile: %s" % e.strerror)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2185 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2186 self._dbg(1, "tarfile: %s %r" % (e.strerror, e.filename))
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40582 diff changeset	2187 except ExtractError as e:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2188 if self.errorlevel > 1:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2189 raise
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2190 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2191 self._dbg(1, "tarfile: %s" % e)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2192
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2193 def extractfile(self, member):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2194 """Extract a member from the archive as a file object. `member' may be
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2195 a filename or a TarInfo object. If `member' is a regular file, a
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2196 file-like object is returned. If `member' is a link, a file-like
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2197 object is constructed from the link's target. If `member' is none of
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2198 the above, None is returned.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2199 The file-like object is read-only and provides the following
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2200 methods: read(), readline(), readlines(), seek() and tell()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2201 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2202 self._check("r")
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2203
43577 ace45b361e87 Patch# 1258 by Christian Heimes: kill basestring. Guido van Rossum <guido@python.org> parents: 43020 diff changeset	2204 if isinstance(member, str):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2205 tarinfo = self.getmember(member)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2206 else:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2207 tarinfo = member
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2208
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2209 if tarinfo.isreg():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2210 return self.fileobject(self, tarinfo)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2211
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2212 elif tarinfo.type not in SUPPORTED_TYPES:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2213 # If a member's type is unknown, it is treated as a
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2214 # regular file.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2215 return self.fileobject(self, tarinfo)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2216
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2217 elif tarinfo.islnk() or tarinfo.issym():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2218 if isinstance(self.fileobj, _Stream):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2219 # A small but ugly workaround for the case that someone tries
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2220 # to extract a (sym)link as a file-object from a non-seekable
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2221 # stream of tar blocks.
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2222 raise StreamError("cannot extract (sym)link as file object")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2223 else:
35276 4adbd331d335 Fix all wrong instances of "it's". Georg Brandl <georg@python.org> parents: 34818 diff changeset	2224 # A (sym)link's file object is its target's file object.
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2225 return self.extractfile(self._find_link_target(tarinfo))
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2226 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2227 # If there's no data associated with the member (directory, chrdev,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2228 # blkdev, etc.), return None instead of a file object.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2229 return None
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2230
65879 be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2231 def _extract_member(self, tarinfo, targetpath, set_attrs=True):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2232 """Extract the TarInfo object tarinfo to a physical
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2233 file called targetpath.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2234 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2235 # Fetch the TarInfo object for the given name
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2236 # and build the destination pathname, replacing
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2237 # forward slashes to platform specific separators.
55748 9a78e3d90617 Merged revisions 74571 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	2238 targetpath = targetpath.rstrip("/")
9a78e3d90617 Merged revisions 74571 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	2239 targetpath = targetpath.replace("/", os.sep)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2240
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2241 # Create all upper directories.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2242 upperdirs = os.path.dirname(targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2243 if upperdirs and not os.path.exists(upperdirs):
45192 142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	2244 # Create directories that are not part of the archive with
142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	2245 # default permissions.
40737 6ebad2a6b990 Merged revisions 53451-53537 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40644 diff changeset	2246 os.makedirs(upperdirs)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2247
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2248 if tarinfo.islnk() or tarinfo.issym():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2249 self._dbg(1, "%s -> %s" % (tarinfo.name, tarinfo.linkname))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2250 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2251 self._dbg(1, tarinfo.name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2252
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2253 if tarinfo.isreg():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2254 self.makefile(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2255 elif tarinfo.isdir():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2256 self.makedir(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2257 elif tarinfo.isfifo():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2258 self.makefifo(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2259 elif tarinfo.ischr() or tarinfo.isblk():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2260 self.makedev(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2261 elif tarinfo.islnk() or tarinfo.issym():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2262 self.makelink(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2263 elif tarinfo.type not in SUPPORTED_TYPES:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2264 self.makeunknown(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2265 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2266 self.makefile(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2267
65879 be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2268 if set_attrs:
be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2269 self.chown(tarinfo, targetpath)
be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2270 if not tarinfo.issym():
be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2271 self.chmod(tarinfo, targetpath)
be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2272 self.utime(tarinfo, targetpath)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2273
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2274 #--------------------------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2275 # Below are the different file methods. They are called via
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2276 # _extract_member() when extract() is called. They can be replaced in a
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2277 # subclass to implement other functionality.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2278
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2279 def makedir(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2280 """Make a directory called targetpath.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2281 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2282 try:
45192 142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	2283 # Use a safe mode for the directory, the real mode is set
142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	2284 # later in _extract_member().
142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	2285 os.mkdir(targetpath, 0o700)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40582 diff changeset	2286 except EnvironmentError as e:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2287 if e.errno != errno.EEXIST:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2288 raise
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2289
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2290 def makefile(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2291 """Make a file called targetpath.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2292 """
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2293 source = self.fileobj
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2294 source.seek(tarinfo.offset_data)
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	2295 target = bltn_open(targetpath, "wb")
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2296 if tarinfo.sparse is not None:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2297 for offset, size in tarinfo.sparse:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2298 target.seek(offset)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2299 copyfileobj(source, target, size)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2300 else:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2301 copyfileobj(source, target, tarinfo.size)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2302 target.seek(tarinfo.size)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2303 target.truncate()
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2304 target.close()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2305
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2306 def makeunknown(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2307 """Make a file from a TarInfo object with an unknown type
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2308 at targetpath.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2309 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2310 self.makefile(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2311 self._dbg(1, "tarfile: Unknown file type %r, " \
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2312 "extracted as regular file." % tarinfo.type)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2313
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2314 def makefifo(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2315 """Make a fifo called targetpath.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2316 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2317 if hasattr(os, "mkfifo"):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2318 os.mkfifo(targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2319 else:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2320 raise ExtractError("fifo not supported by system")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2321
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2322 def makedev(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2323 """Make a character or block device called targetpath.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2324 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2325 if not hasattr(os, "mknod") or not hasattr(os, "makedev"):
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2326 raise ExtractError("special devices not supported by system")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2327
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2328 mode = tarinfo.mode
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2329 if tarinfo.isblk():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2330 mode \|= stat.S_IFBLK
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2331 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2332 mode \|= stat.S_IFCHR
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2333
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2334 os.mknod(targetpath, mode,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2335 os.makedev(tarinfo.devmajor, tarinfo.devminor))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2336
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2337 def makelink(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2338 """Make a (symbolic) link called targetpath. If it cannot be created
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2339 (platform limitation), we try to make a copy of the referenced file
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2340 instead of a link.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2341 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2342 try:
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2343 # For systems that support symbolic and hard links.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2344 if tarinfo.issym():
55748 9a78e3d90617 Merged revisions 74571 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	2345 os.symlink(tarinfo.linkname, targetpath)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2346 else:
32769 983a91b50f6e SF #857297 and 916874, improve handling of hard links when extracting Neal Norwitz <nnorwitz@gmail.com> parents: 32768 diff changeset	2347 # See extract().
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2348 if os.path.exists(tarinfo._link_target):
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2349 os.link(tarinfo._link_target, targetpath)
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2350 else:
64929 260dc36c2c86 Fix a line that got hacked up by r82659. Brian Curtin <brian.curtin@gmail.com> parents: 64915 diff changeset	2351 self._extract_member(self._find_link_target(tarinfo),
260dc36c2c86 Fix a line that got hacked up by r82659. Brian Curtin <brian.curtin@gmail.com> parents: 64915 diff changeset	2352 targetpath)
62898 5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	2353 except symlink_exception:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2354 try:
62898 5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	2355 self._extract_member(self._find_link_target(tarinfo),
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	2356 targetpath)
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2357 except KeyError:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2358 raise ExtractError("unable to resolve link inside archive")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2359
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2360 def chown(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2361 """Set owner of targetpath according to tarinfo.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2362 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2363 if pwd and hasattr(os, "geteuid") and os.geteuid() == 0:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2364 # We have to be root to do so.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2365 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2366 g = grp.getgrnam(tarinfo.gname)[2]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2367 except KeyError:
72272 2bc122347351 Issue #12841: Fix tarfile extraction of non-existent uids/gids. Lars Gustäbel <lars@gustaebel.de> parents: 71855 diff changeset	2368 g = tarinfo.gid
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2369 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2370 u = pwd.getpwnam(tarinfo.uname)[2]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2371 except KeyError:
72272 2bc122347351 Issue #12841: Fix tarfile extraction of non-existent uids/gids. Lars Gustäbel <lars@gustaebel.de> parents: 71855 diff changeset	2372 u = tarinfo.uid
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2373 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2374 if tarinfo.issym() and hasattr(os, "lchown"):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2375 os.lchown(targetpath, u, g)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2376 else:
27972 53c802927202 OS/2 has no concept of file ownership, like DOS & MS Windows version Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 27507 diff changeset	2377 if sys.platform != "os2emx":
53c802927202 OS/2 has no concept of file ownership, like DOS & MS Windows version Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 27507 diff changeset	2378 os.chown(targetpath, u, g)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40582 diff changeset	2379 except EnvironmentError as e:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2380 raise ExtractError("could not change owner")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2381
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2382 def chmod(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2383 """Set file permissions of targetpath according to tarinfo.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2384 """
28214 d4b262fbd04a Test that os.utime and os.chmod actually exist before using them. Jack Jansen <jack.jansen@cwi.nl> parents: 27972 diff changeset	2385 if hasattr(os, 'chmod'):
d4b262fbd04a Test that os.utime and os.chmod actually exist before using them. Jack Jansen <jack.jansen@cwi.nl> parents: 27972 diff changeset	2386 try:
d4b262fbd04a Test that os.utime and os.chmod actually exist before using them. Jack Jansen <jack.jansen@cwi.nl> parents: 27972 diff changeset	2387 os.chmod(targetpath, tarinfo.mode)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40582 diff changeset	2388 except EnvironmentError as e:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2389 raise ExtractError("could not change mode")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2390
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2391 def utime(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2392 """Set modification time of targetpath according to tarinfo.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2393 """
28214 d4b262fbd04a Test that os.utime and os.chmod actually exist before using them. Jack Jansen <jack.jansen@cwi.nl> parents: 27972 diff changeset	2394 if not hasattr(os, 'utime'):
28222 eca697e06a07 Somebody must not have run the test before checking this in -- it had Tim Peters <tim.peters@gmail.com> parents: 28217 diff changeset	2395 return
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2396 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2397 os.utime(targetpath, (tarinfo.mtime, tarinfo.mtime))
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40582 diff changeset	2398 except EnvironmentError as e:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2399 raise ExtractError("could not change modification time")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2400
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2401 #--------------------------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2402 def next(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2403 """Return the next member of the archive as a TarInfo object, when
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2404 TarFile is opened for reading. Return None if there is no more
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2405 available.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2406 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2407 self._check("ra")
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2408 if self.firstmember is not None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2409 m = self.firstmember
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2410 self.firstmember = None
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2411 return m
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2412
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2413 # Read the next block.
32604 08d90460ab4d [Patch 988444] Andrew M. Kuchling <amk@amk.ca> parents: 31314 diff changeset	2414 self.fileobj.seek(self.offset)
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2415 tarinfo = None
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2416 while True:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2417 try:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2418 tarinfo = self.tarinfo.fromtarfile(self)
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2419 except EOFHeaderError as e:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2420 if self.ignore_zeros:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2421 self._dbg(2, "0x%X: %s" % (self.offset, e))
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2422 self.offset += BLOCKSIZE
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2423 continue
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2424 except InvalidHeaderError as e:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2425 if self.ignore_zeros:
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	2426 self._dbg(2, "0x%X: %s" % (self.offset, e))
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2427 self.offset += BLOCKSIZE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2428 continue
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2429 elif self.offset == 0:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2430 raise ReadError(str(e))
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2431 except EmptyHeaderError:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2432 if self.offset == 0:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2433 raise ReadError("empty file")
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2434 except TruncatedHeaderError as e:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2435 if self.offset == 0:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2436 raise ReadError(str(e))
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2437 except SubsequentHeaderError as e:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2438 raise ReadError(str(e))
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2439 break
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2440
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2441 if tarinfo is not None:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2442 self.members.append(tarinfo)
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2443 else:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2444 self._loaded = True
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2445
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2446 return tarinfo
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2447
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2448 #--------------------------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2449 # Little helper methods:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2450
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2451 def _getmember(self, name, tarinfo=None, normalize=False):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2452 """Find an archive member by name from bottom to top.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2453 If tarinfo is given, it is used as the starting point.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2454 """
33507 725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	2455 # Ensure that all members have been loaded.
725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	2456 members = self.getmembers()
725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	2457
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2458 # Limit the member search list up to tarinfo.
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2459 if tarinfo is not None:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2460 members = members[:members.index(tarinfo)]
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2461
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2462 if normalize:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2463 name = os.path.normpath(name)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2464
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2465 for member in reversed(members):
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2466 if normalize:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2467 member_name = os.path.normpath(member.name)
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2468 else:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2469 member_name = member.name
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2470
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2471 if name == member_name:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2472 return member
32604 08d90460ab4d [Patch 988444] Andrew M. Kuchling <amk@amk.ca> parents: 31314 diff changeset	2473
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2474 def _load(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2475 """Read through the entire archive file and look for readable
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2476 members.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2477 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2478 while True:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2479 tarinfo = self.next()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2480 if tarinfo is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2481 break
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2482 self._loaded = True
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2483
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2484 def _check(self, mode=None):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2485 """Check if TarFile is still open, and if the operation's mode
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2486 corresponds to TarFile's mode.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2487 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2488 if self.closed:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2489 raise IOError("%s is closed" % self.__class__.__name__)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2490 if mode is not None and self.mode not in mode:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2491 raise IOError("bad operation for mode %r" % self.mode)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2492
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2493 def _find_link_target(self, tarinfo):
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2494 """Find the target member of a symlink or hardlink member in the
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2495 archive.
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2496 """
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2497 if tarinfo.issym():
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2498 # Always search the entire archive.
76538 0adf4fd8df83 Issue #14160: TarFile.extractfile() failed to resolve symbolic links Lars Gustäbel <lars@gustaebel.de> parents: 74277 diff changeset	2499 linkname = "/".join(filter(None, (os.path.dirname(tarinfo.name), tarinfo.linkname)))
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2500 limit = None
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2501 else:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2502 # Search the archive before the link, because a hard link is
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2503 # just a reference to an already archived file.
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2504 linkname = tarinfo.linkname
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2505 limit = tarinfo
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2506
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2507 member = self._getmember(linkname, tarinfo=limit, normalize=True)
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2508 if member is None:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2509 raise KeyError("linkname %r not found" % linkname)
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2510 return member
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2511
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2512 def __iter__(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2513 """Provide an iterator object.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2514 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2515 if self._loaded:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2516 return iter(self.members)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2517 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2518 return TarIter(self)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2519
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2520 def _dbg(self, level, msg):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2521 """Write debugging output to sys.stderr.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2522 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2523 if level <= self.debug:
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	2524 print(msg, file=sys.stderr)
59304 c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2525
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2526 def __enter__(self):
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2527 self._check()
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2528 return self
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2529
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2530 def __exit__(self, type, value, traceback):
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2531 if type is None:
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2532 self.close()
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2533 else:
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2534 # An exception occurred. We must not call close() because
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2535 # it would try to write end-of-archive blocks and padding.
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2536 if not self._extfileobj:
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2537 self.fileobj.close()
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2538 self.closed = True
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2539 # class TarFile
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2540
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2541 class TarIter:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2542 """Iterator Class.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2543
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2544 for tarinfo in TarFile(...):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2545 suite...
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2546 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2547
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2548 def __init__(self, tarfile):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2549 """Construct a TarIter object.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2550 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2551 self.tarfile = tarfile
34680 481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2552 self.index = 0
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2553 def __iter__(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2554 """Return iterator object.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2555 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2556 return self
41501 0c5019afc55b PEP 3114: rename .next() to .__next__() and add next() builtin. Georg Brandl <georg@python.org> parents: 40893 diff changeset	2557 def __next__(self):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2558 """Return the next item using TarFile's next() method.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2559 When all members have been read, set TarFile as _loaded.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2560 """
34680 481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2561 # Fix for SF #1100429: Under rare circumstances it can
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2562 # happen that getmembers() is called during iteration,
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2563 # which will cause TarIter to stop prematurely.
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2564 if not self.tarfile._loaded:
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2565 tarinfo = self.tarfile.next()
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2566 if not tarinfo:
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2567 self.tarfile._loaded = True
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2568 raise StopIteration
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2569 else:
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2570 try:
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2571 tarinfo = self.tarfile.members[self.index]
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2572 except IndexError:
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2573 raise StopIteration
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2574 self.index += 1
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2575 return tarinfo
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2576
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2577 #--------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2578 # exported functions
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2579 #--------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2580 def is_tarfile(name):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2581 """Return True if name points to a tar archive that we
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2582 are able to handle, else return False.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2583 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2584 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2585 t = open(name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2586 t.close()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2587 return True
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2588 except TarError:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2589 return False
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2590
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	2591 bltn_open = open
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2592 open = TarFile.open

Mercurial > cpython-lru_cache-weakref

annotate Lib/tarfile.py @ 78069:a8aa77458e12 3.2